<div dir="ltr"><div>Hi,<br><br></div>Sorry, got distracted and wanted to make some images available to help with the discussion as we're talking UI.<br><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 5 September 2017 at 19:43, Clark Boylan <span dir="ltr"><<a href="mailto:cboylan@sapwetik.org" target="_blank">cboylan@sapwetik.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail-m_2173680354160285786HOEnZb"><div class="gmail-m_2173680354160285786h5">On Tue, Sep 5, 2017, at 09:25 AM, Darragh Bailey wrote:<br>
> Hi,<br></div></div></blockquote><div> </div><div><snipped> </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>
> 1) Project status view:<br>
><br>
> The main status page can be useful for those supporting zuul in an<br>
> environment to have a quick overview of everything, however feedback<br>
> we've<br>
> received locally suggests most developers are frequently only interested<br>
> in<br>
> a single project and find this to be overwhelming This suggests there<br>
> should be a per-project status view, showing only the pipelines and<br>
> changes<br>
> of interest for a single project, along with any CRD that are linked.<br>
><br>
> Possibly under an endpoint of "/<project>/" as the default view showing<br>
> any<br>
> pipelines relevant to the project along with any changes going through<br>
> them.<br>
><br>
<br>
</span>This is already possible via the status page filter right? The<br>
difference would be that the filter is cookie based and not path based.<br>
Thinking about it, both provide nice features (with cookie based filter<br>
I don't have to remember links I just always get things filtered the way<br>
I want, with url it is easier to link to what you are seeing).<br></blockquote><div><br></div><div>Yes, but at the same time it's not necessarily a great view for all cases<br></div><div><br></div><div>A filter benefits in showing that there are many more items currently being hidden from view, but this can lead to the view being very cluttered when what the user wanted was just to see the state of the jobs for their change only.<br></div><div><br></div><div>Examples:</div><div><a href="https://pasteboard.co/GL5fmnU.png">https://pasteboard.co/GL5fmnU.png</a> <-- filter shows change with dependencies<br></div><div></div><div><a href="https://pasteboard.co/GL5qNmA.png">https://pasteboard.co/GL5qNmA.png</a> <-- busy status view<br></div><div><br></div><div>I'd prefer to revert back to keeping the filter as a way for users to adjust what is displayed, but to decide on what subset of data is going to be made available so they are only filtering a smaller set rather than the entire contents.<br></div><div><br></div>I think a path based approach allows <br></div><div class="gmail_quote"><div><br></div><div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span>
> ----------------------------<br>
><br>
> 2) Project build history view:<br>
><br>
> Returning links at the end of a run in a comment to the raw log files as<br>
> the only build history is somewhat jarring for people used to the<br>
> Travis/Drone/Circle CI services available, there's the expectation that<br>
> it<br>
> should be easily to see the previous runs, seeing whether they passed,<br>
> which jobs within a set failed, and to display the detailed log when<br>
> desired. Providing a reasonable UI is becoming a pre-requisite for<br>
> adoption, no matter how good the functionality, it can be difficult to<br>
> get<br>
> acceptance if the "form" is deemed limiting (non all follow this, but it<br>
> does seem that some of the most vocal objectors can fall into this area).<br>
><br>
> Users can be quick to blame the infrastructure when a chance working on<br>
> their system fails in CI and seeing previous runs is non trivial. Making<br>
> it easy to see that previous changes were working (or failing) can make<br>
> it<br>
> more likely that they will look closer at their own change for the cause<br>
> of<br>
> the failure. It also tends to be easier to understand a failing job, when<br>
> you can easily view the output from a passing one.<br>
<br>
</span>Yup, this is the reason the openstack health dashboard,<br>
<a href="http://status.openstack.org/openstack-health/#/" rel="noreferrer" target="_blank">http://status.openstack.org/op<wbr>enstack-health/#/</a>, exists. Being able to<br>
see the bigger picture and status of a project is important. It also<br>
provides rss feeds so you don't have to actively look at a screen to get<br>
notifications which is nice. A lot of the health dashboard features are<br>
probably fairly openstack specific, like the integration with<br>
elastic-recheck and subunit, but my hunch is that we can probably make a<br>
health dashboard that falls back gracefully to a base set of features<br>
that a "normal" zuul v3 installation would support.<br>
<br>
Rather than start from scratch maybe it makes sense to build on this<br>
health dashboard?<br></blockquote><div><br></div><div>I find the current view very individual job based rather than change based.</div><div><br></div><div>I'm not sure that requiring another service to be in place to display the historical information so users can see how previous build sets as well as ones currently running are performing, unless you meant to try and lift that and add it to the zuul webapp?</div><div><br></div><div>Perhaps openstack health would be better focused on providing statistics and analysis enhancements to zuul? Time most jobs run at, details on failures, which job out of the build set is the most common failing job, timing comparisons over time, etc.<br></div><div><br></div><div>From a raw display of information can I find the last number of runs and see both those currently active and be able to see the x most recent runs and their status, that seems like something that should sit within zuul. It seems annoying that as soon as the build set of jobs for a particular change is complete, they disappear.<br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span>
> Zuul could provide a build history on each set of jobs that has been<br>
> triggered for each project:<br>
> Display the overall result (pass/fail) along side information linking to<br>
> the change triggered, may also include note identifying the triggering<br>
> reason such as change created, ref updated, comment with matching message<br>
> added, etc.<br>
> Clicking on this info should provide extended information on all of the<br>
> individual jobs run, their status and link to the log file. Could always<br>
> load the load the log file from the remote file server and display inline<br>
> using JS and display with syntax highlighting as needed as a future<br>
> enhancement.<br>
<br>
</span>I must be weird, I really dislike the super fancy log setups that<br>
Travis, Shippable, etc use because it is hard to manipulate them to find<br>
what I need. Making things pretty is fine but lets be sure to make it<br>
easy to get the raw data too.<br></blockquote><div><br></div><div>I agree, but our users sometimes disagree, I'm of the opinion that as long as this is something that can be added as an optional enhancement and let whoever wishes to use it, live with it, and don't require it to be applied to all, good enough for me.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><div class="gmail-m_2173680354160285786h5">
><br>
> Possibly under an endpoint of "/<project>/history" (or some other name<br>
> besides history)<br>
><br>
> ----------------------------<br>
><br>
> 3) Change view<br>
><br>
> Users of tools such as GitHub are expecting a link to appear on the PR<br>
> status checks that can then be followed to see the state of the job(s)<br>
> currently executing. Currently this can be enabled through setting of a<br>
> config option "status_url_with_change" in the zuul section. This can also<br>
> be used with Gerrit making it easy to quickly see a single change.<br>
><br>
> Being able to focus on just the jobs associated with the single change<br>
> you<br>
> are interested in, allows developers to focus on the most important<br>
> information relevant to them. There is an expectation with users of other<br>
> CI tools with GitHub that this should be possible and straight forward as<br>
> well.<br>
><br>
> With the existing status view, it is possible to set the url posted back<br>
> to<br>
> Gerrit/GitHub/(other?) such that following it will automatically filter<br>
> the<br>
> view. However the current behaviour is to split everything in the filter<br>
> on<br>
> ',' and show anything matching any of the fields. It is possible to limit<br>
> this by changing the link posted back to use an underscore instead of a<br>
> comma to separate <change>_<interation> (gerrit) or <pr>_<sha1> (github)<br>
> and the filter will work a little better, however as it's doing partial<br>
> matching it's possible for a in gerrit such as 23_7 to match a PR 23 if<br>
> the<br>
> SHA1 starts with 7 for some GitHub project. Low risk, but with sufficient<br>
> changes and sufficient installations it could happen. As the Gerrit<br>
> change<br>
> numbers increase, the risk of this colliding with GitHub PRs decreases.<br>
<br>
</div></div>I'd be wary of forcing users to do a modification to the change atom<br>
itself in order to get it to filter properly. Would it make more sense<br>
to split rules on some other character/string? That way I can take what<br>
gerrit says is the unique identifier for my change, eg '1234,5' and<br>
paste that in directly without needing to change anything. And only care<br>
about the special stuff if I want more than one filter item.<br></blockquote><div><br></div><div>I'm not sure I quite follow you. What I discovered was the filtering was working on div tag 'id', and I noticed changes would have a div id of <change>_<interation> (gerrit) or <pr>_<sha1> (github), which meant you could substitute the usual comma ',' with an underscore and the filter would treat the entire string as the item to search for.</div><div><br></div><div>In the case of the gerrit identifier '1234,5' you can currently enter '1234_5' into the filter in openstack's zuul and it will search for div tags with id starting with '1234_5'. My change wasn't to require user behaviour change, but just to take advantage of the existing zuul filter behaviour by making sure the link posted back by zuul used an underscore instead of a comma to help limit the results returned when you followed the link.<br></div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span>
><br>
> There still ends up there being an amount of additional unnecessary info<br>
> appearing in this in any case, as any projects with changes not matched,<br>
> still have the project name appear as a visual clue that there changes in<br>
> the pipeline but are hidden by the current filter.<br>
><br>
> It would seem more desirable to be able to show a single change by<br>
> itself,<br>
> with dependent changes indicated and expandable to appear as well.<br>
> This could start with something that only works when the change is<br>
> running,<br>
> and if historical data is supported it could then display the archived<br>
> information subsequently, minus any cross-repo dependency information<br>
> unless someone feels that would be useful.<br>
><br>
> Possibly under an endpoint of /<project>/change/<item><br>
><br>
> As the same change may appear under multiple pipelines and also may have<br>
> multiple runs per pipeline, this would probably need some thought as to<br>
> how<br>
> it would work if viewing history was possible before deciding.<br>
><br>
> ----------------------------<br>
><br>
> Are these of any interest?<br>
><br>
> I'm no UI expert, so I'm just identifying some of the pieces that we hear<br>
> feedback over and hoping that if I can help get the necessary information<br>
> exposed and appearing at the right places someone else might know how to<br>
> make it look nice..<br>
<br>
</span>Yup, I think this is good feedback.<br>
<br>
I do think we need to be careful to avoid the major problems that<br>
Jenkins has/had around reporting. Storage and presentation of logs and<br>
historical data was one of the first places Jenkins fell over for us<br>
which is why we ended up doing external log storage and indexing.<br>
<br>
Clark<br></blockquote><div><br></div><div>I think most of Jenkins problems was because it was trying to perform build analysis on all history from the get go. I don't think that needs to live in zuul, openstack health is probably a good place for all analysis and any statistical data extraction to live.</div><div><br></div><div>It does seem like it would make sense to have the zuul UI provide a way to look at previous runs and, really that should only involve maintaining the change information (triggering event?), list of jobs executed, final result and links to the log files from the jobs. Making it easy to browse the change results of what zuul just ran, and continue to be able to see your change jobs even once they are finished would be highly useful to help users self investigate and solve their own issues.<br></div><div><br></div><div><br></div><div>Bit more on the change level view:<br></div><div><br></div><div><div>At a change level it seems likely the most useful information for an
individuals change is what is in the square box in the following image:</div><div><a href="https://pasteboard.co/GL5hpsr.png">https://pasteboard.co/GL5hpsr.png</a></div><div><br></div><div>I
think it's a case of deciding for a given use case what makes the most
sense. For individuals submitting changes, what is the simplest view
that provides the most important information that they need</div><div><br></div><div>Providing
a way for only build set of jobs related to the change the person
clicked through from to be show by default in a particular view
without all the other information on the page would be a good starting
point for many users that just want to see how their change is
progressing. That is so it can be provided as a link to be posted back
on individual changes.</div><div><br></div><div>Using something to help
show if the referenced change/job is part of a dependency chain would be
a nice visual enhancement, but that seems like an extra rather than a
requirement.<br></div><div><br></div><div>From there, the view can have options to switch to:</div><div>* all changes in the same pipeline for the current project</div><div>* all changes in all pipelines for the current project<br></div><div>* current status view</div><div><br></div><div>In many ways this will be a configurable option, because it's all about what is posted back to the change/pr:</div><div>* a link to the main status page</div><div>* a link to a project status page using a path url to tell the zuul UI to only show that project</div><div>* a link to a change status page using a path url to only show that change</div><div><br></div><div>Adding the filter placeholder to the first two would also work, but as mentioned above I'd prefer if that was used as a primarily interactive mechanism for users viewing instead of being hijacked in links published by zuul back to the code review software.<br></div><div><br></div>I
think if we focus on the units that provide useful information, and how
to navigate between them, then make sure it's possible to provide
whichever is of most use to be used directly from the change/PR and
installations can decide on which starting point is of the most use to
them</div><div><br></div></div>-- <br><div class="gmail-m_2173680354160285786gmail_signature">Darragh Bailey<br>"Nothing is foolproof to a sufficiently talented fool"</div>
</div></div></div></div>