Open Stack

Hi,

Sorry, got distracted and wanted to make some images available to help with
the discussion as we're talking UI.

On 5 September 2017 at 19:43, Clark Boylan <cboylan at sapwetik.org> wrote:

> On Tue, Sep 5, 2017, at 09:25 AM, Darragh Bailey wrote:
> > Hi,
>

<snipped>

> 1) Project status view:
> >
> > The main status page can be useful for those supporting zuul in an
> > environment to have a quick overview of everything, however feedback
> > we've
> > received locally suggests most developers are frequently only interested
> > in
> > a single project and find this to be overwhelming This suggests there
> > should be a per-project status view, showing only the pipelines and
> > changes
> > of interest for a single project, along with any CRD that are linked.
> >
> > Possibly under an endpoint of "/<project>/" as the default view showing
> > any
> > pipelines relevant to the project along with any changes going through
> > them.
> >
>
> This is already possible via the status page filter right? The
> difference would be that the filter is cookie based and not path based.
> Thinking about it, both provide nice features (with cookie based filter
> I don't have to remember links I just always get things filtered the way
> I want, with url it is easier to link to what you are seeing).
>

Yes, but at the same time it's not necessarily a great view for all cases

A filter benefits in showing that there are many more items currently being
hidden from view, but this can lead to the view being very cluttered when
what the user wanted was just to see the state of the jobs for their change
only.

Examples:
https://pasteboard.co/GL5fmnU.png <-- filter shows change with dependencies
https://pasteboard.co/GL5qNmA.png <-- busy status view

I'd prefer to revert back to keeping the filter as a way for users to
adjust what is displayed, but to decide on what subset of data is going to
be made available so they are only filtering a smaller set rather than the
entire contents.

I think a path based approach allows



> > ----------------------------
> >
> > 2) Project build history view:
> >
> > Returning links at the end of a run in a comment to the raw log files as
> > the only build history is somewhat jarring for people used to the
> > Travis/Drone/Circle CI services available, there's the expectation that
> > it
> > should be easily to see the previous runs, seeing whether they passed,
> > which jobs within a set failed, and to display the detailed log when
> > desired. Providing a reasonable UI is becoming a pre-requisite for
> > adoption, no matter how good the functionality, it can be difficult to
> > get
> > acceptance if the "form" is deemed limiting (non all follow this, but it
> > does seem that some of the most vocal objectors can fall into this area).
> >
> > Users can be quick to blame the infrastructure when a chance working on
> > their system fails in CI and seeing previous runs is non trivial.  Making
> > it easy to see that previous changes were working (or failing) can make
> > it
> > more likely that they will look closer at their own change for the cause
> > of
> > the failure. It also tends to be easier to understand a failing job, when
> > you can easily view the output from a passing one.
>
> Yup, this is the reason the openstack health dashboard,
> http://status.openstack.org/openstack-health/#/, exists. Being able to
> see the bigger picture and status of a project is important. It also
> provides rss feeds so you don't have to actively look at a screen to get
> notifications which is nice. A lot of the health dashboard features are
> probably fairly openstack specific, like the integration with
> elastic-recheck and subunit, but my hunch is that we can probably make a
> health dashboard that falls back gracefully to a base set of features
> that a "normal" zuul v3 installation would support.
>
> Rather than start from scratch maybe it makes sense to build on this
> health dashboard?
>

I find the current view very individual job based rather than change based.

I'm not sure that requiring another service to be in place to display the
historical information so users can see how previous build sets as well as
ones currently running are performing, unless you meant to try and lift
that and add it to the zuul webapp?

Perhaps openstack health would be better focused on providing statistics
and analysis enhancements to zuul? Time most jobs run at, details on
failures, which job out of the build set is the most common failing job,
timing comparisons over time, etc.

>From a raw display of information can I find the last number of runs and
see both those currently active and be able to see the x most recent runs
and their status, that seems like something that should sit within zuul. It
seems annoying that as soon as the build set of jobs for a particular
change is complete, they disappear.



> > Zuul could provide a build history on each set of jobs that has been
> > triggered for each project:
> > Display the overall result (pass/fail) along side information linking to
> > the change triggered, may also include note identifying the triggering
> > reason such as change created, ref updated, comment with matching message
> > added, etc.
> > Clicking on this info should provide extended information on all of the
> > individual jobs run, their status and link to the log file. Could always
> > load the load the log file from the remote file server and display inline
> > using JS and display with syntax highlighting as needed as a future
> > enhancement.
>
> I must be weird, I really dislike the super fancy log setups that
> Travis, Shippable, etc use because it is hard to manipulate them to find
> what I need. Making things pretty is fine but lets be sure to make it
> easy to get the raw data too.
>

I agree, but our users sometimes disagree, I'm of the opinion that as long
as this is something that can be added as an optional enhancement and let
whoever wishes to use it, live with it, and don't require it to be applied
to all, good enough for me.


> >
> > Possibly under an endpoint of "/<project>/history" (or some other name
> > besides history)
> >
> > ----------------------------
> >
> > 3) Change view
> >
> > Users of tools such as GitHub are expecting a link to appear on the PR
> > status checks that can then be followed to see the state of the job(s)
> > currently executing. Currently this can be enabled through setting of a
> > config option "status_url_with_change" in the zuul section. This can also
> > be used with Gerrit making it easy to quickly see a single change.
> >
> > Being able to focus on just the jobs associated with the single change
> > you
> > are interested in, allows developers to focus on the most important
> > information relevant to them. There is an expectation with users of other
> > CI tools with GitHub that this should be possible and straight forward as
> > well.
> >
> > With the existing status view, it is possible to set the url posted back
> > to
> > Gerrit/GitHub/(other?) such that following it will automatically filter
> > the
> > view. However the current behaviour is to split everything in the filter
> > on
> > ',' and show anything matching any of the fields. It is possible to limit
> > this by changing the link posted back to use an underscore instead of a
> > comma to separate <change>_<interation> (gerrit) or <pr>_<sha1> (github)
> > and the filter will work a little better, however as it's doing partial
> > matching it's possible for a in gerrit such as 23_7 to match a PR 23 if
> > the
> > SHA1 starts with 7 for some GitHub project. Low risk, but with sufficient
> > changes and sufficient installations it could happen. As the Gerrit
> > change
> > numbers increase, the risk of this colliding with GitHub PRs decreases.
>
> I'd be wary of forcing users to do a modification to the change atom
> itself in order to get it to filter properly. Would it make more sense
> to split rules on some other character/string? That way I can take what
> gerrit says is the unique identifier for my change, eg '1234,5' and
> paste that in directly without needing to change anything. And only care
> about the special stuff if I want more than one filter item.
>

I'm not sure I quite follow you. What I discovered was the filtering was
working on div tag 'id', and I noticed changes would have a div id of
<change>_<interation> (gerrit) or <pr>_<sha1> (github), which meant you
could substitute the usual comma ',' with an underscore and the filter
would treat the entire string as the item to search for.

In the case of the gerrit identifier '1234,5' you can currently enter
'1234_5' into the filter in openstack's zuul and it will search for div
tags with id starting with '1234_5'. My change wasn't to require user
behaviour change, but just to take advantage of the existing zuul filter
behaviour by making sure the link posted back by zuul used an underscore
instead of a comma to help limit the results returned when you followed the
link.



> >
> > There still ends up there being an amount of additional unnecessary info
> > appearing in this in any case, as any projects with changes not matched,
> > still have the project name appear as a visual clue that there changes in
> > the pipeline but are hidden by the current filter.
> >
> > It would seem more desirable to be able to show a single change by
> > itself,
> > with dependent changes indicated and expandable to appear as well.
> > This could start with something that only works when the change is
> > running,
> > and if historical data is supported it could then display the archived
> > information subsequently, minus any cross-repo dependency information
> > unless someone feels that would be useful.
> >
> > Possibly under an endpoint of /<project>/change/<item>
> >
> > As the same change may appear under multiple pipelines and also may have
> > multiple runs per pipeline, this would probably need some thought as to
> > how
> > it would work if viewing history was possible before deciding.
> >
> > ----------------------------
> >
> > Are these of any interest?
> >
> > I'm no UI expert, so I'm just identifying some of the pieces that we hear
> > feedback over and hoping that if I can help get the necessary information
> > exposed and appearing at the right places someone else might know how to
> > make it look nice..
>
> Yup, I think this is good feedback.
>
> I do think we need to be careful to avoid the major problems that
> Jenkins has/had around reporting. Storage and presentation of logs and
> historical data was one of the first places Jenkins fell over for us
> which is why we ended up doing external log storage and indexing.
>
> Clark
>

I think most of Jenkins problems was because it was trying to perform build
analysis on all history from the get go. I don't think that needs to live
in zuul, openstack health is probably a good place for all analysis and any
statistical data extraction to live.

It does seem like it would make sense to have the zuul UI provide a way to
look at previous runs and, really that should only involve maintaining the
change information (triggering event?), list of jobs executed, final result
and links to the log files from the jobs. Making it easy to browse the
change results of what zuul just ran, and continue to be able to see your
change jobs even once they are finished would be highly useful to help
users self investigate and solve their own issues.


Bit more on the change level view:

At a change level it seems likely the most useful information for an
individuals change is what is in the square box in the following image:
https://pasteboard.co/GL5hpsr.png

I think it's a case of deciding for a given use case what makes the most
sense. For individuals submitting changes, what is the simplest view that
provides the most important information that they need

Providing a way for only build set of jobs related to the change the person
clicked through from to be show by default in a particular view without all
the other information on the page would be a good starting point for many
users that just want to see how their change is progressing. That is so it
can be provided as a link to be posted back on individual changes.

Using something to help show if the referenced change/job is part of a
dependency chain would be a nice visual enhancement, but that seems like an
extra rather than a requirement.

>From there, the view can have options to switch to:
* all changes in the same pipeline for the current project
* all changes in all pipelines for the current project
* current status view

In many ways this will be a configurable option, because it's all about
what is posted back to the change/pr:
* a link to the main status page
* a link to a project status page using a path url to tell the zuul UI to
only show that project
* a link to a change status page using a path url to only show that change

Adding the filter placeholder to the first two would also work, but as
mentioned above I'd prefer if that was used as a primarily interactive
mechanism for users viewing instead of being hijacked in links published by
zuul back to the code review software.

I think if we focus on the units that provide useful information, and how
to navigate between them, then make sure it's possible to provide whichever
is of most use to be used directly from the change/PR and installations can
decide on which starting point is of the most use to them

-- 
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20170919/96d4ad85/attachment-0001.html>