Open Stack

On Tue, Sep 5, 2017, at 09:25 AM, Darragh Bailey wrote:
> Hi,
> 
> 
> Part of a team using Zuul internally with both GitHub Enterprise and
> Gerrit
> projects. Currently we're using the fork from https://github.com/bonnyci/
> zuul with cherry-picks from the openstack-infra/zuul v2 applied. Hoping
> to
> migrate to v3 in the next few months, so for any improvements we'd like
> to
> try and work on adding them to upstream v3 rather than delivering on our
> current code-base.
> 
> 
> Looking for some guidance/suggestions on where the zuul UI is going and
> how
> our customer needs/wants can be achieved so we can plan to help out
> adding
> the improvements. Perhaps this fits into Monty's request a few months
> back
> for "Priorties, backlog and awesome new future things"?
> 
> Currently the main issue from end users, is the user experience around
> the
> UI when looking at job results once the run is complete, and to a lesser
> extend looking at your jobs in the zuul status page when running.
> 
> 
> I'm wondering if there is a plan to have a full UI covering both
> historical
> job runs for projects as well as live status reporting?
> 
> There seems to be an expectation from many users familiar with the CI
> tools
> that support GitHub that you can easily look at previous build runs, see
> what sets failed, support status badges, report on configuration
> settings,
> etc. see http://beta.drone.io/drone-demos/drone-with-go for an example
> with
> drone. While I'm not expecting that all of these can be delivered
> immediately, rather than hacking on specific UI pieces in isolation, it
> might be more useful to focus on them in the context of how they should
> fit
> into a UI overall.
> 
> I've noticed there is an SQL reporter, should that be leveraged to
> provide
> access to the sort of information that needs to be tracked? Or should
> something else be used such as simple files to identify success/failure?
> 
> 
> Some suggestions below that cover the main aspects that there are asks
> for,
> by no means a complete UI, but perhaps there is something already
> partially
> thought out where I can focus on the pieces that will be of most impact
> initially for our use case.
> 

I can't speak for Jim et al, but have some thoughts below.

> ----------------------------
> 
> 1) Project status view:
> 
> The main status page can be useful for those supporting zuul in an
> environment to have a quick overview of everything, however feedback
> we've
> received locally suggests most developers are frequently only interested
> in
> a single project and find this to be overwhelming This suggests there
> should be a per-project status view, showing only the pipelines and
> changes
> of interest for a single project, along with any CRD that are linked.
> 
> Possibly under an endpoint of "/<project>/" as the default view showing
> any
> pipelines relevant to the project along with any changes going through
> them.
> 

This is already possible via the status page filter right? The
difference would be that the filter is cookie based and not path based.
Thinking about it, both provide nice features (with cookie based filter
I don't have to remember links I just always get things filtered the way
I want, with url it is easier to link to what you are seeing).

> ----------------------------
> 
> 2) Project build history view:
> 
> Returning links at the end of a run in a comment to the raw log files as
> the only build history is somewhat jarring for people used to the
> Travis/Drone/Circle CI services available, there's the expectation that
> it
> should be easily to see the previous runs, seeing whether they passed,
> which jobs within a set failed, and to display the detailed log when
> desired. Providing a reasonable UI is becoming a pre-requisite for
> adoption, no matter how good the functionality, it can be difficult to
> get
> acceptance if the "form" is deemed limiting (non all follow this, but it
> does seem that some of the most vocal objectors can fall into this area).
> 
> Users can be quick to blame the infrastructure when a chance working on
> their system fails in CI and seeing previous runs is non trivial.  Making
> it easy to see that previous changes were working (or failing) can make
> it
> more likely that they will look closer at their own change for the cause
> of
> the failure. It also tends to be easier to understand a failing job, when
> you can easily view the output from a passing one.

Yup, this is the reason the openstack health dashboard,
http://status.openstack.org/openstack-health/#/, exists. Being able to
see the bigger picture and status of a project is important. It also
provides rss feeds so you don't have to actively look at a screen to get
notifications which is nice. A lot of the health dashboard features are
probably fairly openstack specific, like the integration with
elastic-recheck and subunit, but my hunch is that we can probably make a
health dashboard that falls back gracefully to a base set of features
that a "normal" zuul v3 installation would support.

Rather than start from scratch maybe it makes sense to build on this
health dashboard?

> 
> Zuul could provide a build history on each set of jobs that has been
> triggered for each project:
> Display the overall result (pass/fail) along side information linking to
> the change triggered, may also include note identifying the triggering
> reason such as change created, ref updated, comment with matching message
> added, etc.
> Clicking on this info should provide extended information on all of the
> individual jobs run, their status and link to the log file. Could always
> load the load the log file from the remote file server and display inline
> using JS and display with syntax highlighting as needed as a future
> enhancement.

I must be weird, I really dislike the super fancy log setups that
Travis, Shippable, etc use because it is hard to manipulate them to find
what I need. Making things pretty is fine but lets be sure to make it
easy to get the raw data too.

> 
> Possibly under an endpoint of "/<project>/history" (or some other name
> besides history)
> 
> ----------------------------
> 
> 3) Change view
> 
> Users of tools such as GitHub are expecting a link to appear on the PR
> status checks that can then be followed to see the state of the job(s)
> currently executing. Currently this can be enabled through setting of a
> config option "status_url_with_change" in the zuul section. This can also
> be used with Gerrit making it easy to quickly see a single change.
> 
> Being able to focus on just the jobs associated with the single change
> you
> are interested in, allows developers to focus on the most important
> information relevant to them. There is an expectation with users of other
> CI tools with GitHub that this should be possible and straight forward as
> well.
> 
> With the existing status view, it is possible to set the url posted back
> to
> Gerrit/GitHub/(other?) such that following it will automatically filter
> the
> view. However the current behaviour is to split everything in the filter
> on
> ',' and show anything matching any of the fields. It is possible to limit
> this by changing the link posted back to use an underscore instead of a
> comma to separate <change>_<interation> (gerrit) or <pr>_<sha1> (github)
> and the filter will work a little better, however as it's doing partial
> matching it's possible for a in gerrit such as 23_7 to match a PR 23 if
> the
> SHA1 starts with 7 for some GitHub project. Low risk, but with sufficient
> changes and sufficient installations it could happen. As the Gerrit
> change
> numbers increase, the risk of this colliding with GitHub PRs decreases.

I'd be wary of forcing users to do a modification to the change atom
itself in order to get it to filter properly. Would it make more sense
to split rules on some other character/string? That way I can take what
gerrit says is the unique identifier for my change, eg '1234,5' and
paste that in directly without needing to change anything. And only care
about the special stuff if I want more than one filter item.

> 
> There still ends up there being an amount of additional unnecessary info
> appearing in this in any case, as any projects with changes not matched,
> still have the project name appear as a visual clue that there changes in
> the pipeline but are hidden by the current filter.
> 
> It would seem more desirable to be able to show a single change by
> itself,
> with dependent changes indicated and expandable to appear as well.
> This could start with something that only works when the change is
> running,
> and if historical data is supported it could then display the archived
> information subsequently, minus any cross-repo dependency information
> unless someone feels that would be useful.
> 
> Possibly under an endpoint of /<project>/change/<item>
> 
> As the same change may appear under multiple pipelines and also may have
> multiple runs per pipeline, this would probably need some thought as to
> how
> it would work if viewing history was possible before deciding.
> 
> ----------------------------
> 
> Are these of any interest?
> 
> I'm no UI expert, so I'm just identifying some of the pieces that we hear
> feedback over and hoping that if I can help get the necessary information
> exposed and appearing at the right places someone else might know how to
> make it look nice..

Yup, I think this is good feedback.

I do think we need to be careful to avoid the major problems that
Jenkins has/had around reporting. Storage and presentation of logs and
historical data was one of the first places Jenkins fell over for us
which is why we ended up doing external log storage and indexing.

Clark