[openstack-dev] [infra] [cinder] CI via infra for the DRBD Cinder driver
philipp.marek at linbit.com
Mon Jun 22 07:34:58 UTC 2015
> this is a reflection of the discussion I just had on #openstack-infra; it's
> about (re-)using the central CI infrastructure for our Open-Source DRBD
> driver too.
As this is done now, I'd like to discuss a few points.
First of all -- thanks to everybody involved; if you happen to find someone
(Europe is a bad timezone for Openstack, I guess), people on IRC are *very*
> «Yup, so the two things I will start with is that multinode testing is
> still really rudimentary, we only just got tempest sort of working with
> it. ... »
I didn't follow all the conversations on the ML - is there an update for
What's the policy? "Good idea, but not needed" or "A must-have"?
Especially for distributed/replicated storage drivers (like DRBD ;) I guess
it would make sense to have some.
Third point, mostly of interest for other driver maintainers: we're
following the various CI runs, and sporadically check failures to see
whether we can improve our driver or the devstack installation script.
For that we use "Kibana" on http://logstash.openstack.org, with a filter
project:"openstack/cinder" AND build_status:"FAILURE" AND
build_name:"check-tempest-dsvm-full-drbd-devstack-nv" AND "Detailed logs"
This shows the initial line of the failed devstack runs (the one including
the line with the log URL), which we can then paste into a script that
fetches the (for us) relevant files for further analysis.
The newest failures are already at the top.
Another nice feature is using the existing Graphite installation to get
a visual display of the success/failure ratio over time; here we can see
the impact of individual changes, eg. on June 19th we diagnosed (and fixed)
an udev/blkid race with the kernel-attach, since then the number of
failures has clearly gone down. I just pushed another patch that should us
bring even more "into the green area" ;)
One more idea is to watch http://status.openstack.org/zuul/? with a filter
string of "Cinder", and to open reviews that will finish soon, so that
"current" behaviour can be easily checked.
I hope that this helps other people a bit.
: Ing. Philipp Marek
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com :
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
More information about the OpenStack-dev