Open Stack

On Wed, Feb 27, 2013 at 1:18 PM, Sean Dague <sdague at linux.vnet.ibm.com>wrote:

> After lots of running around this morning, we've got a new
> python-quantumclient v.2.1.2 that fixes OpenStack CI. The gate merge was
> only broken for 4 hours, before California woke up, so overall project
> impact was pretty bad.
>
> I think there are a couple lessons learned out of this, at least in my
> opinion. As someone that's chased 2 big breaks like this in 2 days, I'd
> like to figure out how we stop doing that. :)
>
> 1) python-FOOclients should release more often
>
> All it takes is setting a tag on a python-FOOclient to generate a release,
> CI does the push into pypi. We should be releasing early and often here.
> This issue was known to be an issue back in December, which is when the fix
> landed in the python-quantumclient tree, but because there wasn't a
> python-quantumclient release it managed to be a gate blocker.
>
> Release early, release often. clients don't need to be on the 6 month
> cadence that the server projects are on, as clients should work with
> multiple versions of the servers.
>
> It's also probably worth making sure that all cores on a project known and
> understand the python-client release process - https://wiki.openstack.org/
> **wiki/GerritJenkinsGithub#**Tagging_a_Release<https://wiki.openstack.org/wiki/GerritJenkinsGithub#Tagging_a_Release>
>
> It's super easy, and should be doable at any time.
>
> 2) open ended requires are going to hurt us once we go to freeze.
>
> We've had 3 breaks of our gate this week due to pip updates. Horizon due
> to django, Glance due to jsonscheme, and Quantum Client due to pyparsing.
> These typically break stable as well (I known the django one did).
>
> I've chased the gate fixes on 2 of these, it's definitely getting old. I
> understand the desire to not artificially limit ourselves, but it feels
> like once we get into freeze we should start capping these things. We can
> keep them open in open development, but I think we need to start clamping
> these down once we hit freeze.
>

I'm having the same issue with prettytable and some of the client libraries
under devstack right now, too. I wonder if that's the other issue you
chased down?


>
> 3) order in pip-requires matters :)
>
> The eventual fix for python-quantumclient was to bump cliff. There was
> nothing wrong with cliff, but cliff depends on pyparsing as well, and
> didn't specify strict version until 1.3.1. In reality this probably wasn't
> even a needed change for the pypi package, but it did let it get through
> our pip based pep8 scans.


> The CI fix should really have been to move pyparsing up in the file to
> install before cliff, and 1.2.1 would then still work.
>

Yes, cliff didn't require any code changes. I borrowed some setup code from
an unreleased version cmd2 (the library that actually specifies the
pyparsing dependency) and copied it into cliff's requirements list to pin
the version earlier. As you say, it probably would have been possible to do
that right in the quantumclient dependency list, too. OTOH, cliff 1.3.1
also includes some UI fixes requested by the Quantum team, and I had
promised a release soon anyway, so two birds, one stone, etc.

Doug


>
> Anyway, alternative thoughts are really appreciated. I realize point #2 is
> a little contentious, and there is also another thread on it. But it's also
> no good to have git code stop working for people, and us to not realize it
> until someone tries to merge code.
>
>         -Sean
>
> --
> Sean Dague
> IBM Linux Technology Center
> email: sdague at linux.vnet.ibm.com
> alt-email: sldague at us.ibm.com
>
>
> ______________________________**_________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.**org <OpenStack-dev at lists.openstack.org>
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130227/fd444560/attachment.html>