Open Stack

On 02/27/2013 02:48 PM, Doug Hellmann wrote:
> 
> 
> On Wed, Feb 27, 2013 at 1:18 PM, Sean Dague <sdague at linux.vnet.ibm.com
> <mailto:sdague at linux.vnet.ibm.com>> wrote:
> 
>     After lots of running around this morning, we've got a new
>     python-quantumclient v.2.1.2 that fixes OpenStack CI. The gate merge
>     was only broken for 4 hours, before California woke up, so overall
>     project impact was pretty bad.
> 
>     I think there are a couple lessons learned out of this, at least in
>     my opinion. As someone that's chased 2 big breaks like this in 2
>     days, I'd like to figure out how we stop doing that. :)
> 
>     1) python-FOOclients should release more often
> 
>     All it takes is setting a tag on a python-FOOclient to generate a
>     release, CI does the push into pypi. We should be releasing early
>     and often here. This issue was known to be an issue back in
>     December, which is when the fix landed in the python-quantumclient
>     tree, but because there wasn't a python-quantumclient release it
>     managed to be a gate blocker.
> 
>     Release early, release often. clients don't need to be on the 6
>     month cadence that the server projects are on, as clients should
>     work with multiple versions of the servers.
> 
>     It's also probably worth making sure that all cores on a project
>     known and understand the python-client release process -
>     https://wiki.openstack.org/__wiki/GerritJenkinsGithub#__Tagging_a_Release
>     <https://wiki.openstack.org/wiki/GerritJenkinsGithub#Tagging_a_Release>
> 
>     It's super easy, and should be doable at any time.
> 
>     2) open ended requires are going to hurt us once we go to freeze.
> 
>     We've had 3 breaks of our gate this week due to pip updates. Horizon
>     due to django, Glance due to jsonscheme, and Quantum Client due to
>     pyparsing. These typically break stable as well (I known the django
>     one did).
> 
>     I've chased the gate fixes on 2 of these, it's definitely getting
>     old. I understand the desire to not artificially limit ourselves,
>     but it feels like once we get into freeze we should start capping
>     these things. We can keep them open in open development, but I think
>     we need to start clamping these down once we hit freeze.
> 
> 
> I'm having the same issue with prettytable and some of the client
> libraries under devstack right now, too. I wonder if that's the other
> issue you chased down?

I totally understand the argument against stricter pinning, but I'd like
to repeat my argument for it.

That is - while in theory it might seem nice to be able to "get upgrades
for free" from upstreams, experience in this project has shown over and
over that it's almost always a disaster. It _shouldn't_ be - but it is.

>     3) order in pip-requires matters :)
> 
>     The eventual fix for python-quantumclient was to bump cliff. There
>     was nothing wrong with cliff, but cliff depends on pyparsing as
>     well, and didn't specify strict version until 1.3.1. In reality this
>     probably wasn't even a needed change for the pypi package, but it
>     did let it get through our pip based pep8 scans. 
> 
> 
>     The CI fix should really have been to move pyparsing up in the file
>     to install before cliff, and 1.2.1 would then still work.
> 
> 
> Yes, cliff didn't require any code changes. I borrowed some setup code
> from an unreleased version cmd2 (the library that actually specifies the
> pyparsing dependency) and copied it into cliff's requirements list to
> pin the version earlier. As you say, it probably would have been
> possible to do that right in the quantumclient dependency list, too.
> OTOH, cliff 1.3.1 also includes some UI fixes requested by the Quantum
> team, and I had promised a release soon anyway, so two birds, one stone,
> etc.

We were considering the other day, when talking about the
openstack/requirements repo, writing a script that would recursively
expand all of the requirements of the requirements and place all of them
in the master list so that had an explicit list of everything.

>     Anyway, alternative thoughts are really appreciated. I realize point
>     #2 is a little contentious, and there is also another thread on it.
>     But it's also no good to have git code stop working for people, and
>     us to not realize it until someone tries to merge code.
> 
>             -Sean
> 
>     -- 
>     Sean Dague
>     IBM Linux Technology Center
>     email: sdague at linux.vnet.ibm.com <mailto:sdague at linux.vnet.ibm.com>
>     alt-email: sldague at us.ibm.com <mailto:sldague at us.ibm.com>
> 
> 
>     _________________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.__org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>