Open Stack

On Fri, Aug 22, 2014, at 05:55 AM, Ihar Hrachyshka wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> Hi all,
> 
> this week is quite bumpy for unit testing in gate. First, it was
> upgrade to new 'tox' version that broke quite some branches.
> 
I did a ton of work to make the tox upgrade go smoothly because we knew
it would be somewhat painful. About a month ago I sent mail to this list
[0] describing the problem. This thread included a pointer to the bug
filed to track this [1] and example work around changes [2] which I
wrote and proposed for as many projects and branches as I had time to
test at that point.

Updating tox to 1.7.2 is important for a couple reasons. We get a lot of
confused developers wondering why using tox doesn't work to run their
tests when all of our documentation says just run tox. Well you needed a
special version (1.6.1). Communicating that to everyone that tries to
run tox is hard.

It is also important because tox adds new features like the hashseed
randomization. This is the cause of our problems but it is exposing real
bugs in openstack [3]. We should be fixing these issues and hopefully my
proposed workarounds are only temporary.

I decided to push ahead [4] and upgrade tox a couple days ago for a
couple reasons. This is an important change as illustrated above and
feature freeze and stabilization are rapidly approaching and this
probably needed to get in soon to have a chance at getting in at all. I
felt this was appropriate because I had done a ton of work prior to make
things go as smoothly as possible.

Where things did not go smoothly was on the reviews for my workaround.
Some changes were basically ignored [5] others ran into procedural
paperwork associated with stable branches that are not quite appropriate
for changes of this type [6][7]. I get that generally we only want to
backport things from master and that we have some specific way to cherry
pick things, but this type of change is to address issues with
stable/foo directly and has nothing to do with master. I did eventually
go through the "backport" dance for most of these changes despite this
not actually being a true backport.

[0]
http://lists.openstack.org/pipermail/openstack-dev/2014-July/041283.html
[1] https://bugs.launchpad.net/cinder/+bug/1348818
[2] https://review.openstack.org/#/c/109700/
[3]
http://lists.openstack.org/pipermail/openstack-dev/2014-July/041496.html
[4]
http://lists.openstack.org/pipermail/openstack-dev/2014-August/042010.html
[5] https://review.openstack.org/#/c/109749/
[6] https://review.openstack.org/#/c/109759/
[7] https://review.openstack.org/#/c/109750/

With all of that out of the way are there suggestions for how we can do
this better next time? Do we need more time (I gave us about 4 weeks
which seemed like plenty to me)? Perhaps I should send more reminder
emails? Feedback is very welcome.

Thanks,
Clark
>
> And today new testtools 0.9.36 were released and were caught by gate,
> which resulted in the following unit test failures in multiple projects:
> 
> TestCase.setUp was already called. Do not explicitly call setUp from
> your tests. In your own setUp, use super to call the base setUp.
> 
> All branches are affected: havana, icehouse, and master.
> 
> This is because the following check was released with the new version
> of the library:
> https://github.com/testing-cabal/testtools/commit/5c3b92d90a64efaecdc4010a98002bfe8b888517
> 
> And the temporary fix is to merge the version pin patch in global
> requirements, backport it to stable branches, and merge the updates
> from Openstack Proposal Bot to all affected projects. The patch for
> master requirements is: https://review.openstack.org/#/c/116267/
> 
> In the meantime, projects will need to fix their tests not to call
> setUp() and tearDown() twice. This will be the requirement to unpin
> the version of the library.
> 
> So, please review, backport, and make sure it lands in project
> requirements files.
> 
> /Ihar
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
> 
> iQEcBAEBCgAGBQJT9z3cAAoJEC5aWaUY1u57DtsIAOFtK2i4zkMcC79nOrc5w9DW
> oO2b064eyLwwbQEaWeeIL2JBSLBxqNV5zeN0eZB3Sq7LQLv0oPaUNTMFG2+gvask
> JHCTAGKz776Rt7ptcfmpHURwcT9L//+1HXvd+ADtO0sYKwgmvaBF7aA4WFa4TseG
> JCnAsi5OiOZZgTo/6U1B55srHkZr0DWxqTkKKysZJbR2Pr/ZT9io8yu9uucaz9VH
> uNLfggtCcjGgccl7IqSUtVRf3lsSGuvBAxVqMszSFJQmFCjy2E26GfsTApp9KXtQ
> gbCpEns8QCnt6KF9rygjHLMbYikjbITuUfSL2okZelX9VpKNx0CS29K/tRg5/BA=
> =YavB
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev