[openstack-dev] [cinder][nova] gate-cinder-python34 failing test_nova_timeout after novaclient 2.33 release
Matt Riedemann
mriedem at linux.vnet.ibm.com
Wed Oct 28 18:44:33 UTC 2015
On 10/28/2015 12:28 PM, Matt Riedemann wrote:
>
>
> On 10/28/2015 10:41 AM, Ivan Kolodyazhny wrote:
>> Matt,
>>
>> Thank you for bring this topic to the ML.
>>
>> In cinder, we've merged [1] patch to unblock gates. I've proposed other
>> patch [2] to fix global-requirements for the stable/liberty branch.
>>
>>
>> [1] https://review.openstack.org/#/c/239837/
>> [2] https://review.openstack.org/#/c/239799/
>>
>> Regards,
>> Ivan Kolodyazhny,
>> http://blog.e0ne.info/
>>
>> On Thu, Oct 29, 2015 at 12:13 AM, Matt Riedemann
>> <mriedem at linux.vnet.ibm.com <mailto:mriedem at linux.vnet.ibm.com>> wrote:
>>
>>
>>
>> On 10/28/2015 9:22 AM, Matt Riedemann wrote:
>>
>>
>>
>> On 10/28/2015 9:06 AM, Yuriy Nesenenko wrote:
>>
>> Hi. Look at https://review.openstack.org/#/c/239837/
>>
>> On Wed, Oct 28, 2015 at 3:52 PM, Matt Riedemann
>> <mriedem at linux.vnet.ibm.com
>> <mailto:mriedem at linux.vnet.ibm.com>
>> <mailto:mriedem at linux.vnet.ibm.com
>> <mailto:mriedem at linux.vnet.ibm.com>>> wrote:
>>
>> That job is failing at a decent rate, tracking with bug:
>>
>> https://bugs.launchpad.net/cinder/+bug/1510656
>>
>> It lines up with the novaclient 2.33 release on 10/27,
>> I'm checking
>> out what the change was that caused the regression.
>>
>> This is a heads up that rechecks on this failure
>> probably won't help.
>>
>> So far I haven't seen any related patches up to fix it
>> although
>> there were already 2 bugs reported when I got in this
>> morning.
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>>
>>
>> __________________________________________________________________________
>>
>>
>> OpenStack Development Mailing List (not for usage
>> questions)
>> Unsubscribe:
>>
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>>
>> __________________________________________________________________________
>>
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>>
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> Heh, well that's 3 bugs then, I didn't see that one. jgriffith
>> and I
>> were talking in IRC about just handling both exceptions in
>> cinder to fix
>> this but we also agreed that this is a backward incompatible
>> change on
>> the novaclient side, which was also discussed in the original
>> novaclient
>> wishlist bug that prompted the breaking change.
>>
>> Given the backward compat issues, we might not just be breaking
>> cinder
>> here, so I've proposed a revert of the novaclient change with
>> justification in the commit message:
>>
>> https://review.openstack.org/#/c/239941/
>>
>> At least with the cinder change above we're OK for mitaka, and
>> logstash
>> isn't yet showing failures for cinder in stable/liberty, but
>> given the
>> requirements there it will be a failure in cinder python34
>> tests in
>> stalbe/liberty also - so we can backport the cinder fix or
>> block the
>> 2.33 novaclient version on stable/liberty global-requirements
>> depending
>> on what we do with the proposed novaclient revert.
>>
>>
>> I have an alternative to the revert here:
>>
>> https://review.openstack.org/#/c/239963/
>>
>> That makes novaclient.exceptions.RequestTimeout extend
>> requests.Timeout so that older cinder continues to work.
>>
>> I also have changes to block novaclient 2.33.0 in g-r on master and
>> stable/liberty:
>>
>>
>> https://review.openstack.org/#/q/I6e7657b60308b30eed89b269810c1f37cce43063,n,z
>>
>>
>> I personally think we need to block 2.33.0 since it breaks cinder,
>> then release a new version of novaclient with either the revert or
>> the alternative change to extend requests.Timeout.
>>
>> If we block novaclient 2.33.0 then we can also revert the workaround
>> in cinder (which would start breaking if we reverted the new
>> exception type out of novaclient w/o blacklisting 2.33 first).
>>
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> The novaclient revert patch is approved:
>
> https://review.openstack.org/#/c/239941/
>
> Once that is merged I'll propose a release request for novaclient.
>
> I've got g-r patches for master and stable/liberty to block 2.33:
>
> https://review.openstack.org/#/q/I6e7657b60308b30eed89b269810c1f37cce43063,n,z
>
>
> Once those land and we've synced the change to cinder, we can revert the
> cinder workaround.
>
So the revert is merged [1] but now I'm stuck as to the version and/or
git commit to use for the next release.
The changes since 2.33.0 are:
mriedem at ubuntu:~/git/python-novaclient$ git log --oneline --no-merges
2.33.0..
63c7a57 Revert "Do not expose exceptions from requests library"
217e7c1 Updated from global requirements
0cd5812 Remove novaclient.v1_1 module
And when I requested the 2.33.0 release [2], it was on 217e7c1 but they
changed it to be w/o 0cd5812 because that is considered a backward
incompatible change. But it was released with 2961e82 which was the
backward incompatible requests exception change, which we now have a fix
for that we want to release, but would include 0cd5812.
So do we just release novaclient trunk as 3.0? We still have the g-r
blacklist changes for 2.33.0 in mitaka and liberty so we wouldn't be
using that version in tests (assuming that's merged in g-r). So then
people have to move up to 3.0 if they want the latest novaclient changes.
This is a bit gross.
[1] https://review.openstack.org/#/c/239941/
[2] https://review.openstack.org/#/c/239450/
--
Thanks,
Matt Riedemann
More information about the OpenStack-dev
mailing list