[openstack-dev] [cinder][nova] gate-cinder-python34 failing test_nova_timeout after novaclient 2.33 release

Matt Riedemann mriedem at linux.vnet.ibm.com
Thu Oct 29 14:07:38 UTC 2015



On 10/29/2015 5:55 AM, Andrey Kurilin wrote:
>  >But it was released with 2961e82 which was the backward incompatible
> requests exception change, which we now have a fix for that we want to
> release, but would include 0cd5812.
>
> I suppose we need to revert 0cd5812 change too, cut new release and then
> revert revert of 0cd5812 :)
>
> On Wed, Oct 28, 2015 at 8:44 PM, Matt Riedemann
> <mriedem at linux.vnet.ibm.com <mailto:mriedem at linux.vnet.ibm.com>> wrote:
>
>
>
>     On 10/28/2015 12:28 PM, Matt Riedemann wrote:
>
>
>
>         On 10/28/2015 10:41 AM, Ivan Kolodyazhny wrote:
>
>             Matt,
>
>             Thank you for bring this topic to the ML.
>
>             In cinder, we've merged [1] patch to unblock gates. I've
>             proposed other
>             patch [2] to fix global-requirements for the stable/liberty
>             branch.
>
>
>             [1] https://review.openstack.org/#/c/239837/
>             [2] https://review.openstack.org/#/c/239799/
>
>             Regards,
>             Ivan Kolodyazhny,
>             http://blog.e0ne.info/
>
>             On Thu, Oct 29, 2015 at 12:13 AM, Matt Riedemann
>             <mriedem at linux.vnet.ibm.com
>             <mailto:mriedem at linux.vnet.ibm.com>
>             <mailto:mriedem at linux.vnet.ibm.com
>             <mailto:mriedem at linux.vnet.ibm.com>>> wrote:
>
>
>
>                  On 10/28/2015 9:22 AM, Matt Riedemann wrote:
>
>
>
>                      On 10/28/2015 9:06 AM, Yuriy Nesenenko wrote:
>
>                          Hi. Look at
>             https://review.openstack.org/#/c/239837/
>
>                          On Wed, Oct 28, 2015 at 3:52 PM, Matt Riedemann
>                          <mriedem at linux.vnet.ibm.com
>             <mailto:mriedem at linux.vnet.ibm.com>
>                          <mailto:mriedem at linux.vnet.ibm.com
>             <mailto:mriedem at linux.vnet.ibm.com>>
>                          <mailto:mriedem at linux.vnet.ibm.com
>             <mailto:mriedem at linux.vnet.ibm.com>
>                          <mailto:mriedem at linux.vnet.ibm.com
>             <mailto:mriedem at linux.vnet.ibm.com>>>> wrote:
>
>                               That job is failing at a decent rate,
>             tracking with bug:
>
>             https://bugs.launchpad.net/cinder/+bug/1510656
>
>                               It lines up with the novaclient 2.33
>             release on 10/27,
>                          I'm checking
>                               out what the change was that caused the
>             regression.
>
>                               This is a heads up that rechecks on this
>             failure
>                          probably won't help.
>
>                               So far I haven't seen any related patches
>             up to fix it
>                          although
>                               there were already 2 bugs reported when I
>             got in this
>                          morning.
>
>                               --
>
>                               Thanks,
>
>                               Matt Riedemann
>
>
>
>
>             __________________________________________________________________________
>
>
>                               OpenStack Development Mailing List (not
>             for usage
>                          questions)
>                               Unsubscribe:
>
>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
>
>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
>             __________________________________________________________________________
>
>
>                          OpenStack Development Mailing List (not for
>             usage questions)
>                          Unsubscribe:
>
>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>                      Heh, well that's 3 bugs then, I didn't see that
>             one. jgriffith
>             and I
>                      were talking in IRC about just handling both
>             exceptions in
>                      cinder to fix
>                      this but we also agreed that this is a backward
>             incompatible
>                      change on
>                      the novaclient side, which was also discussed in
>             the original
>                      novaclient
>                      wishlist bug that prompted the breaking change.
>
>                      Given the backward compat issues, we might not just
>             be breaking
>                      cinder
>                      here, so I've proposed a revert of the novaclient
>             change with
>                      justification in the commit message:
>
>             https://review.openstack.org/#/c/239941/
>
>                      At least with the cinder change above we're OK for
>             mitaka, and
>                      logstash
>                      isn't yet showing failures for cinder in
>             stable/liberty, but
>                      given the
>                      requirements there it will be a failure in cinder
>             python34
>             tests in
>                      stalbe/liberty also - so we can backport the cinder
>             fix or
>             block the
>                      2.33 novaclient version on stable/liberty
>             global-requirements
>                      depending
>                      on what we do with the proposed novaclient revert.
>
>
>                  I have an alternative to the revert here:
>
>             https://review.openstack.org/#/c/239963/
>
>                  That makes novaclient.exceptions.RequestTimeout extend
>                  requests.Timeout so that older cinder continues to work.
>
>                  I also have changes to block novaclient 2.33.0 in g-r
>             on master and
>                  stable/liberty:
>
>
>             https://review.openstack.org/#/q/I6e7657b60308b30eed89b269810c1f37cce43063,n,z
>
>
>                  I personally think we need to block 2.33.0 since it
>             breaks cinder,
>                  then release a new version of novaclient with either
>             the revert or
>                  the alternative change to extend requests.Timeout.
>
>                  If we block novaclient 2.33.0 then we can also revert
>             the workaround
>                  in cinder (which would start breaking if we reverted
>             the new
>                  exception type out of novaclient w/o blacklisting 2.33
>             first).
>
>
>                  --
>
>                  Thanks,
>
>                  Matt Riedemann
>
>
>
>             __________________________________________________________________________
>
>                  OpenStack Development Mailing List (not for usage
>             questions)
>                  Unsubscribe:
>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>             __________________________________________________________________________
>
>             OpenStack Development Mailing List (not for usage questions)
>             Unsubscribe:
>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>         The novaclient revert patch is approved:
>
>         https://review.openstack.org/#/c/239941/
>
>         Once that is merged I'll propose a release request for novaclient.
>
>         I've got g-r patches for master and stable/liberty to block 2.33:
>
>         https://review.openstack.org/#/q/I6e7657b60308b30eed89b269810c1f37cce43063,n,z
>
>
>         Once those land and we've synced the change to cinder, we can
>         revert the
>         cinder workaround.
>
>
>     So the revert is merged [1] but now I'm stuck as to the version
>     and/or git commit to use for the next release.
>
>     The changes since 2.33.0 are:
>
>     mriedem at ubuntu:~/git/python-novaclient$ git log --oneline
>     --no-merges 2.33.0..
>     63c7a57 Revert "Do not expose exceptions from requests library"
>     217e7c1 Updated from global requirements
>     0cd5812 Remove novaclient.v1_1 module
>
>     And when I requested the 2.33.0 release [2], it was on 217e7c1 but
>     they changed it to be w/o 0cd5812 because that is considered a
>     backward incompatible change. But it was released with 2961e82 which
>     was the backward incompatible requests exception change, which we
>     now have a fix for that we want to release, but would include 0cd5812.
>
>     So do we just release novaclient trunk as 3.0? We still have the g-r
>     blacklist changes for 2.33.0 in mitaka and liberty so we wouldn't be
>     using that version in tests (assuming that's merged in g-r). So then
>     people have to move up to 3.0 if they want the latest novaclient
>     changes.
>
>     This is a bit gross.
>
>     [1] https://review.openstack.org/#/c/239941/
>     [2] https://review.openstack.org/#/c/239450/
>
>
>     --
>
>     Thanks,
>
>     Matt Riedemann
>
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Best regards,
> Andrey Kurilin.
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Yeah that's exactly what we're doing.  See:

https://review.openstack.org/#/c/240052/

https://review.openstack.org/#/c/240020/

https://review.openstack.org/#/c/240241/

-- 

Thanks,

Matt Riedemann




More information about the OpenStack-dev mailing list