Open Stack

Thanks Matt.

Agreed on 1, 2, 3, and 4. The reason for #5 is that there are other changes (for which we have FFE's) which will be merged and a new version of the client requested.

Specifically, this change

https://review.openstack.org/290177

Thanks,

-amrith

> -----Original Message-----
> From: Matt Riedemann [mailto:mriedem at linux.vnet.ibm.com]
> Sent: Wednesday, March 09, 2016 2:53 PM
> To: openstack-dev at lists.openstack.org
> Subject: Re: [openstack-dev] [tc] [trove][stable] proboscis tests randomly
> failing in stable branches; bug 1538506
> 
> 
> 
> On 3/9/2016 1:34 PM, Amrith Kumar wrote:
> > Matt,
> >
> > We discussed this at the Trove meeting. Here's my current understanding
> of the situation:
> >
> > 1. Your change https://review.openstack.org/#/c/290048/ which reverts
> the trove client side of the change should be merged.
> >
> > 2. This change (the Trove API side)
> https://review.openstack.org/#/c/245845/ should also be reverted. I'm
> assuming that you'll submit a change for this as well for completeness.
> 
> I can submit a change for this.
> 
> >
> > 3. We need to blacklist python-troveclient 2.1.0 on Liberty, this is
> > your change https://review.openstack.org/#/c/290021/
> >
> > 4. We need to blacklist python-troveclient 2.1.0 on master, this is
> currently *NOT* what your change https://review.openstack.org/290168 does.
> I disagree with the approach of just increasing the minimum requirement
> from 1.2.0 to >=2.1.0.
> 
> I'm OK with blacklisting 2.1.0 on master and can make that change in the
> same patch.
> 
> >
> > 5. We need to request a new python-troveclient. Whether it should be
> called 2.1.1 or 2.2.0, I'm not positive. My thought is 2.1.1. But out of
> an abundance of caution, I'm going to look to #openstack-
> release/#openstack-infra for guidance on this.
> 
> There are no other changes to python-troveclient since 2.1.0 was released:
> 
> user at xubuntu:~/git/python-troveclient$ git log --oneline --no-merges
> 2.1.0..
> user at xubuntu:~/git/python-troveclient$
> 
> 
> So 2.1.1 is fine.
> 
> >
> > Thanks,
> >
> > -amrith
> >
> >> -----Original Message-----
> >> From: Matt Riedemann [mailto:mriedem at linux.vnet.ibm.com]
> >> Sent: Wednesday, March 09, 2016 12:42 PM
> >> To: openstack-dev at lists.openstack.org
> >> Subject: Re: [openstack-dev] [trove][stable] proboscis tests randomly
> >> failing in stable branches; bug 1538506
> >>
> >>
> >>
> >> On 3/9/2016 8:54 AM, Amrith Kumar wrote:
> >>> Matt,
> >>>
> >>> As I understand it, you have 4 changes up for review.
> >>>
> >>>       Change [1] seeks to revert the change [6] to python-troveclient
> on
> >>>       master and reinstate the slave_of parameter.
> >>>
> >>>       Change [2] is doing for stable/liberty, the same things that
> were
> >>>       done for master in [9] on master, and [7] on Kilo earlier.
> >>>
> >>>       Change [3] is looking to blacklist python-troveclient 2.1.0 on
> >>>       stable/liberty.
> >>>
> >>>       Change [4] is looking to bump the minimum python-troveclient
> >> version
> >>>       on master to 2.1.0.
> >>
> >> Right, and there was actually another for that beat me to this:
> >>
> >> https://review.openstack.org/#/c/290115/
> >>
> >>>
> >>> Here are three questions I have:
> >>>
> >>> (1) Wouldn't backward compatibility also require that you revert the
> >>> other change that removed slave_of on the server [5] ? After all,
> >>> just like a python client that calls create() with slave_of, a REST
> >>> client could call the endpoint with slave_of. What is the policy for
> >>> REST API backward compatibility?
> >>
> >> I guess it depends on the Trove team. In Nova, backward compatibility
> >> in the API is serious business and that's why we have left all sorts
> >> of warts in the v2.0 API and couldn't remove them. But with the v2.1
> >> API and microversions, we're able to move the API forward (Ironic
> >> also has microversions, and I think cinder/neutron/keystone are
> >> working on adding that support).
> >>
> >> So if maintaining backward compatibility in the trove API is
> >> important to the trove team and it's users, then yes the API change
> >> should probably be reverted. For anyone doing CD with Trove, they are
> >> already broken, but people upgrading from liberty to mitaka could save
> themselves the break.
> >>
> >>>
> >>> (2) Wouldn't [4] just block 2.1.0 in global-requirements on master
> >>> (mitaka)?
> >>
> >> I'm not sure I understand, [4] here bumps the minimum required
> >> version of python-troveclient to be 2.1.0, not block it. As noted in
> >> the review, I don't know that it's really necessary to bump to 2.1.0
> >> iff we land and release [1].
> >>
> >>>
> >>> (3) Something, potentially your patch set [2], should also fix the
> >>> test that is invoking create() with slave_of, no?
> >>
> >> [2] is stable/liberty which still has slave_of in the create API, so
> >> I don't think that needs to be fixed in stable/liberty.
> >>
> >>>
> >>> -amrith
> >>>
> >>> [1] https://review.openstack.org/290048/
> >>> [2] https://review.openstack.org/290023/
> >>> [3] https://review.openstack.org/290021/
> >>> [4] https://review.openstack.org/290168/
> >>> [5] https://review.openstack.org/#/c/245845/
> >>> [6] https://review.openstack.org/#/c/245738/
> >>> [7] https://review.openstack.org/#/c/246735/
> >>>
> >>> On 03/08/2016 08:24 PM, Matt Riedemann wrote:
> >>>>
> >>>>
> >>>> On 3/8/2016 4:44 PM, Matt Riedemann wrote:
> >>>>>
> >>>>>
> >>>>> On 3/8/2016 3:17 PM, Matt Riedemann wrote:
> >>>>>>
> >>>>>>
> >>>>>> On 3/8/2016 1:18 PM, Amrith Kumar wrote:
> >>>>>>> Matt,
> >>>>>>>
> >>>>>>> See inline below.
> >>>>>>>
> >>>>>>> -amrith
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Matt Riedemann [mailto:mriedem at linux.vnet.ibm.com]
> >>>>>>>> Sent: Tuesday, March 08, 2016 2:00 PM
> >>>>>>>> To: openstack-dev at lists.openstack.org
> >>>>>>>> Subject: Re: [openstack-dev] [trove][stable] proboscis tests
> >>>>>>>> randomly failing in stable branches; bug 1538506
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 3/8/2016 12:52 PM, Matt Riedemann wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On 3/8/2016 12:35 PM, Amrith Kumar wrote:
> >>>>>>>>>> Matt,
> >>>>>>>>>>
> >>>>>>>>>> The correct solution for liberty is that we should fix the
> tests.
> >>>>>>>>>> Here's why I believe that this is the case.
> >>>>>>>>>>
> >>>>>>>>>> In pertinent part, the backtrace from your bug includes:
> >>>>>>>>>>
> >>>>>>>>>> 2016-01-27 07:02:06.777 | File
> >>>>>>>>>> "/home/jenkins/workspace/periodic-trove-python27-liberty/trov
> >>>>>>>>>> e/
> >>>>>>>>>> tests/
> >>>>>>>>>>
> >>>>>>>>>> api/replication.py",
> >>>>>>>>>> line 83, in create_slave
> >>>>>>>>>> 2016-01-27 07:02:06.777 | slave_of=instance_info.id)
> >>>>>>>>>>
> >>>>>>>>>> This is in the tests, not the client.
> >>>>>>>>>>
> >>>>>>>>>> The test is now generating a parameter that the client no
> >>>>>>>>>> longer understands.
> >>>>>>>>>>
> >>>>>>>>>> For a user, here are the various situations that could occur.
> >>>>>>>>>>
> >>>>>>>>>> 1. User running python-troveclient from the latest 2.1.0 and
> >>>>>>>>>> a server from liberty. All is good.
> >>>>>>>>>> 2. User running an old python-troveclient and a server from
> >>>>>>>>>> liberty.
> >>>>>>>>>> All is good.
> >>>>>>>>>
> >>>>>>>>> Will this work with python-troveclient 1.2.0 which is the
> >>>>>>>>> minimum required troveclient for stable/liberty?
> >>>>>>>>>
> >>>>>>>>> https://github.com/openstack/requirements/blob/stable/liberty/
> >>>>>>>>> gl
> >>>>>>>>> obal-r
> >>>>>>>>>
> >>>>>>>>> equirements.txt#L174
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> 3. User running an old python-troveclient and a new server
> >>>>>>>>>> from mitaka, this is not supported.
> >>>>>>>>>
> >>>>>>>>> How is this not supported?  If it's not supported, the minimum
> >>>>>>>>> required version of troveclient in master g-r is wrong, since
> >>>>>>>>> it hasn't changed since liberty:
> >>>>>>>>>
> >>>>>>>>> https://github.com/openstack/requirements/blob/master/global-r
> >>>>>>>>> eq
> >>>>>>>>> uireme
> >>>>>>>>>
> >>>>>>>>> nts.txt#L202
> >>>>>>>>
> >>>>>>>> I've confirmed that running master (mitaka) unit tests for
> >>>>>>>> trove against python-troveclient 1.2.0 don't work:
> >>>>>>>>
> >>>>>>>> http://paste.openstack.org/show/489719/
> >>>>>>>
> >>>>>>> [amrith] Just like the bug you listed, this appears to be a case
> >>>>>>> of a broken test. Would you please LP it.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> 4. User running a current python-troveclient from the latest
> >>>>>>>>>> 2.1.0 and a server from mitaka, All is good.
> >>>>>>>>>>
> >>>>>>>>>> What you have here is a case where a test is generating an
> >>>>>>>>>> argument
> >>>>>>>>>> (slave_of) that the client (master, 2.1.0) no longer
> understands.
> >>>>>>>>>> There's nothing amiss there, except that the test needs to be
> >>>>>>>>>> fixed.
> >>>>>>>>>>
> >>>>>>>>>> When you get back, let's continue the conversation either in
> >>>>>>>>>> email or IRC #openstack-dev. Looking forward to hearing your
> >>>>>>>>>> feedback on this.
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>>
> >>>>>>>>>> -amrith
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Matt Riedemann [mailto:mriedem at linux.vnet.ibm.com]
> >>>>>>>>>>> Sent: Tuesday, March 08, 2016 12:11 PM
> >>>>>>>>>>> To: openstack-dev at lists.openstack.org
> >>>>>>>>>>> Subject: Re: [openstack-dev] [trove][stable] proboscis tests
> >>>>>>>>>>> randomly failing in stable branches; bug 1538506
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On 3/8/2016 10:17 AM, Matt Riedemann wrote:
> >>>>>>>>>>>> This is a call for help on resolving bug 1538506 [1] where
> >>>>>>>>>>>> the proboscis tests randomly fail on the stable branches
> >>>>>>>>>>>> with something
> >>>>>>>>>>> like:
> >>>>>>>>>>>>
> >>>>>>>>>>>> TypeError: create() got an unexpected keyword argument
> >> 'slave_of'
> >>>>>>>>>>>>
> >>>>>>>>>>>> Craig Vyvial has a proposed stable/kilo change [2] but it
> >>>>>>>>>>>> has some issues, at least from me as a reviewer of that
> change.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The tests are failing the periodic-stable job daily [3].
> >>>>>>>>>>>>
> >>>>>>>>>>>> Can we get someone to help out with this? Otherwise I'm
> >>>>>>>>>>>> inclined to say the tests should be disabled on the stable
> >>>>>>>>>>>> branches, but that's pretty drastic.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Note that this is the kind of thing that breaks stable
> >>>>>>>>>>>> branch policy, which is going to be part of a review when
> >>>>>>>>>>>> applying for the stable:follows-policy tag. [4]  And the
> >>>>>>>>>>>> stable:follows-policy tag may be used to determine when a
> >>>>>>>>>>>> stable branch for a project goes end of life if it's not
> >>>>>>>>>>>> being maintained.
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1] https://bugs.launchpad.net/trove/+bug/1538506
> >>>>>>>>>>>> [2] https://review.openstack.org/#/c/276934/
> >>>>>>>>>>>> [3] http://goo.gl/fqf11U
> >>>>>>>>>>>> [4]
> >>>>>>>>>>>> http://governance.openstack.org/reference/tags/stable_follo
> >>>>>>>>>>>> ws
> >>>>>>>>>>>> -polic
> >>>>>>>>>>>>
> >>>>>>>>>>>> y.h
> >>>>>>>>>>>> tml
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> This is my solution for liberty, cap
> >>>>>>>>>>> python-troveclient<2.1.0 in
> >>>>>>>>>>> global- requirements on stable/liberty [1].
> >>>>>>>>>>>
> >>>>>>>>>>> Then there is a change to trove on stable/liberty to use the
> >>>>>>>>>>> g-r version range for python-troveclient for unit tests [2].
> >>>>>>>>>>>
> >>>>>>>>>>> Alternatively, we could avoid the cap in stable/liberty by:
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Reverting https://review.openstack.org/#/c/245738/
> >>>>>>>>>>> 2. Blacklisting python-troveclient 2.1.0 in g-r 3. Release
> >>>>>>>>>>> python- troveclient 2.2.0.
> >>>>>>>>>>>
> >>>>>>>>>>> It's getting late in the mitaka release to be messing with
> >>>>>>>>>>> this though since we're already past client freeze.
> >>>>>>>>>>>
> >>>>>>>>>>> [1] https://review.openstack.org/#/c/290021/
> >>>>>>>>>>> [2] https://review.openstack.org/#/c/290023/
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>>
> >>>>>>>>>>> Matt Riedemann
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> ____________________________________________________________
> >>>>>>>>>>> __
> >>>>>>>>>>> ______
> >>>>>>>>>>>
> >>>>>>>>>>> ______
> >>>>>>>>>>>
> >>>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>>>>>>>> Unsubscribe:
> >>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
> >>>>>>>>>>> e
> >>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
> >>>>>>>>>>> k-
> >>>>>>>>>>> dev
> >>>>>>>>>>
> >>>>>>>>>> _____________________________________________________________
> >>>>>>>>>> __
> >>>>>>>>>> ______
> >>>>>>>>>>
> >>>>>>>>>> _____
> >>>>>>>>>>
> >>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>>>>>>> Unsubscribe:
> >>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> >>>>>>>>>> -d
> >>>>>>>>>> ev
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>>
> >>>>>>>> Matt Riedemann
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________________________
> >>>>>>>> __
> >>>>>>>> _________
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>>>>> Unsubscribe:
> >>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-d
> >>>>>>>> ev
> >>>>>>>
> >>>>>>> ________________________________________________________________
> >>>>>>> __
> >>>>>>> ________
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>>>> Unsubscribe:
> >>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-de
> >>>>>>> v
> >>>>>>>
> >>>>>>
> >>>>>> Well, the point I'm trying to get to is, I don't think
> >>>>>> troveclient <
> >>>>>> 2.1.0 will work with the trove-api in mitaka.
> >>>>>>
> >>>>>> For example, in my change to revert the slave_of removal in the
> >> client:
> >>>>>>
> >>>>>> https://review.openstack.org/#/c/290048/
> >>>>>>
> >>>>>> That fails in the API:
> >>>>>>
> >>>>>> http://logs.openstack.org/48/290048/2/check/gate-trove-functional
> >>>>>> -d
> >>>>>> svm-mysql/ddd3089/logs/screen-tr-api.txt.gz#_2016-03-08_19_30_32_
> >>>>>> 79
> >>>>>> 0
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> "Validation error: instance Additional properties are not allowed
> >>>>>> (u'slave_of' was unexpected)"
> >>>>>>
> >>>>>> So if my client application that I wrote in liberty was using
> >>>>>> slave_of, it's happily working until the cloud that my
> >>>>>> application is running on upgrades to mitaka, then things break.
> >>>>>>
> >>>>>> Part of the issue here is the backward incompatible change in the
> >>>>>> trove API. The other part is, the trove API in mitaka requires
> >>>>>> troveclient>=2.1.0 because that's what removed slave_of.
> >>>>>>
> >>>>>> Right?
> >>>>>>
> >>>>>> I'm trying to sort out what a valid troveclient is for master,
> >>>>>> because I don't really trust what's in global-requirements since
> >>>>>> trove hasn't been testing with those versions, at least not at
> >>>>>> the lower bound of 1.2.0.
> >>>>>>
> >>>>>
> >>>>> Here is a change that bumps the minimum required version of
> >>>>> python-troveclient for mitaka to 2.1.0:
> >>>>>
> >>>>> https://review.openstack.org/#/c/290168/
> >>>>>
> >>>>
> >>>> Here is my preferred option now.
> >>>>
> >>>> 1. Revert the slave_of removal from python-troveclient and provide
> >>>> it as a compat shim until liberty-eol:
> >>>>
> >>>> https://review.openstack.org/#/c/290048/
> >>>>
> >>>> If slave_of is used, a deprecation warning is emitted (which we
> >>>> should have been doing when this was deprecated in the first place).
> >>>> It also never sends slave_of to the API, it's just a proxy to
> >>>> replica_of. So this should work with both the liberty and mitaka
> >>>> versions of the trove API.
> >>>>
> >>>> 2. Block python-troveclient 2.1.0 from stable/liberty g-r:
> >>>>
> >>>> https://review.openstack.org/#/c/290021/
> >>>>
> >>>> That blocks the bad 2.1.0 version from being used in stable/liberty
> >>>> which is needed for...
> >>>>
> >>>> 3. Using the g-r versions of python-troveclient when testing on
> >>>> stable/liberty:
> >>>>
> >>>> https://review.openstack.org/#/c/290023/
> >>>>
> >>>> If trove is following global-requirements, it should be using the
> >>>> versions of python-troveclient from global-requirements rather than
> >>>> installing it from the latest tarball on trunk. master and
> >>>> stable/kilo are already doing this.
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>> ____________________________________________________________________
> >>> __ ____ OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >> --
> >>
> >> Thanks,
> >>
> >> Matt Riedemann
> >>
> >>
> >> _____________________________________________________________________
> >> _____ OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > ______________________________________________________________________
> > ____ OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> --
> 
> Thanks,
> 
> Matt Riedemann
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev