[openstack-dev] [Neutron][LBaaS][heat] Removing LBaaS v1 - are weready?

Adrian Otto adrian.otto at rackspace.com
Thu Mar 24 15:04:49 UTC 2016



> On Mar 24, 2016, at 7:48 AM, Hongbin Lu <hongbin.lu at huawei.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Assaf Muller [mailto:assaf at redhat.com]
>> Sent: March-24-16 9:24 AM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] [Neutron][LBaaS][heat] Removing LBaaS v1 -
>> are weready?
>> 
>> On Thu, Mar 24, 2016 at 1:48 AM, Takashi Yamamoto
>> <yamamoto at midokura.com> wrote:
>>> On Thu, Mar 24, 2016 at 6:17 AM, Doug Wiegley
>>> <dougwig at parksidesoftware.com> wrote:
>>>> Migration script has been submitted, v1 is not going anywhere from
>> stable/liberty or stable/mitaka, so it’s about to disappear from master.
>>>> 
>>>> I’m thinking in this order:
>>>> 
>>>> - remove jenkins jobs
>>>> - wait for heat to remove their jenkins jobs ([heat] added to this
>>>> thread, so they see this coming before the job breaks)
>>> 
>>> magnum is relying on lbaasv1.  (with heat)
>> 
>> Is there anything blocking you from moving to v2?
> 
> A ticket was created for that: https://blueprints.launchpad.net/magnum/+spec/migrate-to-lbaas-v2 . It will be picked up by contributors once it is approved. Please give us sometimes to finish the work.

Approved.

>>>> - remove q-lbaas from devstack, and any references to lbaas v1 in
>> devstack-gate or infra defaults.
>>>> - remove v1 code from neutron-lbaas
>>>> 
>>>> Since newton is now open for commits, this process is going to get
>> started.
>>>> 
>>>> Thanks,
>>>> doug
>>>> 
>>>> 
>>>> 
>>>>> On Mar 8, 2016, at 11:36 AM, Eichberger, German
>> <german.eichberger at hpe.com> wrote:
>>>>> 
>>>>> Yes, it’s Database only — though we changed the agent driver in the
>> DB from V1 to V2 — so if you bring up a V2 with that database it should
>> reschedule all your load balancers on the V2 agent driver.
>>>>> 
>>>>> German
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 3/8/16, 3:13 AM, "Samuel Bercovici" <SamuelB at Radware.com> wrote:
>>>>>> 
>>>>>> So this looks like only a database migration, right?
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Eichberger, German [mailto:german.eichberger at hpe.com]
>>>>>> Sent: Tuesday, March 08, 2016 12:28 AM
>>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1 -
>> are weready?
>>>>>> 
>>>>>> Ok, for what it’s worth we have contributed our migration script:
>>>>>> https://review.openstack.org/#/c/289595/ — please look at this as
>> a
>>>>>> starting point and feel free to fix potential problems…
>>>>>> 
>>>>>> Thanks,
>>>>>> German
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 3/7/16, 11:00 AM, "Samuel Bercovici" <SamuelB at Radware.com>
>> wrote:
>>>>>> 
>>>>>>> As far as I recall, you can specify the VIP in creating the LB so
>> you will end up with same IPs.
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Eichberger, German [mailto:german.eichberger at hpe.com]
>>>>>>> Sent: Monday, March 07, 2016 8:30 PM
>>>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1 -
>> are weready?
>>>>>>> 
>>>>>>> Hi Sam,
>>>>>>> 
>>>>>>> So if you have some 3rd party hardware you only need to change
>> the
>>>>>>> database (your steps 1-5) since the 3rd party hardware will just
>>>>>>> keep load balancing…
>>>>>>> 
>>>>>>> Now for Kevin’s case with the namespace driver:
>>>>>>> You would need a 6th step to reschedule the loadbalancers with
>> the V2 namespace driver — which can be done.
>>>>>>> 
>>>>>>> If we want to migrate to Octavia or (from one LB provider to
>> another) it might be better to use the following steps:
>>>>>>> 
>>>>>>> 1. Download LBaaS v1 information (Tenants, Flavors, VIPs, Pools,
>>>>>>> Health Monitors , Members) into some JSON format file(s) 2.
>> Delete LBaaS v1 3.
>>>>>>> Uninstall LBaaS v1 4. Install LBaaS v2 5. Transform the JSON
>>>>>>> format file into some scripts which recreate the load balancers
>>>>>>> with your provider of choice —
>>>>>>> 
>>>>>>> 6. Run those scripts
>>>>>>> 
>>>>>>> The problem I see is that we will probably end up with different
>>>>>>> VIPs so the end user would need to change their IPs…
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> German
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 3/6/16, 5:35 AM, "Samuel Bercovici" <SamuelB at Radware.com>
>> wrote:
>>>>>>> 
>>>>>>>> As for a migration tool.
>>>>>>>> Due to model changes and deployment changes between LBaaS v1 and
>> LBaaS v2, I am in favor for the following process:
>>>>>>>> 
>>>>>>>> 1. Download LBaaS v1 information (Tenants, Flavors, VIPs, Pools,
>>>>>>>> Health Monitors , Members) into some JSON format file(s) 2.
>> Delete LBaaS v1 3.
>>>>>>>> Uninstall LBaaS v1 4. Install LBaaS v2 5. Import the data from 1
>>>>>>>> back over LBaaS v2 (need to allow moving from falvor1-->flavor2,
>>>>>>>> need to make room to some custom modification for mapping
>> between
>>>>>>>> v1 and v2
>>>>>>>> models)
>>>>>>>> 
>>>>>>>> What do you think?
>>>>>>>> 
>>>>>>>> -Sam.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Fox, Kevin M [mailto:Kevin.Fox at pnnl.gov]
>>>>>>>> Sent: Friday, March 04, 2016 2:06 AM
>>>>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1 -
>> are weready?
>>>>>>>> 
>>>>>>>> Ok. Thanks for the info.
>>>>>>>> 
>>>>>>>> Kevin
>>>>>>>> ________________________________________
>>>>>>>> From: Brandon Logan [brandon.logan at RACKSPACE.COM]
>>>>>>>> Sent: Thursday, March 03, 2016 2:42 PM
>>>>>>>> To: openstack-dev at lists.openstack.org
>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1 -
>> are weready?
>>>>>>>> 
>>>>>>>> Just for clarity, V2 did not reuse tables, all the tables it
>> uses are only for it.  The main problem is that v1 and v2 both have a
>> pools resource, but v1 and v2's pool resource have different attributes.
>> With the way neutron wsgi works, if both v1 and v2 are enabled, it will
>> combine both sets of attributes into the same validation schema.
>>>>>>>> 
>>>>>>>> The other problem with v1 and v2 running together was only
>> occurring when the v1 agent driver and v2 agent driver were both in use
>> at the same time.  This may actually have been fixed with some agent
>> updates in neutron, since that is common code.  It needs to be tested
>> out though.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Brandon
>>>>>>>> 
>>>>>>>>> On Thu, 2016-03-03 at 22:14 +0000, Fox, Kevin M wrote:
>>>>>>>>> Just because you had thought no one was using it outside of a
>> PoC doesn't mean folks aren''t using it in production.
>>>>>>>>> 
>>>>>>>>> We would be happy to migrate to Octavia. We were planning on
>> doing just that by running both v1 with haproxy namespace, and v2 with
>> Octavia and then pick off upgrading lb's one at a time, but the reuse
>> of the v1 tables really was an unfortunate decision that blocked that
>> activity.
>>>>>>>>> 
>>>>>>>>> We're still trying to figure out a path forward.
>>>>>>>>> 
>>>>>>>>> We have an outage window next month. after that, it could be
>>>>>>>>> about 6 months before we could try a migration due to
>> production
>>>>>>>>> load picking up for a while. I may just have to burn out all
>> the
>>>>>>>>> lb's switch to v2, then rebuild them by hand in a marathon
>>>>>>>>> outage :/
>>>>>>>>> 
>>>>>>>>> And then there's this thingy that also critically needs fixing:
>>>>>>>>> https://bugs.launchpad.net/neutron/+bug/1457556
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Kevin
>>>>>>>>> ________________________________________
>>>>>>>>> From: Eichberger, German [german.eichberger at hpe.com]
>>>>>>>>> Sent: Thursday, March 03, 2016 12:47 PM
>>>>>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1
>> - are weready?
>>>>>>>>> 
>>>>>>>>> Kevin,
>>>>>>>>> 
>>>>>>>>> If we are offering a migration tool it would be namespace ->
>>>>>>>>> namespace (or maybe Octavia since [1]) - given the limitations
>>>>>>>>> nobody should be using the namespace driver outside a PoC so I
>>>>>>>>> am a bit confused why customers can't self migrate. With 3rd
>>>>>>>>> party Lbs I would assume vendors proving those scripts to make
>>>>>>>>> sure their particular hardware works with those. If you indeed
>>>>>>>>> need a migration from LBaaS
>>>>>>>>> V1 namespace -> LBaaS V2 namespace/Octavia please file an RfE
>>>>>>>>> with your use case so we can discuss it further...
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> German
>>>>>>>>> 
>>>>>>>>> [1] https://review.openstack.org/#/c/286380
>>>>>>>>> 
>>>>>>>>> From: "Fox, Kevin M"
>>>>>>>>> <Kevin.Fox at pnnl.gov<mailto:Kevin.Fox at pnnl.gov>>
>>>>>>>>> Reply-To: "OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)"
>>>>>>>>> <openstack-dev at lists.openstack.org<mailto:openstack-
>> dev at lists.op
>>>>>>>>> enst
>>>>>>>>> a
>>>>>>>>> c
>>>>>>>>> k.org>>
>>>>>>>>> Date: Wednesday, March 2, 2016 at 5:17 PM
>>>>>>>>> To: "OpenStack Development Mailing List (not for usage
>> questions)"
>>>>>>>>> <openstack-dev at lists.openstack.org<mailto:openstack-
>> dev at lists.op
>>>>>>>>> enst
>>>>>>>>> a
>>>>>>>>> c
>>>>>>>>> k.org>>
>>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1
>> - are weready?
>>>>>>>>> 
>>>>>>>>> no removal without an upgrade path. I've got v1 LB's and there
>> still isn't a migration script to go from v1 to v2.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Kevin
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ________________________________
>>>>>>>>> From: Stephen Balukoff
>>>>>>>>> [sbalukoff at bluebox.net<mailto:sbalukoff at bluebox.net>]
>>>>>>>>> Sent: Wednesday, March 02, 2016 4:49 PM
>>>>>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1
>> - are weready?
>>>>>>>>> 
>>>>>>>>> I am also on-board with removing LBaaS v1 as early as possible
>> in the Newton cycle.
>>>>>>>>> 
>>>>>>>>> On Wed, Mar 2, 2016 at 9:44 AM, Samuel Bercovici
>> <SamuelB at radware.com<mailto:SamuelB at radware.com>> wrote:
>>>>>>>>> Thank you all for your response.
>>>>>>>>> 
>>>>>>>>> In my opinion given that UI/HEAT will make Mitaka and will have
>> one cycle to mature, it makes sense to remove LBaaS v1 in Newton.
>>>>>>>>> Do we want do discuss an upgrade process in the summit?
>>>>>>>>> 
>>>>>>>>> -Sam.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> From: Bryan Jones
>>>>>>>>> [mailto:jonesbr at us.ibm.com<mailto:jonesbr at us.ibm.com>]
>>>>>>>>> Sent: Wednesday, March 02, 2016 5:54 PM
>>>>>>>>> To:
>>>>>>>>> openstack-dev at lists.openstack.org<mailto:openstack-
>> dev at lists.ope
>>>>>>>>> nsta
>>>>>>>>> c
>>>>>>>>> k
>>>>>>>>> .org>
>>>>>>>>> 
>>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1
>> - are weready?
>>>>>>>>> 
>>>>>>>>> And as for the Heat support, the resources have made Mitaka,
>> with additional functional tests on the way soon.
>>>>>>>>> 
>>>>>>>>> blueprint:
>>>>>>>>> https://blueprints.launchpad.net/heat/+spec/lbaasv2-suport
>>>>>>>>> gerrit topic:
>>>>>>>>> https://review.openstack.org/#/q/topic:bp/lbaasv2-suport
>>>>>>>>> BRYAN M. JONES
>>>>>>>>> Software Engineer - OpenStack Development
>>>>>>>>> Phone: 1-507-253-2620<tel:1-507-253-2620>
>>>>>>>>> E-mail: jonesbr at us.ibm.com<mailto:jonesbr at us.ibm.com>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ----- Original message -----
>>>>>>>>> From: Justin Pomeroy
>>>>>>>>> <jpomero at linux.vnet.ibm.com<mailto:jpomero at linux.vnet.ibm.com>>
>>>>>>>>> To:
>>>>>>>>> openstack-dev at lists.openstack.org<mailto:openstack-
>> dev at lists.ope
>>>>>>>>> nsta
>>>>>>>>> c
>>>>>>>>> k
>>>>>>>>> .org>
>>>>>>>>> Cc:
>>>>>>>>> Subject: Re: [openstack-dev] [Neutron][LBaaS]Removing LBaaS v1
>> - are we ready?
>>>>>>>>> Date: Wed, Mar 2, 2016 9:36 AM
>>>>>>>>> 
>>>>>>>>> As for the horizon support, much of it will make Mitaka.  See
>> the blueprint and gerrit topic:
>>>>>>>>> 
>>>>>>>>> https://blueprints.launchpad.net/horizon/+spec/horizon-lbaas-
>> v2-
>>>>>>>>> ui
>>>>>>>>> https://review.openstack.org/#/q/topic:bp/horizon-lbaas-v2-ui,n,
>>>>>>>>> z
>>>>>>>>> 
>>>>>>>>> - Justin
>>>>>>>>> 
>>>>>>>>> On 3/2/16 9:22 AM, Doug Wiegley wrote:
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> A few things:
>>>>>>>>> 
>>>>>>>>> - It's not proposed for removal in Mitaka. That patch is for
>> Newton.
>>>>>>>>> - HEAT and Horizon are planned for Mitaka (see
>>>>>>>>> neutron-lbaas-dashboard for the latter.)
>>>>>>>>> - I don't view this as a "keep or delete" question. If
>>>>>>>>> sufficient folks are interested in maintaining it, there is a
>>>>>>>>> third option, which is that the code can be maintained in a
>>>>>>>>> separate repo, by a separate team (with or without the current
>>>>>>>>> core team's blessing.)
>>>>>>>>> 
>>>>>>>>> No decisions have been made yet, but we are on the cusp of some
>> major maintenance changes, and two deprecation cycles have passed.
>> Which path forward is being discussed at today's Octavia meeting, or
>> feedback is of course welcomed here, in IRC, or anywhere.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> doug
>>>>>>>>> 
>>>>>>>>> On Mar 2, 2016, at 7:06 AM, Samuel Bercovici
>> <SamuelB at Radware.com<mailto:SamuelB at radware.com>> wrote:
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> I have just notices the following change:
>> https://review.openstack.org/#/c/286381 which aims to remove LBaaS v1.
>>>>>>>>> Is this planned for Mitaka or for Newton?
>>>>>>>>> 
>>>>>>>>> While LBaaS v2 is becoming the default, I think that we should
>> have the following before we replace LBaaS v1:
>>>>>>>>> 1.      Horizon Support - was not able to find any real
>> activity on it
>>>>>>>>> 2.      HEAT Support - will it be ready in Mitaka?
>>>>>>>>> 
>>>>>>>>> Do you have any other items that are needed before we get rid
>> of LBaaS v1?
>>>>>>>>> 
>>>>>>>>> -Sam.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>> ________________________________________________________________
>>>>>>>>> ____ _ _ ____ OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-
>> r
>>>>>>>>> eque s t @lists.openstack.org>?subject:unsubscribe
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> de
>>>>>>>>> v
>>>>>>>>> 
>>>>>>>>> 
>> ________________________________________________________________
>>>>>>>>> ____ _ _ ____ OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-
>> request at lists.openstack.org?subject:unsubscribe<mailto:
>>>>>>>>> O penStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> de
>>>>>>>>> v
>>>>>>>>> 
>>>>>>>>> 
>> ________________________________________________________________
>>>>>>>>> ____ _ _ ____ OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-
>> request at lists.openstack.org?subject:unsubscribe<mailto:
>>>>>>>>> O penStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> de
>>>>>>>>> v
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>> ________________________________________________________________
>>>>>>>>> ____ _ _ ____ OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-
>> request at lists.openstack.org?subject:unsubscribe<ht
>>>>>>>>> tp:/ / O
>>>>>>>>> penStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> de
>>>>>>>>> v
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Stephen Balukoff
>>>>>>>>> Principal Technologist
>>>>>>>>> Blue Box, An IBM Company
>>>>>>>>> www.blueboxcloud.com<http://www.blueboxcloud.com>
>>>>>>>>> sbalukoff at blueboxcloud.com<mailto:sbalukoff at blueboxcloud.com>
>>>>>>>>> 206-607-0660 x807
>>>>>>>>> 
>> ________________________________________________________________
>>>>>>>>> ____ _ _ ____ OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> de
>>>>>>>>> v
>>>>>>>>> 
>>>>>>>>> 
>> ________________________________________________________________
>>>>>>>>> ____ _ _ ____ OpenStack Development Mailing List (not for usage
>>>>>>>>> questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> de
>>>>>>>>> v
>>>>>>>> 
>>>>>>>> 
>> _________________________________________________________________
>>>>>>>> _____ _ ___ OpenStack Development Mailing List (not for usage
>>>>>>>> questions)
>>>>>>>> Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> dev
>>>>>>>> 
>>>>>>>> 
>> _________________________________________________________________
>>>>>>>> _____ _ ___ OpenStack Development Mailing List (not for usage
>>>>>>>> questions)
>>>>>>>> Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> dev
>>>>>>>> 
>>>>>>>> 
>> _________________________________________________________________
>>>>>>>> _____ _ ___ OpenStack Development Mailing List (not for usage
>>>>>>>> questions)
>>>>>>>> Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-
>> dev
>>>>>>> 
>> __________________________________________________________________
>>>>>>> _____ ___ OpenStack Development Mailing List (not for usage
>>>>>>> questions)
>>>>>>> Unsubscribe:
>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>> 
>> __________________________________________________________________
>>>>>>> _____ ___ OpenStack Development Mailing List (not for usage
>>>>>>> questions)
>>>>>>> Unsubscribe:
>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>> 
>> ___________________________________________________________________
>>>>>> _______ OpenStack Development Mailing List (not for usage
>>>>>> questions)
>>>>>> Unsubscribe:
>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>> 
>> ___________________________________________________________________
>>>>>> _______ OpenStack Development Mailing List (not for usage
>>>>>> questions)
>>>>>> Unsubscribe:
>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> 
>> ____________________________________________________________________
>>>>> ______ OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>>> 
>>>> 
>> _____________________________________________________________________
>>>> _____ OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> 
>> ______________________________________________________________________
>>> ____ OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> _______________________________________________________________________
>> ___
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-
>> request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list