[openstack-dev] [Quantum][LBaaS] Service type framework: todo & cleanup
Salvatore Orlando
sorlando at nicira.com
Fri May 3 13:35:14 UTC 2013
The time for the meeting looks good to me. Thanks Eugene.
I do not want to get into the merit of whether a specific proposal here is
right or wrong; I would only stress the concept that what we're trying to
address here is how to support multiple drivers in the LB plugin with a
solution that can be reused for doing the same thing in the Fw and VPN
plugins.
If discussing and solving service insertion/chaining is a prerequisite
here, then we'll go for it. However I'm not convinced we need to address
this now.
Of course, I'm not saying we should not discuss it! I just to keep this
discussion focused on the goal initially set by Eugene.
Salvatore
On 3 May 2013 15:23, Eugene Nikanorov <enikanorov at mirantis.com> wrote:
> Hi Sumit,
>
> I see that meaning of terms service-type/service type provider is still
> floating and the meaning I've proposed went to service-name/catategory. I'd
> object against introducing more notions here.
>
> Agree about many-to-many relationship between ServiceOffering and
> ServiceTypeProvider.
> But I don't get the purpose of ServiceOffering entity in the form you're
> proposing. It looks more like a service chain with it's ordered list of
> services.
> To me ServiceOffering is services grouped by additional parameter like
> insertion mode or quality (as I suggested in last email)
>
> Also, I'd like to propose a time for meeting: 9 AM PST Monday, 6th May.
>
> Thanks,
> Eugene.
>
>
> On Fri, May 3, 2013 at 9:45 AM, Sumit Naiksatam <sumitnaiksatam at gmail.com>wrote:
>
>> Hi All, Here is an enhanced simplified model. :-) I was brainstorming
>> with Kanzhe and he had some great input. This should help to satisfy
>> the multi-service appliance use case as well. The wiki is updated
>> (
>> https://wiki.openstack.org/wiki/Quantum/ServiceTypeFramework#Simplified_model.2Fframework
>> )
>> but here is a snapshot:
>>
>> Service Types:
>> Each service-type is a unique combination of the following:
>> String: Uuid (generated)
>> String: Service-name/category. E.g. LB, FW, etc. (required)
>> String: Insertion Model (required)
>> String: Service Type Provider: ServiceType:DriverClass (required),
>> does not need to be visible to the tenant
>> String: Service Quality (optional) E.g. Gold, Silver, etc. default:
>> Default
>> String: Vendor (optional)
>> Srting: Version (optional)
>>
>> * Service Quality is a pre-defined list of strings
>> * Service-name/categories is a pre-defined list of strings
>> * Insertion Model is a pre-defined list of strings
>>
>> Service Offerings:
>> A Service offering is an ordered list of one or more service-types.
>> This accounts for the use cases for discrete services and also
>> multi-service appliances.
>> String: Uuid (generated)
>> String: Service-offering-name (required)
>> List: service-types (required)
>>
>> E.g. [LB-service-type-uuid]
>> This is a service offering where a user can choose the specific
>> Loadbalancer service. The plugin and driver for this Loadbalancer
>> service are captured in the service-type identified by
>> LB-service-type-uuid.
>>
>> E.g. [LB-service-type-uuid, FW-service-type-uuid]
>> This is a service offering where the provider offers services which
>> can be potentially realized on a multi-service appliance. As before,
>> the plugin and driver each of the services are captured in their
>> respective service-type definitions.
>>
>> Resources' relationship:
>> 1 Service-type --> [1..many] Service-offerings
>> 1 Service-offering --> [1..many] Service-types
>>
>> Workflow:
>> 1. User queries the list of service-types/offerings
>> 2. User chooses one service-offering. The plugin/backend knows to map
>> this to a particular provider driver class to realize this resource.
>>
>> (if you reached this far, thanks for sticking with this, and apologies
>> for the long email!)
>>
>> Thanks,
>> ~Sumit.
>>
>> On Thu, May 2, 2013 at 3:54 AM, Salvatore Orlando <sorlando at nicira.com>
>> wrote:
>> >
>> > Hi again,
>> >
>> > It's good we're fleshing out options a little bit better.
>> > I will take a better look at Sumit's alternative proposal. At a first
>> glance it looks like it's just making the service type flat, removing the
>> need for grouping, but I'd rather have a better look at that.
>> >
>> > And then again, I would pledge to adhere to the initial goal fixed by
>> Eugene: allowing the selection of a specific driver, in a vendor-agnostic
>> way (I added the last bit).
>> > Service insertion models can come later in the process. I don't think
>> what we currently have in the source code tree would leverage them anyway.
>> >
>> > Salvatore
>> >
>> > On 2 May 2013 08:42, Eugene Nikanorov <enikanorov at mirantis.com> wrote:
>> >>
>> >> Salvatore, Kanzhe,
>> >>
>> >> I don't mind renaming ServiceGroup to ServiceOffering especially if it
>> makes more sense to you :)
>> >> I've updated the wiki.
>> >
>> >
>> > As long as it's just naming but we mean the same thing, it's fine.
>> > the problem would be if we actually mean two different things!
>> >
>> >>
>> >> Regarding the workflow: I specifically made this a two step process to
>> avoid taking decision at plugin (which could be additional complex problem)
>> and put it onto User.
>> >> Vendor choosing at plugin is analog of scheduling and it seems that
>> we're going to have just too much of flexibility here.
>> >
>> >
>> > To which argument regarding the workflow are you referring? The one
>> about allowing the plugin to choose among multiple options?
>> > If yes, I think yours is the right the way to go at the moment.
>> >
>> >
>> >>
>> >> Another solution to this could be Sumit's suggestion where we merge
>> ServiceTypeProvider with ServiceOffering parameters.
>> >> In fact as a first implementation step, I'd just leave
>> ServiceTypeProvider as simple as just a pointer to driver class.
>> >
>> >
>> > Sumit is actually proposing a sort of linearization. This flat model is
>> ok, but I think allowing for grouping might be better since many people are
>> already working on service appliance which offer multiple advanced services.
>> > Also, we have committed to split L3 functionality from the core plugin.
>> This means that for Havana we may add also a L3 service provider to this
>> list.
>> > And at the end of the day, a service offering with a single service
>> provider is probably the same thing.
>> >
>> >>
>> >>
>> >> > Conversely, if we go for directly specifying the service provider
>> type when creating a LB resource (see wiki page), then one will always
>> >> > be telling to the plugin which driver should be used.
>> >> > With this approach there would be no space for this use case, and
>> probably also the concept of service group will become irrelevant.
>> >> My impression was that ServiceOffering is grouping of services around
>> additional parameters such as insertion mode or service quality.
>> >> So to me it doesn't seem that ServiceOffering will be irrelevant in
>> the case you're talking about.
>> >
>> >
>> > Cool. I just wanted to make sure of that
>> >>
>> >>
>> >> Thanks,
>> >> Eugene.
>> >>
>> >>
>> >>
>> >> On Thu, May 2, 2013 at 3:50 AM, Sumit Naiksatam <
>> sumitnaiksatam at gmail.com> wrote:
>> >>>
>> >>> I too feel that we could potentially simplify the model. Here is a
>> potentially simpler mode/workflow:
>> >>>
>> https://wiki.openstack.org/wiki/Quantum/ServiceTypeFramework#Simplified_model.2Fframework
>> >>>
>> >>> Please let me know what you guys think.
>> >>>
>> >>> Thanks,
>> >>> ~Sumit.
>> >>>
>> >>>
>> >>> On Wed, May 1, 2013 at 4:38 PM, Kanzhe Jiang <
>> kanzhe.jiang at bigswitch.com> wrote:
>> >>>>
>> >>>> Hi Eugene,
>> >>>>
>> >>>> Thank you for the serviceType proposal. ServiceType,
>> ServiceTypeProvider, and InsertionMode make lots of sense. However, I have
>> a hard time to capture the meaning of ServiceGroup.
>> >>>> Salvatore's suggestion seems to be more that just a name change if I
>> understood correctly. ServiceOffering is a top-level object that consists
>> of serviceType, serviceTypeProvider and insertionMode. The user/tenant
>> workflow then becomes a one-step process, picking a serviceOffering from an
>> available list. A logic service is then created based on the selected
>> serviceOffering.
>> >>>>
>> >>>> Kanzhe
>> >>>>
>> >>>>
>> >>>> On Wed, May 1, 2013 at 4:12 PM, Salvatore Orlando <
>> sorlando at nicira.com> wrote:
>> >>>>>
>> >>>>> Youcef,
>> >>>>>
>> >>>>> I seem to recall you already made this point a while ago.
>> >>>>> I've tried to look back at old email threads, and it seems the use
>> case for a scenario like the one you describe would be when a given service
>> type/group/offering can be either be realized to provider 'a' or 'b', in
>> this case it would be the plugin to decide which driver to use, according
>> to some plugin-specific logic that is beyond the scope of this discussion.
>> >>>>> Note that this is different from the case where one chooses between
>> vendor A and B by specifying different service offering.
>> >>>>>
>> >>>>> If I got your use case correctly, then I'm ok with designing the
>> feature in this way (actually I would have something like "LB":
>> [VendorXDriver, VendorYDriver]).
>> >>>>>
>> >>>>> Conversely, if we go for directly specifying the service provider
>> type when creating a LB resource (see wiki page), then one will always be
>> telling to the plugin which driver should be used.
>> >>>>> With this approach there would be no space for this use case, and
>> probably also the concept of service group will become irrelevant.
>> >>>>>
>> >>>>> Salvatore
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On 1 May 2013 23:09, Youcef Laribi <Youcef.Laribi at eu.citrix.com>
>> wrote:
>> >>>>>>
>> >>>>>> Eugene, Salvatore,
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> To clarify my understanding of the concept of ServiceGroup (or
>> ServiceOffering to use Salvatore’s terminology), are you allowing for the
>> same ServiceType (e.g. LB) to be offered by multiple vendors (e.g a
>> ServiceOffering could be { “LB”:”LBVendorXDriver”, “FW”:”FWVendorADriver”,
>> “LB”:”LBVendorYDriver”} ? And in this case is “step 2” in the “workflow for
>> a user” section about choosing one vendor for each service type if there is
>> more than one?
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thanks
>> >>>>>>
>> >>>>>> Youcef
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> From: Salvatore Orlando [mailto:sorlando at nicira.com]
>> >>>>>> Sent: Wednesday, May 01, 2013 2:19 PM
>> >>>>>> To: OpenStack Development Mailing List
>> >>>>>> Subject: Re: [openstack-dev] [Quantum][LBaaS] Service type
>> framework: todo & cleanup
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Eugene,
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> I quickly went through your wiki and I think most of it makes
>> total sense to me.
>> >>>>>>
>> >>>>>> I would just consider renaming "Service Groups" to "Service
>> Offering" because it's less ambiguous.
>> >>>>>>
>> >>>>>> This concept, at least for reading, is made available to tenant.
>> Exposing it as an "Offering" of services that can be deployed makes more
>> sense.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> I see you also have two todos: for the first, I'd say let's keep
>> it open to future implementation with multiple plugins (if we don't do
>> that, we might end up in a pickle), for the second, I'd say it's a cool
>> feature, but probably not worth the hassle.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> I would keep the insertion model out for now; not because I do not
>> agree with it, but because we might incur the same problem we had with
>> Service Type, and also the LB API.
>> >>>>>>
>> >>>>>> Since we did it without a backing implementation, we then had to
>> tweak it. We can always augment the service type once the service
>> insertion/chaining framework will be ready.
>> >>>>>>
>> >>>>>> And as you know, steering the discussion onto service insertion
>> kills productivity.
>> >>>>>>
>> >>>>>> I like the idea anyway - and we already proposed to include it
>> into the servicetypespec in Grizzly - for instance it could be used to say
>> whether the corresponding implementation is backed by a service VM, an ADC
>> with context, or whether it's a physical device attached at L2, and so on.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On the REST call dispatching section, I don't see anything wrong
>> with your idea, the only change I would propose is that when you create a
>> VIP you specify the service offering, not the service provider. You do not
>> want to expose that kind of detail.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On 1 May 2013 21:32, Eugene Nikanorov <enikanorov at mirantis.com>
>> wrote:
>> >>>>>>
>> >>>>>> Salvatore, folks,
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Friday also works for me. To have something concrete to discuss,
>> I've created a page with a summary of the ideas in my first email:
>> >>>>>>
>> >>>>>> https://wiki.openstack.org/wiki/Quantum/ServiceTypeFramework
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> May be if everyone is ok with this proposal then our meeting will
>> be short (if we even need one) :)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>> Eugene.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On Wed, May 1, 2013 at 6:15 PM, Salvatore Orlando <
>> sorlando at nicira.com> wrote:
>> >>>>>>
>> >>>>>> Eugene,
>> >>>>>>
>> >>>>>> I did not see this message and I replied on the other post.
>> >>>>>>
>> >>>>>> I will not be available tomorrow between 6AM PST and 1PM PST.
>> Friday would work for me, but honestly I think we might want to flesh out
>> ideas and terminology a little bit better offline before going into a
>> meeting; I think having a clearer picture will boost the productivity of
>> the meeting.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> You've set the discussion on the right tracks with your first
>> email.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Salvatore
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On 1 May 2013 15:04, Eugene Nikanorov <enikanorov at mirantis.com>
>> wrote:
>> >>>>>>
>> >>>>>> So folks,
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> How about meet tomorrow, Thursday, 10 AM PST at #quantum-lbaas and
>> discuss these things?
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>> Eugene.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On Tue, Apr 30, 2013 at 8:31 PM, Sumit Naiksatam <
>> sumitnaiksatam at gmail.com> wrote:
>> >>>>>>
>> >>>>>> Hi Eugene,
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thanks for bringing this up. We tried to discuss some of these in
>> the "services chaining, insertion, and steering" session during the summit.
>> Based on the discussion and feedback (and also from implementing a
>> prototype involving multiple service "types") my responses inline.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> ~Sumit.
>> >>>>>>
>> >>>>>> On Tue, Apr 30, 2013 at 8:34 AM, Eugene Nikanorov <
>> enikanorov at mirantis.com> wrote:
>> >>>>>>
>> >>>>>> Hi folks,
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> One of the major features that we need to implement during Havana
>> (H-1 at best) is true flexible multivendor support.
>> >>>>>>
>> >>>>>> There could be different ways of giving tenant ability to choose
>> vendor of requested loadbalancer, but we need to evaluate how service types
>> could help us on this way.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> The reason I'm writing this is that I'd like to make some
>> terminology cleanup and set up key design points to discuss and move
>> forward.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Let's start with what we have on service types today:
>> >>>>>>
>> >>>>>> 1) Description of the feature:
>> https://wiki.openstack.org/wiki/Quantum/ServiceInsertion
>> >>>>>>
>> >>>>>> 2) Code:
>> https://github.com/openstack/quantum/blob/master/quantum/db/servicetype_db.py
>> >>>>>>
>> >>>>>> 3) seems to be framework is not used (and doesn't have respective
>> CLI code) so changing it would not be painfull if we want to.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> I'd like to set service insertion problem aside and focus on
>> service types as a way to choose vendor.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> So, on terminology side we have the following enitities (from code
>> and docs):
>> >>>>>>
>> >>>>>> - Service Type - defines a set of service providers
>> (plugins:drivers)
>> >>>>>>
>> >>>>>> - Service Definition - defines one service provider
>> >>>>>>
>> >>>>>> - Service Class - a type of service (?!) (LB, FW, VPN)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> 1) So, first of all terminology seems to be quite confusing,
>> especially word "type" is overloaded.
>> >>>>>>
>> >>>>>> I think we need to propose better names for the these entities
>> just to simplify our thinking about the integration with services.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> So, my proposal would be:
>> >>>>>>
>> >>>>>> - Service Type - LB, FW, VPN
>> >>>>>>
>> >>>>>> - Service Type Provider - defines one service provider (former
>> Service Definition) (alt: Service Type Definition)
>> >>>>>>
>> >>>>>> - Service Group - what used to be Service Type. (alt: Service Set,
>> ??)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Sumit: How about a top level "service_type" class (consistent with
>> current definition) with attributes like:
>> >>>>>>
>> >>>>>> - Category, e.g. LB, FW, VPN
>> >>>>>>
>> >>>>>> - Vendor, e.g. XYZ
>> >>>>>>
>> >>>>>> - Insertion Model, e.g. L3, L2, Bump-in-the-wire, Tap (I know you
>> mentioned that you want to keep this aside for now, but I am stating it
>> here for completeness)
>> >>>>>>
>> >>>>>> - Provider, name of the class which implements this service (one
>> per type as you suggest later)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Until agreed, I'll use old terminology.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> 2) The relationship between Service Types and Service Definitions.
>> >>>>>>
>> >>>>>> It is not obvious (nowhere stated or enforced) but Service Type
>> must have just one Service Definition of the same Service Class,
>> >>>>>>
>> >>>>>> otherwise, Service Type is useless for API call dipatching. E.g.
>> there should be 1:1 mapping between Service Type and Service Definition for
>> certain Service Class.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Sumit: We also need a base service class definition that can be
>> used across services. This should support a contract for creating multiple
>> logical instances of the service (e.g. one more logical Firewalls
>> devices/instances as requested by a tenant). It should also support the
>> attributes required to help service insertion (since the service
>> implementor knows how the service will attach to the network).
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> 3) API calls dispatching. The most important question: whether or
>> not to support multiple plugins per Service Class.
>> >>>>>>
>> >>>>>> In other words, whether to have several Service plugins (several
>> lbaas plugins, for ex) loaded simultaneosly.
>> >>>>>>
>> >>>>>> My opinion is that we don't need that as multi-vendor support
>> could be implemented more simple plugin-side drivers.
>> >>>>>>
>> >>>>>> To make this shorter, I'll skip explanation for now.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Sumit: A complementary part of this discussion is the ability for
>> one plugin to support multiple services. We discussed this briefly during
>> the summit, and I believe we agreed to fix this in the current
>> implementation.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> So in order to move forward I'd suggest we have a meeting with the
>> following agenda:
>> >>>>>>
>> >>>>>> 1. Terminology
>> >>>>>>
>> >>>>>> 2. Service type framework refactoring.
>> >>>>>>
>> >>>>>> 3. API calls dispatching: hints and control flow.
>> >>>>>>
>> >>>>>> I'll prepare some suggestions/diagrams about this.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> We could have a meeting someday this week at 8:00 - 10:00 AM PST,
>> late evening PST will also work for me.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Sumit: 8 - 10 AM PST sounds good to me! ;-)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>> Eugene.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> OpenStack-dev mailing list
>> >>>>>> OpenStack-dev at lists.openstack.org
>> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> OpenStack-dev mailing list
>> >>>>>> OpenStack-dev at lists.openstack.org
>> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> OpenStack-dev mailing list
>> >>>>>> OpenStack-dev at lists.openstack.org
>> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> OpenStack-dev mailing list
>> >>>>>> OpenStack-dev at lists.openstack.org
>> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> OpenStack-dev mailing list
>> >>>>> OpenStack-dev at lists.openstack.org
>> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Kanzhe Jiang
>> >>>> MTS at BigSwitch
>> >>>>
>> >>>> _______________________________________________
>> >>>> OpenStack-dev mailing list
>> >>>> OpenStack-dev at lists.openstack.org
>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> OpenStack-dev mailing list
>> >>> OpenStack-dev at lists.openstack.org
>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>
>> >>
>> >>
>> >> _______________________________________________
>> >> OpenStack-dev mailing list
>> >> OpenStack-dev at lists.openstack.org
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >
>> >
>> > _______________________________________________
>> > OpenStack-dev mailing list
>> > OpenStack-dev at lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130503/7228b519/attachment.html>
More information about the OpenStack-dev
mailing list