<div dir="ltr">Folks,<div><br></div><div>Initial implementation is here: <a href="https://review.openstack.org/#/c/105982/">https://review.openstack.org/#/c/105982/</a></div><div>It's pretty much complete (in terms of code parts) but may require some adjustments and of course fixes.</div>
<div><br></div><div>I'm working on the client to test this end-to-end.</div><div><br></div><div>Thanks,</div><div>Eugene.</div><div><br></div><div>P.S. Almost got it under 1k lines!</div><div><br></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Thu, Jul 17, 2014 at 7:36 AM, Stephen Balukoff <span dir="ltr"><<a href="mailto:sbalukoff@bluebox.net" target="_blank">sbalukoff@bluebox.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Salvatore!<div><br></div><div>Thank you for reading through my book-length e-mail and responding to all my points!<br><div><br></div><div>Unfortunately, I have more responses for you, inline:</div><div><div class="gmail_extra">
<br><div class="gmail_quote"><div class="">On Wed, Jul 16, 2014 at 4:22 PM, Salvatore Orlando <span dir="ltr"><<a href="mailto:sorlando@nicira.com" target="_blank">sorlando@nicira.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi Stephen,<div><br></div><div>Thanks for your exhaustive comments!</div>
</div></blockquote><div><br></div>
</div><div>I'm always happy to exhaust others with my comments. ;)</div><div class=""><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div><div class="gmail_extra">
<div class="gmail_quote"><div><div><br></div></div><div>I think your points are true and valid for most cloud operators; besides the first all the point you provided indeed pertain operators and vendors. However you can't prove, I think, the opposite - that is to say that no cloud operator will find multi-service flavors useful. At the end of the day Openstack is always about choice - in this case the choice of having flavours spanning services or flavours limited to a single service.</div>
<div>This discussion however will just end up slowly drifting into the realm of the theoretical and hypotethical and therefore won't bring anything good to our cause. Who know, in a few post we might just end up calling godwin's law!</div>
</div></div></div></div></div></blockquote><div><br></div></div><div>That's certainly true. But would you be willing to agree that both the model and logic behind single-service_type flavors is likely to be simpler to implement, troubleshoot, and maintain than multi-service_type flavors?</div>
<div><br></div><div>If you agree, then I would say: Let's go with single-service_type flavors for now so that we can actually get an implementation done by Juno (and thus free up development that is currently being blocked by lack of flavors), and leave the more complicated multi-service_type flavors for some later date when there's a more obvious need for them.</div>
<div><br></div><div>For what it's worth, I'm not against multi-service_type flavors if someone can come up with a good usage scenario that is best solved using the same. But I think it's more complication than we want or need right now, and shooting for it now is likely to ensure we wouldn't get flavors in time for Juno.</div>
<div class="">
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra">
<div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div>There are other considerations which could be made, but since they're dependent on features which do not yet exist (NFV, service insertion, chaining and steering) I think there is no point in arguing over it.<br>
</div></div></blockquote><div><br></div></div><div>Agreed. Though, I don't think single-service flavors paint us into a corner here at all. Again, things get complicated enough when it comes to service insertion, chaining, steering, etc. that what we'll really need at that point is actual orchestration. Flavors alone will not solve these problems, and orchestration can work with many single-service flavors to provide the illusion of multi-service flavors.</div>
</div></div></div></blockquote><div><br></div></div><div>Don't take it the wrong way - but this is what I mean by "theoretical and hypothetical". I agree with you. I think that's totally possible. But there are so many pieces which are yet missing from the puzzle that this discussion is probably worthless. Anyway, I started it, and I'm the one to be punished for it!</div>
<div>
<div></div></div></div></div></div></blockquote><div><br></div></div><div>Hah! Indeed. Ok, I'll stop speculating down that path for now, eh. ;)</div><div class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra">
<div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>In conclusion I think the idea makes sense, and is a minor twist in the current design which should not either make the feature too complex neither prevent any other use case for which the flavours are being conceived. For the very same reason however, it is worth noting that this is surely not an aspect which will cause me or somebody else to put a veto on this work item.</div>
</div></blockquote><div><br></div></div><div>I don't think this is a minor twist in the current design, actually:</div><div>* We'll have to deal with cases like the above where no valid service profiles can be found for a given kind of flavor (which we can avoid entirely if a flavor can have service profiles valid for only one kind of service).</div>
</div></div></div></blockquote><div><br></div></div><div>Point taken, but does not require a major change to the design since a service flavour like this should probably be caught by a validation routine. Still you'd need more pervasive validation in different points of the API.</div>
</div></div></div></blockquote><div><br></div></div><div>... which sounds like significantly more complication to me. But at this point, we're arguing over what a "minor twist" is, which is not likely to lead to anywhere useful...</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote">
<div>* When and if tags/capabilities/extensions get introduced, we would need to provide an additional capabilities list on the service profiles in order to be able to select which service profiles provide the capabilities requested.</div>
</div></div></div></div></div></blockquote><div><br></div></div><div>Might be... but I don't see how that would be worse with multiple service types, especially if profiles are grouped by type.</div></div></div></div>
</blockquote><div><br></div></div><div>Presumably, with single-service_type flavors, all service profiles associated with the flavor should be capable of providing all the features advertised as being provided by the flavor (first in the 'description' and possibly later programmatically via an extensions list). This means we don't have to check to see whether a service profile associated with the flavor can provide for all the extensions advertised in the flavor description because by creating the association, the operator is implying it can.</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote">
<div>* The above point makes things much more complicated when it comes to scheduling algorithms for choosing which service profile to use when multiple can meet the need for a given service. What does 'weight' mean if all but two low-weight service profiles get eliminated as not suitable?</div>
</div></div></div></div></div></blockquote><div><br></div></div><div>I'm really not following you. Sorry about my ignorance. Isn't this something you would need to address even in the case you have a flavour with multiple service profiles all for the same service type, as the spec currently mandates?</div>
</div></div></div></blockquote><div><br></div></div><div>No, I'm probably simply not making my point clearly. What I mean is probably best illustrated by an example:</div><div><br></div><div>Suppose I'm using service profile weights in my scheduling algorithm to determine which service profile will get used when a user requests an instance of an advanced service using the flavor framework.</div>
<div><br></div><div>For a single-service_type flavor, let's say I have the following service profiles:</div><div><br></div><div>"Profile A" => weight 30</div><div>"Profile B" => weight 50</div>
<div>"Profile C" => weight 20</div><div><br></div><div>I don't have to mention the type of service in the above profiles because it's implied by the service_type attribute of the flavor. Given the above weights, it's clear to see that if a user requests a service instance, there's a 50% chance it'll be a "Profile B" instance. Then, later, when the Operator looks at the relative split of services as deployed by users, she should expect to see a "30 / 50 / 20" percentage split. A "5 / 90 / 5" split would be a solid indication that "something is wrong" that the operator needs to look into. (Maybe "Profile A" deployments fail silently 75% of the time, or maybe users have figured out that they hate "profile C" instance types and keep deploying / destroying instances until they land on a "profile B" instance.)</div>
<div><br></div><div>Now, doing the same exercise for a multi-service_type flavor, let's say we have the following service profiles:</div><div><br></div><div>"Profile X" => service_types = [LBaaS, VPNaaS], weight = 30</div>
<div>"Profile Y" => service_types = [LBaaS, VPNaaS], weight = 50</div><div>"Profile Z" => service_types = [FWaaS, VPNaaS], weight = 20</div><div><br></div><div>Given the above:</div><div>* If a user tries to deploy an LBaaS instance, it's less obvious that the instance will have a 62.5% chance of ending up on Profile Y, and a 37.5% chance of ending up on Profile X.</div>
<div>* It's also less obvious that even though "Profile Z" has a low weight, if the user wants a FWaaS instance, there's a 100% chance it'll end up on Profile Z.</div><div>* From the Operator's perspective, it's no longer obvious that a "5 / 90 / 5" split of services on this flavor indicates "something is wrong." In fact, it's much more difficult to determine this.</div>
<div><br></div><div>Does this example make sense to you?</div><div class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div class="gmail_extra">
<div class="gmail_quote"><div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Another aspect to consider is how the flavours will work when the advanced service type they refer to is not consumable through the neutron API, which would be the case with an independent load balancing API endpoint. But this is probably another story.</div>
</div></blockquote><div><br></div></div><div>As far as I'm aware, flavors will only ever apply to advanced services consumable through the Neutron API. If this weren't the case, what's the point of having a flavor describing the service at all? If you're talking about Octavia here-- well, our plan is to have Octavia essentially be an other load balancer vendor, interfaced through a driver in the Neutron LBaaS extension. (This is also why so many developers interested in seeing Octavia come to light are spending all their time right now improving Neutron LBaaS-- we want it to be feature-rich enough in Juno for us to be able to actually do interesting things with Octavia without having to resort to building our own independent API endpoint if at all possible.) If Octavia ever splits off and has its own API endpoint, it would need to implement something like the Neutron flavor framework itself-- and in this case, what we decide to do there should not affect the Neutron flavor framework at all (or visa versa).</div>
</div></div></div></div></div></blockquote><div><br></div></div><div>This is a bit off topic, and again I am at fault for driving you there. The thing is that I've spoken with several people, and they've all got different opinions regarding where Load Balancing should go in the medium term. These range from a completely standalone service to what you've just mentioned here.</div>
<div>My opinion, which is probably irrelevant, is that I'm supportive of the load balancing effort evolving independently from the rest of neutron and staying loosely coupled with it. However, I believe that there should a single API endpoint for all the network services. In other words a single neutron API offering all service to consumer and different standalone services providing those service at control/data plane.</div>
</div></div></div></blockquote><div><br></div></div><div>Eh-- let's cross that bridge when we get there, eh. I think it will become more obvious as projects like Octavia gain maturity whether they should be split off and become completely independent, be loosely coupled, or simply remain a "vendor" of Neutron LBaaS. :) How "flavors" get handled in these scenarios is part of that discussion-- but that discussion probably isn't relevant right now.</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote">
<div><br></div><div>If you want to provide the illusion of two different top-level services / API endpoints having the same "flavor," then I would say, "that's what orchestration is for."</div></div>
</div></div></div></div></blockquote><div><br></div></div><div>Totally agree on this point. </div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div><div><div class="gmail_extra">
<div class="gmail_quote"><div>
<br></div><div>On Tue, Jul 15, 2014 at 2:07 PM, Eugene Nikanorov <span dir="ltr"><<a href="mailto:enikanorov@mirantis.com" target="_blank">enikanorov@mirantis.com</a>></span> wrote:<div><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Hi Stephen,<div><br></div><div>So, as was discussed, existing proposal has some aspects which better to be postponed, like extension list on the flavor (instead of tags). </div></div></blockquote><div><br>
</div>
</div><div>Agreed-- I think we need to more fully flesh out how extension list / tags should work here before we implement it. But this doesn't prevent us from rolling forward with a "version 1" of flavors so that we can start to use some of the benefits of having flavors (like the ability to use multiple service profiles with a single driver/provider, or multiple service profiles for a single kind of service).</div>
<div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Particularly that idea has several drawbacks:</div>
<div> - it makes public API inflexible</div><div> - turning features on/off is not what flavors should be doing, it's a task for policy framework and not flavors</div><div> - flavor-based rest call dispatching is quite complex solution giving no benefits for service plugins</div>
</div></blockquote><div><br></div></div><div>I'm confused as to what you mean by "that idea" here. Are you taking about the "extension list"? If this is the case, I agree that that aspect needs to be refined and should probably be postponed if possible.</div>
</div></div></div></div></div></div></blockquote><div><br></div></div><div>I was under the impression that the extension list was a set of neutron extensions supported by all the service providers in the flavour. This could probably be enforced as an API level constraint. </div>
</div></div></div></blockquote><div><br></div></div><div>I don't think there's much agreement on what the extension list should actually be at the present time (honestly, it seem close enough to the 'tags' proposal in Eugene's original spec that I keep mistaking one for the other). This is one of the reasons I'm in favor of deferring that discussion until version 2 of flavors, and simply working with the free-form 'description' field for now which is informational but shouldn't be considered programmatically consumable.</div>
<div class="">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote">
<div><div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>While this is not explicitly written in proposal - that's what implied there.</div>
<div>I think that one is a major blocker of the proposal right now, it deserves future discussion and not essential to the problem flavors are supposed to solve.</div></div></blockquote><div><br></div></div><div>Yes, I think there are many benefits we can get out of the flavor framework without having to have an extensions list / tags at this revision. But I'm curious: Did we ever define what we were actually trying to solve with flavors? Maybe that's the reason the discussion on this has been all of the place: People are probably making assumptions about the problem we're trying to solve and we need to get on the same page about this.</div>
</div></div></div></div></div></div></blockquote><div><br></div></div><div>That is what I've been saying for over a year. Whatever you call it "service type", "service provider" or "flavour" it appears that's impossible to find two persons who think about it in the same way.</div>
</div></div></div></blockquote><div><br></div></div><div>Haha! Great! Well, when it comes to proposals like this, I prefer to think of them in terms of "what problem are we trying to solve" and then limiting scope so we can actually produce something before the cold death of the universe.</div>
<div><br></div><div>So! With that in mind, I agree with what Eugene said on this:</div><div><br></div></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div class="gmail_extra"><div class="gmail_quote">
<div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px"> The original problem has several aspects: </div></div></div></div></div><div class=""><div><div class="gmail_extra"><div class="gmail_quote">
<div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">
1) providing users with some information about what service implementation they get (capabilities)</div></div></div></div></div><div><div class="gmail_extra"><div class="gmail_quote"><div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">
2) providing users with ability to specify (choose, actually) some implementation details that don't relate to a logical configuration</div></div></div></div></div><div><div class="gmail_extra"><div class="gmail_quote">
<div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">(capacity, insertion mode, HA mode, resiliency, security standards, etc)</div></div></div></div></div><div><div class="gmail_extra"><div class="gmail_quote">
<div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">3) providing operators a way to setup different modes of one driver</div></div></div></div></div><div><div class="gmail_extra"><div class="gmail_quote">
<div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">4) providing operators to seamlessly change drivers backing up existing logical configurations (now it's not so easy to do because logical config is tightly coupled with provider/driver)</div>
</div></div></div></div><div><div class="gmail_extra"><div class="gmail_quote"><div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px"><br></div></div></div></div></div><div><div class="gmail_extra">
<div class="gmail_quote"><div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">The proposal we're discussing right is mostly covering points (2), (3) and (4) which is already a good thing.</div>
</div></div></div></div><div><div class="gmail_extra"><div class="gmail_quote"><div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">So for now I'd propose to put 'information about service implementation' in the description to cover (1)</div>
</div></div></div></div></div></blockquote><div><div class="gmail_extra"><div class="gmail_quote"><div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px"><br></div></div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">
If there are problems people are trying to solve with flavors that are not among the above 4 points, I would suggest that these either become a part of a later revision of flavors, or simply get discussed as a new entity entirely (depending on what people are after).</div>
<div style="font-family:arial,sans-serif;font-size:12.800000190734863px"><br></div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">Anyone have objections to this?</div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">
<br></div><div style="font-family:arial,sans-serif;font-size:12.800000190734863px">(Sorry for the abruptness on my part on this: We really need to get flavors into Juno because we have features of LBaaS which are probably going to depend on it. I realize I've been dominating this discussion because of the time pressure here, but I really am interested in learning whether flavors as proposed in the spec right now provide useful functionality to the other Neutron Advanced Services, other than LBaaS.)</div>
<div class="">
<div style="font-family:arial,sans-serif;font-size:12.800000190734863px"><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote"><div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div>Other than that, I personally don't have much disagreements on the proposal.</div>
<div><br></div><div>The question about service type on the flavor is minor IMO. We can allow it to be NULL, which would mean multiservice flavor.</div><div>However, multiservice flavors may put some minor requirements to driver API (that's mainly because of how flavor plugin interacts with service plugins)</div>
</div></blockquote><div><br></div></div><div>Yes, I think single-service flavors is almost certainly going to be a simpler thing to implement, too-- and if we want to get flavors in for Juno (which I know *we* really want to do), then I think reducing the complexity here is probably a good idea, at least for the first revision.</div>
</div></div></div></div></div></div></blockquote><div><br></div></div><div>Yes. At the end of the day I have no problem with going with single service flavours. Mostly because it will be already a win if we manage to bind this framework to a single advanced service in the juno release cycle.</div>
<div></div></div></div></div></blockquote><div><br></div></div><div>Yay! That's the goal: Get this into Juno! And it seems to me like a good first step toward multi-service_type flavors, if we as a group decide that's where we want to go.</div>
<div><br></div></div>Thanks,</div><div class=""><div class="gmail_extra">Stephen<br clear="all"><div><br></div>-- <br><span></span>Stephen Balukoff
<br>Blue Box Group, LLC
<br><a href="tel:%28800%29613-4305%20x807" value="+18006134305" target="_blank">(800)613-4305 x807</a>
</div></div></div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>