<div dir="ltr">Hi Neil,<div><br></div><div>Please find my reply inline.<br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Nov 6, 2015 at 1:08 PM, Neil Jerram <span dir="ltr"><<a href="mailto:Neil.Jerram@metaswitch.com" target="_blank">Neil.Jerram@metaswitch.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
Yes, maybe. I'm interested in a pluggable IPAM module that will allocate an IP address for a VM that depends on where that VM's host is in the physical data center network. Is that similar to your requirement?</div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
<br></div></div></blockquote><div>We have a similar requirement where we want to pick a network thats accessible in the rack that VM belongs to. We have L3 Top-of-rack, so the network is confined to the rack. Right now, we are achieving this by naming physical network name in a certain way, but thats not going to scale.</div><div><br></div><div>We also want to be able to make scheduling decisions based on IP availability. So we need to know rack <-> network <-> mapping. We can't embed all factors in a name. It will be impossible to make scheduling decisions by parsing name and comparing. GoDaddy has also been doing something similar [1], [2].</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
</div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
I don't yet know whether that might lead me to want to store additional data in the Neutron DB. My intuition though is that it shouldn't, and that any additional data or state that I need for this IPAM module should be stored separately from the Neutron DB.</div></div></blockquote><div><br></div><div> Where are you planning to store that information? If we need similar information, and if more folks need it, we can add it to Neutron DB in IPAM tables. </div><div><br></div><div>[1] <a href="http://www.dorm.org/blog/openstack-architecture-at-go-daddy-part-3-nova/#Scheduler_Customization">http://www.dorm.org/blog/openstack-architecture-at-go-daddy-part-3-nova/#Scheduler_Customization</a></div><div>[2] <a href="http://www.dorm.org/blog/openstack-architecture-at-go-daddy-part-2-neutron/#Customizations_to_Abstract_Away_Layer_2">http://www.dorm.org/blog/openstack-architecture-at-go-daddy-part-2-neutron/#Customizations_to_Abstract_Away_Layer_2</a></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
</div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
Regards, </div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
Neil </div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
<br></div></div></blockquote><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
</div>
<div style="width:100%;font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-size:initial;font-family:Calibri,'Slate Pro',sans-serif,sans-serif;color:rgb(31,73,125);text-align:initial;background-color:rgb(255,255,255)">
</div>
<table width="100%" style="background-color:white;border-spacing:0px">
<tbody>
<tr>
<td colspan="2" style="font-size:initial;text-align:initial;background-color:rgb(255,255,255)">
<div style="border-style:solid none none;border-top-color:rgb(181,196,223);border-top-width:1pt;padding:3pt 0in 0in;font-family:Tahoma,'BB Alpha Sans','Slate Pro';font-size:10pt">
<div><b>From: </b>Shraddha Pandhe</div>
<div><b>Sent: </b>Friday, 6 November 2015 20:23</div>
<div><b>To: </b>OpenStack Development Mailing List (not for usage questions)</div>
<div><b>Reply To: </b>OpenStack Development Mailing List (not for usage questions)</div><span class="">
<div><b>Subject: </b>Re: [openstack-dev] [Neutron][IPAM] Arbitrary JSON blobs in ipam db tables</div>
</span></div>
</td>
</tr>
</tbody>
</table>
<div style="border-style:solid none none;border-top-color:rgb(186,188,209);border-top-width:1pt;font-size:initial;text-align:initial;background-color:rgb(255,255,255)">
</div>
<br><div><div class="h5">
<div>
<div dir="ltr">Bumping this up :)
<div><br>
</div>
<div><br>
</div>
<div>Folks, does anyone else have a similar requirement to ours? Are folks making scheduling decisions based on networking?</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Nov 5, 2015 at 12:24 PM, Shraddha Pandhe <span dir="ltr">
<<a href="mailto:spandhe.openstack@gmail.com" target="_blank">spandhe.openstack@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">Hi,
<div><br>
</div>
<div>
<div>I agree with all of you about the REST Apis.</div>
<div><br>
</div>
<div>As I said before, I had to bring up the idea of JSON blob because based on previous discussions, it looked like neutron community was not willing to enhance the schemas for different ipam dbs. Entire rationale behind pluggable IPAM is to provide flexibility.
So, community should be open to ideas for enhancing the schema to incorporate more information in the db tables. I would be extremely happy if use cases for different companies are considered and schema is enhanced to include specific columns in db schemas
instead of a column with random JSON blob.</div>
<div><br>
</div>
<div>Lets pick up subnets db table for example. We have some use cases where it would be great if following information is associated with the subnet db table</div>
<div><br>
</div>
<div>1. Rack switch info</div>
<div>2. Backplane info</div>
<div>3. DHCP ip helpers</div>
<div>4. Option to tag allocation pools inside subnets</div>
<div>5. Multiple gateway addresses</div>
<div><br>
</div>
<div>We also want to store some information about the backplanes locally, so a different table might be useful.</div>
<div><br>
</div>
<div>In a way, this information is not specific to our company. Its generic information which ought to go with the subnets. Different companies can use this information differently in their IPAM drivers. But, the information needs to be made available to justify
the flexibility of ipam</div>
<div><br>
</div>
<div>In Yahoo! OpenStack is still not the source of truth for this kind of information and database limitation is one of the reasons. I would prefer to avoid having our own database to make sure that our use-cases are always shared with the community.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Nov 5, 2015 at 9:37 AM, Kyle Mestery <span dir="ltr">
<<a href="mailto:mestery@mestery.com" target="_blank">mestery@mestery.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>On Thu, Nov 5, 2015 at 10:55 AM, Jay Pipes <span dir="ltr">
<<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
On 11/04/2015 04:21 PM, Shraddha Pandhe wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hi Salvatore,<br>
<br>
Thanks for the feedback. I agree with you that arbitrary JSON blobs will<br>
make IPAM much more powerful. Some other projects already do things like<br>
this.<br>
</blockquote>
<br>
:( Actually, though "powerful" it also leads to implementation details leaking directly out of the public REST API. I'm very negative on this and would prefer an actual codified REST API that can be relied on regardless of backend driver or implementation.<br>
</blockquote>
<div><br>
</div>
</span>
<div>I agree with Jay here. We've had people propose similar things in Neutron before, and I've been against them. The entire point of the Neutron REST API is to not leak these details out. It dampens the strength of the logical model, and it tends to have
users become reliant on backend implementations.<br>
<br>
</div>
<span>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
e.g. In Ironic, node has driver_info, which is JSON. it also has an<br>
'extras' arbitrary JSON field. This allows us to put any information in<br>
there that we think is important for us.<br>
</blockquote>
<br>
Yeah, and this is a bad thing, IMHO. Public REST APIs should be structured, not a Wild West free-for-all. The biggest problem with using free-form JSON blobs in RESTful APIs like this is that you throw away the ability to evolve the API in a structured, versioned
way. Instead of evolving the API using microversions, instead every vendor just jams whatever they feel like into the JSON blob over time. There's no way for clients to know what the server will return at any given time.<br>
<br>
Achieving consensus on a REST API that meets the needs of a variety of backend implementations is *hard work*, yes, but it's what we need to do if we are to have APIs that are viewed in the industry as stable, discoverable, and reliably useful.<br>
</blockquote>
<div><br>
</div>
</span>
<div>++, this is the correct way forward.<br>
<br>
</div>
<div>Thanks,<br>
</div>
<div>Kyle<br>
<br>
</div>
<div>
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Best,<br>
-jay<br>
<br>
Best,<br>
-jay<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Hoping to get some positive feedback from API and DB lieutenants too.<br>
<br>
<br>
On Wed, Nov 4, 2015 at 1:06 PM, Salvatore Orlando<br>
<<a href="mailto:salv.orlando@gmail.com" target="_blank">salv.orlando@gmail.com</a> <mailto:<a href="mailto:salv.orlando@gmail.com" target="_blank">salv.orlando@gmail.com</a>>> wrote:<br>
<br>
Arbitrary blobs are a powerful tools to circumvent limitations of an<br>
API, as well as other constraints which might be imposed for<br>
versioning or portability purposes.<br>
The parameters that should end up in such blob are typically<br>
specific for the target IPAM driver (to an extent they might even<br>
identify a specific driver to use), and therefore an API consumer<br>
who knows what backend is performing IPAM can surely leverage it.<br>
<br>
Therefore this would make a lot of sense, assuming API portability<br>
and not leaking backend details are not a concern.<br>
The Neutron team API & DB lieutenants will be able to provide more<br>
input on this regard.<br>
<br>
In this case other approaches such as a vendor specific extension<br>
are not a solution - assuming your granularity level is the<br>
allocation pool; indeed allocation pools are not first-class neutron<br>
resources, and it is not therefore possible to have APIs which<br>
associate vendor specific properties to allocation pools.<br>
<br>
Salvatore<br>
<br>
On 4 November 2015 at 21:46, Shraddha Pandhe<br>
<<a href="mailto:spandhe.openstack@gmail.com" target="_blank">spandhe.openstack@gmail.com</a> <mailto:<a href="mailto:spandhe.openstack@gmail.com" target="_blank">spandhe.openstack@gmail.com</a>>><br>
wrote:<br>
<br>
Hi folks,<br>
<br>
I have a small question/suggestion about IPAM.<br>
<br>
With IPAM, we are allowing users to have their own IPAM drivers<br>
so that they can manage IP allocation. The problem is, the new<br>
ipam tables in the database have the same columns as the old<br>
tables. So, as a user, if I want to have my own logic for ip<br>
allocation, I can't actually get any help from the database.<br>
Whereas, if we had an arbitrary json blob in the ipam tables, I<br>
could put any useful information/tags there, that can help me<br>
for allocation.<br>
<br>
Does this make sense?<br>
<br>
e.g. If I want to create multiple allocation pools in a subnet<br>
and use them for different purposes, I would need some sort of<br>
tag for each allocation pool for identification. Right now,<br>
there is no scope for doing something like that.<br>
<br>
Any thoughts? If there are any other way to solve the problem,<br>
please let me know<br>
<br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div>