<div dir="ltr">Hi Carl,<div><br></div><div>Please find me reply inline</div><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 9, 2015 at 9:49 AM, Carl Baldwin <span dir="ltr"><<a href="mailto:carl@ecbaldwin.net" target="_blank">carl@ecbaldwin.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span>On Fri, Nov 6, 2015 at 2:59 PM, Shraddha Pandhe <span dir="ltr"><<a href="mailto:spandhe.openstack@gmail.com" target="_blank">spandhe.openstack@gmail.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div>We have a similar requirement where we want to pick a network thats accessible in the rack that VM belongs to. We have L3 Top-of-rack, so the network is confined to the rack. Right now, we are achieving this by naming physical network name in a certain way, but thats not going to scale.</div><div><br></div><div>We also want to be able to make scheduling decisions based on IP availability. So we need to know rack <-> network <-> mapping. We can't embed all factors in a name. It will be impossible to make scheduling decisions by parsing name and comparing. GoDaddy has also been doing something similar [1], [2].</div></div></div></div></blockquote></span></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span><div><br></div></span><div>This is precisely the use case that the large deployers team (LDT) has brought to Neutron [1]. In fact, GoDaddy has been at the forefront of that request. We've had discussions about this since just after Vancouver on the ML. I've put up several specs to address it [2] and I'm working another revision of it. My take on it is that Neutron needs a model for a layer 3 network (IpNetwork) which would group the rack networks. The IpNetwork would be visible to the end user and there will be a network <-> host mapping. I am still aiming to have working code for this in Mitaka. I discussed this with the LDT in Tokyo and they seemed to agree. We had a session on this in the Neutron design track [3][4] though that discussion didn't produce anything actionable.</div><div><br></div></div></div></div></blockquote><div>Thats great. L3 layer network model is definitely one of our most important requirements. All our go-forward deployments are going to be L3. So this is a big deal for us. </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div></div><div>Solving this problem at the IPAM level has come up in discussion but I don't have any references for that. It is something that I'm still considering but I haven't worked out all of the details for how this can work in a portable way. Could you describe how you imagine how this flow would work from a user's perspective? Specifically, when a user wants to boot a VM, what precise API calls would be made to achieve this on your network and how where would the IPAM data come in to play?</div></div></div></div></blockquote><div><br></div><div>Here's what the flow looks like to me.</div><div><br></div><div>1. User sends a boot request as usual. The user need not know all the network and subnet information beforehand. All he would do is send a boot request.</div><div><br></div><div>2. The scheduler will pick a node in an L3 rack. The way we map nodes <-> racks is as follows:</div><div> a. For VMs, we store rack_id in nova.conf on compute nodes</div><div> b. For Ironic nodes, right now we have static IP allocation, so we practically know which IP we want to assign. But when we move to dynamic allocation, we would probably use 'chassis' or 'driver_info' fields to store the rack id.</div><div><br></div><div>3. Nova compute will try to pick a network ID for this instance. At this point, it needs to know what networks (or subnets) are available in this rack. Based on that, it will pick a network ID and send port creation request to Neutron. At Yahoo, to avoid some back-and-forth, we send a fake network_id and let the plugin do all the work.</div><div><br></div><div>4. We need some information associated with the network/subnet that tells us what rack it belongs to. Right now, for VMs, we have that information embedded in physnet name. But we would like to move away from that. If we had a column for subnets - e.g. tag, it would solve our problem. Ideally, we would like a column 'rack id' or a new table 'racks' that maps to subnets, or something. We are open to different ideas that work for everyone. This is where IPAM can help.</div><div><br></div><div>5. We have another requirement where we want to store multiple gateway addresses for a subnet, just like name servers.</div><div><br></div><div><br></div><div>We also have a requirement where we want to make scheduling decisions based on IP availability. We want to allocate multiple IPs to the hosts. e.g. We want to allocate X IPs to a host. The flow in that case would be</div><div><br></div><div>1. User sends a boot request with --num-ips X</div><div> The network/subnet level complexities need not be exposed to the user. For better experience, all we want our users to tell us is the number of IPs they want. </div><div><br></div><div>2. When the scheduler tries to find an appropriate host in L3 racks, we want it to find a rack that can satisfy this IP requirement. So, the scheduler will basically say, "give me all racks that have >X IPs available". If we have a 'Racks' table in IPAM, that would help.</div><div> Once the scheduler gets a rack, it will apply remaining filters to narrow down to one host and call nova-compute. The IP count will be propagated to nova compute from scheduler.</div><div><br></div><div><br></div><div>3. Nova compute will call Neutron and send the node details and IP count along. Neutron IPAM driver will then look at the node details, query the database to find a network in that rack and allocate X IPs from the subnet.</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>Carl </div><div><br></div><div>[1] <a href="https://bugs.launchpad.net/neutron/+bug/1458890" target="_blank">https://bugs.launchpad.net/neutron/+bug/1458890</a></div><div>[2] <a href="https://review.openstack.org/#/c/225384/" target="_blank">https://review.openstack.org/#/c/225384/</a></div><div>[3] <a href="https://etherpad.openstack.org/p/mitaka-neutron-next-network-model" target="_blank">https://etherpad.openstack.org/p/mitaka-neutron-next-network-model</a></div><div>[4] <a href="https://www.openstack.org/summit/tokyo-2015/schedule/design-summit" target="_blank">https://www.openstack.org/summit/tokyo-2015/schedule/design-summit</a></div></div></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div>