Open Stack

On 01/09/2013 05:15 AM, Robert Collins wrote:
> On 7 January 2013 23:01, Gary Kotton<gkotton at redhat.com>  wrote:
>> On 01/06/2013 09:42 PM, Robert Collins wrote:
>>> Hey, so I've been poking at quantum with the baremetal hypervisor on
>>> classic l2 switches - no openflow.
>>>
>>> Here is what I think needs to be done to make it all work, and what I
>>> think 'it' is :)
>>>
>>> So the goal is to have a regular L2 network, with quantum + baremetal
>>> nova managing dhcp on it.
>>
>> Why have Nova managing the DHCP. Quantum has IPAM and a option of using a
>> DHCP agent. The advantage of this is that the DHCP agent will only allocate
>> IP addresses for devices that are registered as Quantum ports.
> I phrased that badly. I mean 'quantum + baremetal nova' as an
> aggregate, not that nova would do DHCP. In fact, the point of this is
> to stop nova doing DHCP, though it still needs to orchestrate the
> *TFTP* aspects of it.
>
> Specifically, I want the dnsmasq opts file to look like this:
> tag:tag0,option:router,192.0.2.36
> tag:tftp-host.novabaremetalhostname1,option:bootfile-name,pxelinux.0
> tag:tftp-host.novabaremetalhostname1,option:server-ip-address,192.0.2.33
> tag:tftp-host.novabaremetalhostname1,opttion:tftp-server,192.0.2.33
> tag:tftp-host.novabaremetalhostname2,option:bootfile-name,pxelinux.0
> tag:tftp-host.novabaremetalhostname2,option:server-ip-address,192.0.2.34
> tag:tftp-host.novabaremetalhostname2,opttion:tftp-server,192.0.2.34
>
> Where novabaremetalhostname2 is the nova compute host that is doing
> baremetal (and there may be many such, though commonly it will be 2
> for HA).
>
> And the host file will contain:
> 52:54:00:97:71:c3,192-0-2-34.openstacklocal,192.0.2.37,set:tftp-host.novabaremetalhostname2
> To trigger 52:54:00:97:71:c3 to TFTP boot from tftp-host.novabaremetalhostname2
>
> Once that it setup, until that baremetal host is taken out of
> rotation, its all static.

Thanks for the explanation. I have a number of comments regarding the above.
It is certainly doable and would require a few changes to the Quantum 
DHCP agent. This can b configured according to the type of port (this 
may be via information in the device_id field).

Please note that the DHCP agent is plugable. At the moment a dnsmaq 
implementation is used. This can be replaced by any other implementation 
(this is via the configuration flag "dhcp_driver = 
quantum.agent.linux.dhcp.Dnsmasq".

>>> Ignoring quantum for the moment, what needs to happen is that:
>>>    - machines attempt PXE booting using their own MAC address
>>>    - if the machine is not known to nova, something sensible (e.g.
>>> allocate from a pool, or log and ditch, or allocate a number and do
>>> hardware discovery)
>>>    - if the machine is known to nova, boot it providing appropriate
>>> bootfile-name, server-ip-address and tftp-server options.
>>>    - nova's baremetal hypervisor then handles tftp and the rest of the
>>> boot process.
>>>
>>> What I have so far is this:
>>>    - quantum with the ovs plugin
>>>    - with a provider network configured (e.g.
>>> tenant_network_type=none
>>> network_vlan_ranges=ctlplane
>>> bridge_mappings=ctlplane:br-ctlplane
>>>
>>> and a network and subnet made with that:
>>> $ quantum net-create ctlplane --tenant-id $tenantuuid
>>> --provider:network_type flat --provider:physical_network ctlplane
>>>
>>> $ subnet-create --ip-version 4 --allocation-pool
>>> start=192.0.2.34,end=192.0.2.38 --gateway=192.0.2.33 $netuuid
>>> 192.0.2.32/29
>>>
>>> Will answer DHCP for DHCP requests coming from the physical L2 network
>>> if the MAC has been registered as a port.
>>>
>>> My plan for unknown machines for now is to ignore them (can just have
>>> a deliberately slow DHCP server running on the same network, as a
>>> stopgap), but long term would like to teach quantum a fallback
>>> configuration for unknown requests. [which might be to proxy-DHCP the
>>> request].
>>
>> The Quantum DHCP agent will only provide IP's for MAC addresses that are
>> known to the agent.
> Today for sure. Would you be opposed to changing that (when
> configured)? The idea is to be able to inventory a rack by turning it
> on, rather than by doing manual data entry. We would TFTP boot a
> nondestructive ramdisk on every machine, which would report back to an
> API.
>
>> When you create a Quantum port you are able to specify the MAC address of
>> the port. This may require a few tweaks to the Nova code when creating the
>> Quantum port. Today you are able to do a nova boot and pass the port id a
>> parameter. So if you create the Quantum port outside of Nova then you can do
>> this with the existing API's.
> Thanks, this kindof works. I found two things were odd: firstly, the
> port is deleted when the instance is, so you have to create it every
> time.

This is certainly something can be optimized. One idea would be to cache 
ports instead of deleting them and reuse them. This would save 
interactions with the Quantum service.
>   Secondly, as there is no baremetal way to boot a specific
> baremetal node, you can only do this with one unprovisioned bare metal
> node at a time - its clearly the right path for the baremetal compute
> code to drive things, but its not something users can do sensibly by
> hand.
>
> Summarising:
>   - manual mac address stuff needs no quantum changes AFAICT
>   - the TFTP boot configuration I describe above does need changes. I
> was thinking of this model:
>     - have a TFTP server object
>     - that gets mapped onto a subnet
>     - and individual ports are able to be set 'boot off<this tftp server>'
>     - its up to nova to tell quantum if the set of baremetal boot
> machines changes, to update configs.
>
> Thoughts (and pointers to where in the code to start changing things)
> appreciated!

Regarding Quantum I suggest that you draft a blueprint. 
(http://wiki.openstack.org/Quantum/BlueprintTemplate). This can either 
be achieved by adding an extension to the subnet. Examples of extension 
can be seen at 
https://github.com/openstack/quantum/tree/master/quantum/extensions. You 
will need to integrate the extension into the various plugins...

>
> -Rob
>