In the Triple-O docs for unicast DHCP relay, it doesn't exactly say which IP address to target. Without deploying Triple-O, I'm not clear if the relay IP should be the bridge interface or the DHCP device. The first method makes sense because the gateway for that subnet wouldn't be connected to the Ironic controller by layer 2 (unless we used VXLAN over the physical network). As an experiment, I created a second subnet on my provisioning network. The original DHCP device port now has two IP addresses, one on each subnet. That makes the second method possible if I targeted its original IP address. Thanks for the help and please let me know which method is correct. Tom King On Fri, May 29, 2020 at 3:15 AM Dan Sneddon <dsneddon@redhat.com> wrote:

You probably want to enable Neutron segments and use the Neutron routed networks feature so you can use different subnets on different segments (layer 2 domains AKA VLANs) of the same network. You specify different values such as IP allocation pools and router address(es) for each subnet, and Ironic and Neutron will do the right thing. You need to enable segments in the Neutron configuration and restart the Neutron server. I don’t think you will have to recreate the network. Behind the scenes, dnsmasq will be configured with multiple subnets and address scopes within the Neutron DHCP agent and the Ironic Inspector agent.

Each segment/subnet will be given a different VLAN ID. As Dmitry mentioned, TripleO uses that method for the provisioning network, so you can use that as an example. The provisioning network in TripleO is the one referred to as the “control plane” network.

-Dan

On Fri, May 29, 2020 at 12:51 AM Dmitry Tantsur <dtantsur@redhat.com> wrote:

...

Hi Tom,

I know for sure that people are using DHCP relay with ironic, I think the TripleO documentation may give you some hints (adjusted to your presumably non-TripleO environment): http://tripleo.org/install/advanced_deployment/routed_spine_leaf_network.htm...

Dmitry

On Thu, May 28, 2020 at 11:06 PM Amy Marrich <amy@demarco.com> wrote:

...

Hey Tom,

Forwarding to the OpenStack discuss list where you might get more assistance.

Thanks,

Amy (spotz)

On Thu, May 28, 2020 at 3:32 PM Thomas King <thomas.king@gmail.com> wrote:

...

Good day,

We have Ironic running and connected via VLANs to nearby machines. We want to extend this to other parts of our product development lab without extending VLANs.

Using DHCP relay, we would point to a single IP address to serve DHCP requests but I'm not entirely sure of the Neutron network/subnet configuration, nor which IP address should be used for the relay agent on the switch.

Is DHCP relay supported by Neutron?

My guess is to add a subnet in the provisioning network and point the relay agent to the linuxbridge interface's IP: 14: brq467f6775-be: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether e2:e9:09:7f:89:0b brd ff:ff:ff:ff:ff:ff inet 10.10.0.1/16 scope global brq467f6775-be valid_lft forever preferred_lft forever inet6 fe80::5400:52ff:fe85:d33d/64 scope link valid_lft forever preferred_lft forever

Thank you, Tom King _______________________________________________ openstack-mentoring mailing list openstack-mentoring@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-mentoring

-- Dan Sneddon | Senior Principal Software Engineer dsneddon@redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter

We do have the Ironic inspector enabled but mainly use out-of-band such as iDRAC. I am indeed not using segments. I'll need to research that a bit more. One important note, we are only using provider networks with no Neutron routers. All routing is done on the physical network which aligns with the docs for segments. The provisioning subnet is on 10.10.0.0/16 for the directly attached nodes. As a test, I created a second subnet, 10.100.0.0/16, on the same Neutron network with DHCP enabled, so now I have two subnets on the same network and Neutron DHCP port. However, if DHCP relay requires different segments per remote network... The Networking service defines a segment using the following components: - Unique physical network name - Segmentation type - Segmentation ID Does having unique physical network names also mean unique physical interfaces? Does this mean no flat network for segments? If I create 10.100.0.0/16 in rack A1 and the controller is in D30, am I pointing the DHCP relay to the DHCP agent's 10.10.0.0/16 IP address? +--------------------+-------+---------------------------+ | Agent Type | Alive | Binary | +--------------------+-------+---------------------------+ | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Linux bridge agent | :-) | neutron-linuxbridge-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Metering agent | :-) | neutron-metering-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | DHCP agent | :-) | neutron-dhcp-agent | | L3 agent | :-) | neutron-l3-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Metadata agent | :-) | neutron-metadata-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | +--------------------+-------+---------------------------+ On Mon, Jun 1, 2020 at 4:34 PM <dsneddon@redhat.com> wrote:

You will have to target two IP addresses with DHCP relay if you are using Ironic Inspector. The first is the IP where Ironic Inspector is listening with dnsmasq, usually the IP of the host itself. I know this doesn't lend itself to HA scenarios, but you might also be able to forward to the broadcast IP of the subnet where the Ironic Inspector will be running (I haven't tested this, but it is a common use case for DHCP relay).

The second IP address is that of the Neutron DHCP agent, and that will be used for deploying bare metal nodes. IIRC, this IP is shared with the Neutron router for the network if you are using the L3 agent as well.

If you are not running Ironic Inspector (and manually entering in baremetal host details instead), then you can forward DHCP relay only to the Neutron DHCP agent.

Both of these IP addresses will be on the "root" subnet which is associated with the segment with the controller node(s).

It sounds like you created a second subnet, but I'm not sure if you created the second subnet on a different segment from the first subnet. In Neutron routed networking, the segments determine whether a subnet is local or remote to the controller node(s). Typically the first segment would be the one local to the controller(s). Are you sure you enabled the segments plugin and created your second subnet on a new segment?

Another approach which does not involve DHCP relay is to deploy DHCP agents locally on compute nodes local to each segment. This way all DHCP will be done within the same L2 domain, and you will not have to configure DHCP relay on your router serving each segment/subnet.

See the docs for more info:

https://docs.openstack.org/newton/networking-guide/config-routed-networks.ht...

-Dan

On Fri, 2020-05-29 at 10:47 -0600, Thomas King wrote:

...

In the Triple-O docs for unicast DHCP relay, it doesn't exactly say which IP address to target. Without deploying Triple-O, I'm not clear if the relay IP should be the bridge interface or the DHCP device.

The first method makes sense because the gateway for that subnet wouldn't be connected to the Ironic controller by layer 2 (unless we used VXLAN over the physical network).

As an experiment, I created a second subnet on my provisioning network. The original DHCP device port now has two IP addresses, one on each subnet. That makes the second method possible if I targeted its original IP address.

Thanks for the help and please let me know which method is correct.

Tom King

On Fri, May 29, 2020 at 3:15 AM Dan Sneddon <dsneddon@redhat.com> wrote:

...

You probably want to enable Neutron segments and use the Neutron routed networks feature so you can use different subnets on different segments (layer 2 domains AKA VLANs) of the same network. You specify different values such as IP allocation pools and router address(es) for each subnet, and Ironic and Neutron will do the right thing. You need to enable segments in the Neutron configuration and restart the Neutron server. I don’t think you will have to recreate the network. Behind the scenes, dnsmasq will be configured with multiple subnets and address scopes within the Neutron DHCP agent and the Ironic Inspector agent.

Each segment/subnet will be given a different VLAN ID. As Dmitry mentioned, TripleO uses that method for the provisioning network, so you can use that as an example. The provisioning network in TripleO is the one referred to as the “control plane” network.

-Dan

On Fri, May 29, 2020 at 12:51 AM Dmitry Tantsur < dtantsur@redhat.com> wrote:

...

Hi Tom,

I know for sure that people are using DHCP relay with ironic, I think the TripleO documentation may give you some hints (adjusted to your presumably non-TripleO environment):

http://tripleo.org/install/advanced_deployment/routed_spine_leaf_network.htm...

...

Dmitry

On Thu, May 28, 2020 at 11:06 PM Amy Marrich <amy@demarco.com> wrote:

...

Hey Tom,

Forwarding to the OpenStack discuss list where you might get more assistance.

Thanks,

Amy (spotz)

On Thu, May 28, 2020 at 3:32 PM Thomas King < thomas.king@gmail.com> wrote:

...

Good day,

We have Ironic running and connected via VLANs to nearby machines. We want to extend this to other parts of our product development lab without extending VLANs.

Using DHCP relay, we would point to a single IP address to serve DHCP requests but I'm not entirely sure of the Neutron network/subnet configuration, nor which IP address should be used for the relay agent on the switch.

Is DHCP relay supported by Neutron?

My guess is to add a subnet in the provisioning network and point the relay agent to the linuxbridge interface's IP: 14: brq467f6775-be: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether e2:e9:09:7f:89:0b brd ff:ff:ff:ff:ff:ff inet 10.10.0.1/16 scope global brq467f6775-be valid_lft forever preferred_lft forever inet6 fe80::5400:52ff:fe85:d33d/64 scope link valid_lft forever preferred_lft forever

Thank you, Tom King _______________________________________________ openstack-mentoring mailing list openstack-mentoring@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-mentoring

-- Dan Sneddon | Senior Principal Software Engineer dsneddon@redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter -- Dan Sneddon | Senior Principal Software Engineer dsneddon@redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter

Well, I'm using a flat network (i.e., access mode switchport), not a tagged switchport. The remote nodes will be on their own subnet and VLAN/segment that will *not *be attached to the controller. "In that case, DHCP requests from all segments that are not attached to the controller(s) need to be forwarded to the controllers via DHCP relay." Agreed. The question is which DHCP agent IP address? DHCP agent IP #1 = 10.10.1.1/16, corresponds to an attached segment/subnet. DHCP agent IP #2 = 10.100.1.1/16, corresponds to a subnet in a completely separate rack with no direct connection to the controller. Even if I separate IP #2 onto a different segment with its own DHCP agent port, am I sending DHCP relay to IP #1 or #2? I'm assuming #1. Tom On Mon, Jun 1, 2020 at 6:40 PM <dsneddon@redhat.com> wrote:

The use case for routed networks is when you have multiple distinct subnets which are not connected at layer 2 and only have connectivity to one another via the router gateways on each network. A segment can be thought of as a VLAN, although depending on topology a different VLAN ID is not always used. The key is that there is no layer 2 connectivity between segments, traffic has to be routed between them.

The situation where you would use DHCP relay is when you are not assigning DHCP agents to the compute nodes, and you have compute nodes on segments that the controllers are not attached to. In that case, DHCP requests from all segments that are not attached to the controller(s) need to be forwarded to the controllers via DHCP relay.

If you have a flat network, then you have no need for DHCP relay, the DHCP agents can receive and respond to requests over layer 2. This applies even if you have multiple subnets on the same segment.

On Mon, 2020-06-01 at 18:02 -0600, Thomas King wrote:

...

We do have the Ironic inspector enabled but mainly use out-of-band such as iDRAC.

I am indeed not using segments. I'll need to research that a bit more.

One important note, we are only using provider networks with no Neutron routers. All routing is done on the physical network which aligns with the docs for segments. The provisioning subnet is on 10.10.0.0/16 for the directly attached nodes. As a test, I created a second subnet, 10.100.0.0/16, on the same Neutron network with DHCP enabled, so now I have two subnets on the same network and Neutron DHCP port. However, if DHCP relay requires different segments per remote network...

...

The Networking service defines a segment using the following components:

Unique physical network name Segmentation type Segmentation ID

Does having unique physical network names also mean unique physical interfaces? Does this mean no flat network for segments? If I create 10.100.0.0/16 in rack A1 and the controller is in D30, am I pointing the DHCP relay to the DHCP agent's 10.10.0.0/16 IP address?

+--------------------+-------+---------------------------+ | Agent Type | Alive | Binary | +--------------------+-------+---------------------------+ | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Linux bridge agent | :-) | neutron-linuxbridge-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Metering agent | :-) | neutron-metering-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | DHCP agent | :-) | neutron-dhcp-agent | | L3 agent | :-) | neutron-l3-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Metadata agent | :-) | neutron-metadata-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | | Baremetal Node | :-) | ironic-neutron-agent | +--------------------+-------+---------------------------+

On Mon, Jun 1, 2020 at 4:34 PM <dsneddon@redhat.com> wrote:

...

You will have to target two IP addresses with DHCP relay if you are using Ironic Inspector. The first is the IP where Ironic Inspector is listening with dnsmasq, usually the IP of the host itself. I know this doesn't lend itself to HA scenarios, but you might also be able to forward to the broadcast IP of the subnet where the Ironic Inspector will be running (I haven't tested this, but it is a common use case for DHCP relay).

The second IP address is that of the Neutron DHCP agent, and that will be used for deploying bare metal nodes. IIRC, this IP is shared with the Neutron router for the network if you are using the L3 agent as well.

If you are not running Ironic Inspector (and manually entering in baremetal host details instead), then you can forward DHCP relay only to the Neutron DHCP agent.

Both of these IP addresses will be on the "root" subnet which is associated with the segment with the controller node(s).

It sounds like you created a second subnet, but I'm not sure if you created the second subnet on a different segment from the first subnet. In Neutron routed networking, the segments determine whether a subnet is local or remote to the controller node(s). Typically the first segment would be the one local to the controller(s). Are you sure you enabled the segments plugin and created your second subnet on a new segment?

Another approach which does not involve DHCP relay is to deploy DHCP agents locally on compute nodes local to each segment. This way all DHCP will be done within the same L2 domain, and you will not have to configure DHCP relay on your router serving each segment/subnet.

See the docs for more info:

https://docs.openstack.org/newton/networking-guide/config-routed-networks.ht...

...

-Dan

...

In the Triple-O docs for unicast DHCP relay, it doesn't exactly say which IP address to target. Without deploying Triple-O, I'm not clear if the relay IP should be the bridge interface or the DHCP device.

The first method makes sense because the gateway for that subnet wouldn't be connected to the Ironic controller by layer 2 (unless we used VXLAN over the physical network).

As an experiment, I created a second subnet on my provisioning network. The original DHCP device port now has two IP addresses, one on each subnet. That makes the second method possible if I targeted its original IP address.

Thanks for the help and please let me know which method is correct.

Tom King

On Fri, May 29, 2020 at 3:15 AM Dan Sneddon <dsneddon@redhat.com> wrote:

...

You probably want to enable Neutron segments and use the Neutron routed networks feature so you can use different subnets on different segments (layer 2 domains AKA VLANs) of the same network. You specify different values such as IP allocation pools and router address(es) for each subnet, and Ironic and Neutron will do the right thing. You need to enable segments in the Neutron configuration and restart the Neutron server. I don’t think you will have to recreate the network. Behind the scenes, dnsmasq will be configured with multiple subnets and address scopes within

On Fri, 2020-05-29 at 10:47 -0600, Thomas King wrote: the

...

...

Neutron DHCP agent and the Ironic Inspector agent.

Each segment/subnet will be given a different VLAN ID. As Dmitry mentioned, TripleO uses that method for the provisioning network, so you can use that as an example. The provisioning network in TripleO is the one referred to as the “control plane” network.

-Dan

On Fri, May 29, 2020 at 12:51 AM Dmitry Tantsur < dtantsur@redhat.com> wrote:

...

Hi Tom,

I know for sure that people are using DHCP relay with ironic, I think the TripleO documentation may give you some hints (adjusted to your presumably non-TripleO environment):

http://tripleo.org/install/advanced_deployment/routed_spine_leaf_network.htm...

...

...

...

...

Dmitry

On Thu, May 28, 2020 at 11:06 PM Amy Marrich <amy@demarco.com

...

...

wrote: > Hey Tom, > > Forwarding to the OpenStack discuss list where you might get > more assistance. > > Thanks, > > Amy (spotz) > > On Thu, May 28, 2020 at 3:32 PM Thomas King < > thomas.king@gmail.com> wrote: > > Good day, > > > > We have Ironic running and connected via VLANs to nearby > > machines. We want to extend this to other parts of our > > product development lab without extending VLANs. > > > > Using DHCP relay, we would point to a single IP address to > > serve DHCP requests but I'm not entirely sure of the Neutron > > network/subnet configuration, nor which IP address should be > > used for the relay agent on the switch. > > > > Is DHCP relay supported by Neutron? > > > > My guess is to add a subnet in the provisioning network and > > point the relay agent to the linuxbridge interface's IP: > > 14: brq467f6775-be: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu > > 1500 qdisc noqueue state UP group default qlen 1000 > > link/ether e2:e9:09:7f:89:0b brd ff:ff:ff:ff:ff:ff > > inet 10.10.0.1/16 scope global brq467f6775-be > > valid_lft forever preferred_lft forever > > inet6 fe80::5400:52ff:fe85:d33d/64 scope link > > valid_lft forever preferred_lft forever > > > > Thank you, > > Tom King > > _______________________________________________ > > openstack-mentoring mailing list > > openstack-mentoring@lists.openstack.org > >

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-mentoring

...

...

...

-- Dan Sneddon | Senior Principal Software Engineer dsneddon@redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter

-- Dan Sneddon | Senior Principal Software Engineer dsneddon@redhat.com | redhat.com/cloud dsneddon:irc | @dxs:twitter