Re: [ironic][nova][neutron][cloud-init] Infiniband Support in OpenStack

22 Jan 2020


      On Wed, 22 Jan 2020 at 08:37, Kumari, Madhuri <madhuri.kumari@intel.com> wrote:
...
Hi Mark,
Thanks for your response. I read the blog and my requirement is shared IB network as of now. We have developed a Neutron ML2 driver that manages the IB subnet[1].
Regardless of that, I think my current issue is because the cloud-init is not able to recognize the port as IB port. How did it work for you? You can see my ironic port details.
[1] https://opendev.org/x/networking-omnipath/
I see, you didn't mention that you are using Omnipath which is based
on IB but has differences.

Here's the relevant code in cloudinit:
https://github.com/canonical/cloud-init/blob/9bfb2ba7268e2c3c932023fc3d3020c....
In order for an interface to be picked up, it needs to have a type of
32 and the ethernet format MAC address of the port must match bytes
13-15 and 18-20 of the IB hardware address.
...
...
...
-----Original Message-----
...
...
From: Mark Goddard <mark@stackhpc.com>
...
...
Sent: Monday, January 20, 2020 2:10 PM
...
...
To: Kumari, Madhuri <madhuri.kumari@intel.com>
...
...
Cc: openstack-discuss@lists.openstack.org
...
...
Subject: Re: [ironic][nova][neutron][cloud-init] Infiniband Support in
...
...
OpenStack
...
...
...
...
On Fri, 17 Jan 2020 at 15:34, Kumari, Madhuri <madhuri.kumari@intel.com>
...
...
wrote:
...
...
...
...
...
...
Hi,
...
...
...
...
...
...
...
...
...
...
...
...
I am trying to deploy a node with infiniband in Ironic without any success.
...
...
...
...
...
...
...
...
...
...
...
...
The node has two interfaces, eth0 and ib0. The deployment is successful,
...
...
node becomes active but is not reachable. I debugged and checked that the
...
...
issue is with cloud-init. The cloud-init fails to configure the network interfaces
...
...
on the node complaining that the MAC address of infiniband port(ib0) is not
...
...
known to the node. Ironic provides a fake MAC address for infiniband ports
...
...
and cloud-init is supposed to generate the actual MAC address of infiband
...
...
ports[1]. But it fails[2] before reaching there.
...
...
...
...
...
...
I have posted the issue in cloud-init[3] as well.
...
...
...
...
...
...
...
...
...
...
...
...
Can someone please help me with this issue? How do we specify
...
...
“TYPE=InfiniBand” from OpenStack? Currently the type sent is “phy” only.
...
...
...
...
...
...
...
...
...
...
...
...
[1]
...
...
...
https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/
...
...
...
helpers/openstack.py#L686
...
...
...
...
...
...
[2]
...
...
...
https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/
...
...
...
helpers/openstack.py#L677
...
...
...
...
...
...
[3] https://bugs.launchpad.net/cloud-init/+bug/1857031
...
...
...
...
...
Hi Madhuri,
...
...
...
...
Please see my blog post:
...
...
https://www.stackhpc.com/bare-metal-infiniband.html. One major question
...
...
to ask is whether you want shared IB network or multi-tenant isolation. The
...
...
latter is significantly more challenging. It's probably best if you read that
...
...
article and raise any further questions here or IRC. I'll be out of the office
...
...
until Wednesday.
...
...
...
...
Mark
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Regards,
...
...
...
...
...
...
Madhuri
...
...
...
...
...
...