Hi Mark,

Thanks for your response. I read the blog and my requirement is shared IB network as of now. We have developed a Neutron ML2 driver that manages the IB subnet[1].

Regardless of that, I think my current issue is because the cloud-init is not able to recognize the port as IB port. How did it work for you? You can see my ironic port details.

[1] https://opendev.org/x/networking-omnipath/

>>-----Original Message-----

>>From: Mark Goddard <mark@stackhpc.com>

>>Sent: Monday, January 20, 2020 2:10 PM

>>To: Kumari, Madhuri <madhuri.kumari@intel.com>

>>Cc: openstack-discuss@lists.openstack.org

>>Subject: Re: [ironic][nova][neutron][cloud-init] Infiniband Support in

>>OpenStack

>>

>>On Fri, 17 Jan 2020 at 15:34, Kumari, Madhuri <madhuri.kumari@intel.com>

>>wrote:

>>>

>>> Hi,

>>>

>>>

>>>

>>> I am trying to deploy a node with infiniband in Ironic without any success.

>>>

>>>

>>>

>>> The node has two interfaces, eth0 and ib0. The deployment is successful,

>>node becomes active but is not reachable. I debugged and checked that the

>>issue is with cloud-init. The cloud-init fails to configure the network interfaces

>>on the node complaining that the MAC address of infiniband port(ib0) is not

>>known to the node. Ironic provides a fake MAC address for infiniband ports

>>and cloud-init is supposed to generate the actual MAC address of infiband

>>ports[1]. But it fails[2] before reaching there.

>>>

>>> I have posted the issue in cloud-init[3] as well.

>>>

>>>

>>>

>>> Can someone please help me with this issue? How do we specify

>>“TYPE=InfiniBand” from OpenStack? Currently the type sent is “phy” only.

>>>

>>>

>>>

>>> [1]

>>> https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/

>>> helpers/openstack.py#L686

>>>

>>> [2]

>>> https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/

>>> helpers/openstack.py#L677

>>>

>>> [3] https://bugs.launchpad.net/cloud-init/+bug/1857031

>>>

>>Hi Madhuri,

>>

>>Please see my blog post:

>>https://www.stackhpc.com/bare-metal-infiniband.html. One major question

>>to ask is whether you want shared IB network or multi-tenant isolation. The

>>latter is significantly more challenging. It's probably best if you read that

>>article and raise any further questions here or IRC. I'll be out of the office

>>until Wednesday.

>>

>>Mark

>>>

>>>

>>>

>>>

>>> Regards,

>>>

>>> Madhuri

>>>

>>>