Open Stack

On Wed, 22 Jan 2020 at 08:37, Kumari, Madhuri <madhuri.kumari at intel.com> wrote:
>
> Hi Mark,
>
>
>
> Thanks for your response. I read the blog and my requirement is shared IB network as of now. We have developed a Neutron ML2 driver that manages the IB subnet[1].
>
> Regardless of that, I think my current issue is because the cloud-init is not able to recognize the port as IB port. How did it work for you? You can see my ironic port details.
>
>
>
> [1] https://opendev.org/x/networking-omnipath/

I see, you didn't mention that you are using Omnipath which is based
on IB but has differences.

Here's the relevant code in cloudinit:
https://github.com/canonical/cloud-init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init__.py#L785.
In order for an interface to be picked up, it needs to have a type of
32 and the ethernet format MAC address of the port must match bytes
13-15 and 18-20 of the IB hardware address.

>
>
>
>
>
> >>-----Original Message-----
>
> >>From: Mark Goddard <mark at stackhpc.com>
>
> >>Sent: Monday, January 20, 2020 2:10 PM
>
> >>To: Kumari, Madhuri <madhuri.kumari at intel.com>
>
> >>Cc: openstack-discuss at lists.openstack.org
>
> >>Subject: Re: [ironic][nova][neutron][cloud-init] Infiniband Support in
>
> >>OpenStack
>
> >>
>
> >>On Fri, 17 Jan 2020 at 15:34, Kumari, Madhuri <madhuri.kumari at intel.com>
>
> >>wrote:
>
> >>>
>
> >>> Hi,
>
> >>>
>
> >>>
>
> >>>
>
> >>> I am trying to deploy a node with infiniband in Ironic without any success.
>
> >>>
>
> >>>
>
> >>>
>
> >>> The node has two interfaces, eth0 and ib0. The deployment is successful,
>
> >>node becomes active but is not reachable. I debugged and checked that the
>
> >>issue is with cloud-init. The cloud-init fails to configure the network interfaces
>
> >>on the node complaining that the MAC address of infiniband port(ib0) is not
>
> >>known to the node. Ironic provides a fake MAC address for infiniband ports
>
> >>and cloud-init is supposed to generate the actual MAC address of infiband
>
> >>ports[1]. But it fails[2] before reaching there.
>
> >>>
>
> >>> I have posted the issue in cloud-init[3] as well.
>
> >>>
>
> >>>
>
> >>>
>
> >>> Can someone please help me with this issue? How do we specify
>
> >>“TYPE=InfiniBand” from OpenStack? Currently the type sent is “phy” only.
>
> >>>
>
> >>>
>
> >>>
>
> >>> [1]
>
> >>> https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/
>
> >>> helpers/openstack.py#L686
>
> >>>
>
> >>> [2]
>
> >>> https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/
>
> >>> helpers/openstack.py#L677
>
> >>>
>
> >>> [3] https://bugs.launchpad.net/cloud-init/+bug/1857031
>
> >>>
>
> >>Hi Madhuri,
>
> >>
>
> >>Please see my blog post:
>
> >>https://www.stackhpc.com/bare-metal-infiniband.html. One major question
>
> >>to ask is whether you want shared IB network or multi-tenant isolation. The
>
> >>latter is significantly more challenging. It's probably best if you read that
>
> >>article and raise any further questions here or IRC. I'll be out of the office
>
> >>until Wednesday.
>
> >>
>
> >>Mark
>
> >>>
>
> >>>
>
> >>>
>
> >>>
>
> >>> Regards,
>
> >>>
>
> >>> Madhuri
>
> >>>
>
> >>>