Hi Clark,
Thank you for your response.
I think the infiniband MAC address should be mac[36:-14] + mac[51:] as suggested here[1]. I have specified the right MAC address as per this but it still fails. Please see the following output:
ib0 interface details:
4: ib0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN group default qlen 256
link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:01:01:67:0f:b9 brd 00:ff:ff:ff:ff:12:40:1b:80:00:00:00:00:00:00:00:ff:ff:ff:ff
Ironic port details:
+-----------------------+----------------------------------------------------------------+
| Field | Value |
+-----------------------+----------------------------------------------------------------+
| address | 00:11:75:67:0f:b9 |
| created_at | 2020-01-16T08:24:15+00:00 |
| extra | {'client-id': '0xfe800000000000000011750101670fb9'} |
| internal_info | {'tenant_vif_port_id': 'c71b2b31-4231-423c-b512-962623220ddf'} |
| is_smartnic | False |
| local_link_connection | {} |
| node_uuid | 05cce921-931f-4755-ad87-fc41d79a8988 |
| physical_network | None |
| portgroup_uuid | None |
| pxe_enabled | False |
| updated_at | 2020-01-16T08:59:47+00:00 |
| uuid | 9921139a-63cc-4456-8e85-f7673c5c2b3b |
+-----------------------+----------------------------------------------------------------+
>>-----Original Message-----
>>From: Clark Boylan <cboylan@sapwetik.org>
>>Sent: Saturday, January 18, 2020 3:41 AM
>>To: openstack-discuss@lists.openstack.org
>>Subject: Re: [ironic][nova][neutron][cloud-init] Infiniband Support in
>>OpenStack
>>
>>On Fri, Jan 17, 2020, at 7:31 AM, Kumari, Madhuri wrote:
>>>
>>> Hi,
>>>
>>>
>>> I am trying to deploy a node with infiniband in Ironic without any success.
>>>
>>>
>>> The node has two interfaces, eth0 and ib0. The deployment is
>>> successful, node becomes active but is not reachable. I debugged and
>>> checked that the issue is with cloud-init. The cloud-init fails to
>>> configure the network interfaces on the node complaining that the MAC
>>> address of infiniband port(ib0) is not known to the node. Ironic
>>> provides a fake MAC address for infiniband ports and cloud-init is
>>> supposed to generate the actual MAC address of infiband ports[1]. But
>>> it fails[2] before reaching there.
>>
>>Reading the cloud-init code [4][5] it appears that the ethernet format MAC
>>should match bytes 13-15 + 18-20 of the infiniband address. Is the problem
>>here that the fake MAC supplied is unrelated to the actual infiniband
>>address? If so I think you'll either need cloud-init to ignore unknown
>>interfaces (as proposed in the cloud-init bug), or have Ironic supply the mac
>>address as bytes 13-15 + 18-20 of the actual infiniband address.
>>
>>>
>>> I have posted the issue in cloud-init[3] as well.
>>>
>>>
>>> Can someone please help me with this issue? How do we specify
>>> “TYPE=InfiniBand” from OpenStack? Currently the type sent is “phy” only.
>>>
>>>
>>> [1]
>>> helpers/openstack.py#L686
>>>
>>> [2]
>>> helpers/openstack.py#L677
>>>
>>
>>init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init
>>__.py#L793-L795
>>init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init
>>__.py#L844-L846