Hi Clark, Thank you for your response. I think the infiniband MAC address should be mac[36:-14] + mac[51:] as suggested here[1]. I have specified the right MAC address as per this but it still fails. Please see the following output: ib0 interface details: 4: ib0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state DOWN group default qlen 256 link/infiniband 80:00:00:02:fe:80:00:00:00:00:00:00:00:11:75:01:01:67:0f:b9 brd 00:ff:ff:ff:ff:12:40:1b:80:00:00:00:00:00:00:00:ff:ff:ff:ff Ironic port details: +-----------------------+----------------------------------------------------------------+ | Field | Value | +-----------------------+----------------------------------------------------------------+ | address | 00:11:75:67:0f:b9 | | created_at | 2020-01-16T08:24:15+00:00 | | extra | {'client-id': '0xfe800000000000000011750101670fb9'} | | internal_info | {'tenant_vif_port_id': 'c71b2b31-4231-423c-b512-962623220ddf'} | | is_smartnic | False | | local_link_connection | {} | | node_uuid | 05cce921-931f-4755-ad87-fc41d79a8988 | | physical_network | None | | portgroup_uuid | None | | pxe_enabled | False | | updated_at | 2020-01-16T08:59:47+00:00 | | uuid | 9921139a-63cc-4456-8e85-f7673c5c2b3b | +-----------------------+----------------------------------------------------------------+ [1] https://github.com/canonical/cloud-init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init__.py#L795<https://github.com/canonical/cloud-init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init__.py>
-----Original Message----- From: Clark Boylan <cboylan@sapwetik.org> Sent: Saturday, January 18, 2020 3:41 AM To: openstack-discuss@lists.openstack.org Subject: Re: [ironic][nova][neutron][cloud-init] Infiniband Support in OpenStack
On Fri, Jan 17, 2020, at 7:31 AM, Kumari, Madhuri wrote:
Hi,
I am trying to deploy a node with infiniband in Ironic without any success.
The node has two interfaces, eth0 and ib0. The deployment is successful, node becomes active but is not reachable. I debugged and checked that the issue is with cloud-init. The cloud-init fails to configure the network interfaces on the node complaining that the MAC address of infiniband port(ib0) is not known to the node. Ironic provides a fake MAC address for infiniband ports and cloud-init is supposed to generate the actual MAC address of infiband ports[1]. But it fails[2] before reaching there.
Reading the cloud-init code [4][5] it appears that the ethernet format MAC should match bytes 13-15 + 18-20 of the infiniband address. Is the problem here that the fake MAC supplied is unrelated to the actual infiniband address? If so I think you'll either need cloud-init to ignore unknown interfaces (as proposed in the cloud-init bug), or have Ironic supply the mac address as bytes 13-15 + 18-20 of the actual infiniband address.
I have posted the issue in cloud-init[3] as well.
Can someone please help me with this issue? How do we specify “TYPE=InfiniBand” from OpenStack? Currently the type sent is “phy” only.
[1] https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/ helpers/openstack.py#L686
[2] https://github.com/canonical/cloud-init/blob/master/cloudinit/sources/ helpers/openstack.py#L677
[4] https://github.com/canonical/cloud-<https://github.com/canonical/cloud-init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init__.py#L793-L795> init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init __.py#L793-L795 [5] https://github.com/canonical/cloud-<https://github.com/canonical/cloud-init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init__.py#L844-L846> init/blob/9bfb2ba7268e2c3c932023fc3d3020cdc6d6cc18/cloudinit/net/__init __.py#L844-L846