simple build fails to allocate network interface

tim+openstack.org at coote.org tim+openstack.org at coote.org
Fri Jun 10 15:06:00 UTC 2022


Hi Brian

You’re correct about that. The Attachment to public networks means, I think, that it needs to be a private network. In my follow up, I managed to get much more detail about a specific server creation, and showed, I think, that the vm had been trying to use a couple of dhcp servers (v4 and v6), that afaict didn’t exist - I couldn’t ping them and didn’t recognise the network segments. I couldn’t work out where they came from. I got that detail by reviewing this: https://bit.ly/3MG6jnR

atm, I’m trying to see if I can get this to work: https://bit.ly/3mD60PU However, I immediately ran into some VirtuaBox changes that invalidated the Vagrantfile. I think that I can fix these.

The experience highlights to me the need for a decent test environment. Even then, I suspect that it’s very easy to have some of the latent bugs that you get in distributed systems, which undergrad courses note (https://bit.ly/2OgyWiI: I’m particularly fond of the example that lurks in Cassandra from that lecture set), and which AWS had to resort to formal proofs with TLA+ to even spot.

Tim

> On 10 Jun 2022, at 14:56, Brian Haley <haleyb.dev at gmail.com> wrote:
> 
> Hi,
> 
> From your log below you have this:
> 
>  nova.exception.ExternalNetworkAttachForbidden: It
>  is not allowed to create an interface on external network
>  205fa7d7-7067-4f63-b3d9-6a9b37b
> 
> That comes from nova/network/neutron.py:
> 
>                if net.get('router:external') and not net.get('shared'):
>                    raise exception.ExternalNetworkAttachForbidden(
>                        network_uuid=net['id'])
> 
> And from your original email 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f is the "public" network, not the "shared" network you created.
> 
> My guess if you don't have the "shared" attribute set on that network, or you're using the wrong network. What does 'openstack network show 205fa7d7-7067-4f63-b3d9-6a9b37b4f11f' output look like?
> 
> -Brian
> 
> On 6/10/22 06:52, tim+openstack.org at coote.org wrote:
>>> On 9 Jun 2022, at 13:20, Rodolfo Alonso Hernandez <ralonsoh at redhat.com <mailto:ralonsoh at redhat.com>> wrote:
>>> 
>>> Hello:
>>> 
>>> This seems to be a problem in Neutron. Please provide what backend you are using, and the Neutron server logs. If the ML2 driver has agents (DHCP, ML2 agent), provide the agents logs too.
>>> 
>>> Regards.
>>> 
>> Hullo
>> thanks for the response. I don’t know what to look for wrt ml2 driver - I’m using the default `stack.sh` installation script from the devstack repo.
>> The neutron log has helped, I think. I found this useful as a starting point:https://bit.ly/3MG6jnR <https://bit.ly/3MG6jnR> as it enables log extraction of events relating to the creation of a specific server (although that’s still quite large for a mailing list).
>> I think that this may help with your question (but as I said, I don’t really understand the question).
>> There may be a dhcp issue: I cannot ping either ipv4 or ipv6 dhcp addresses mentioned in the log from the vm. I’ve no idea where they come from, however.
>> Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f
>> be4f] Updated VIF entry in instance network info cache for port 5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc. {{(pid=20725) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3420}}
>> Jun 10 10:13:49 localhost.localdomain nova-compute[20725]: DEBUG nova.network.neutron [req-1c6a3f76-b810-4f78-89ab-ba8c0d1a2b39 req-172db7ab-dbda-4bad-912c-3bc0f26069a5 service nova] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1f
>> be4f] Updating instance_info_cache with network_info: [{"id": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "address": "fa:16:3e:02:7a:6a", "network": {"id": "2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63", "bridge": "br-int", "label": "private", "subnets": [{"cidr": "10.0.0.0/26", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.51", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}, {"cidr": "fd5b:4cf4:f7c1::/64", "dns": [], "gateway": {"address": "fd5b:4cf4:f7c1::1", "type": "gateway", "version": 6, "meta": {}}, "ips": [{"address": "fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a", "type": "fixed", "version": 6, "meta": {}, "floating_ips": []}], "routes": [], "version": 6, "meta": {"ipv6_address_mode": "slaac", "dhcp_server": "fd5b:4cf4:f7c1::1"}}], "meta": {"injected": false, "tenant_id": "6c0b876005f44086bd9bde5fef672dfe", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5ae4c0d6-2a", "ovs_interfaceid": "5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=20725) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}}
>> Jun 10 10:14:11 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}}
>> Jun 10 10:15:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}}
>> Jun 10 10:16:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}}
>> Jun 10 10:17:12 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}}
>> Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(name=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, resource={'id': '5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc', 'name': '', 'network_id': '2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'tenant_id': '6c0b876005f44086bd9bde5fef672dfe', 'mac_address': 'fa:16:3e:02:7a:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'device_owner': 'compute:nova', 'standard_attr_id': 41, 'fixed_ips': [{'subnet_id': 'e7377bed-3e22-4e8d-9c2d-ea7ba740fcfd', 'ip_address': '10.0.0.51'}, {'subnet_id': 'f915b88c-9988-4e1f-9060-a6295465699a', 'ip_address': 'fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': ['ed75ab0b-d9a4-4023-9204-93038729f6d3'], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'localhost.localdomain', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'tags': [], 'created_at': '2022-06-10T10:13:44Z', 'updated_at': '2022-06-10T10:13:46Z', 'revision_number': 3, 'project_id': '6c0b876005f44086bd9bde5fef672dfe'}, resource_type=ports, if_exists=True) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}}
>> Jun 10 10:17:50 localhost.localdomain neutron-server[20918]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLSwitchPortCommand(lport=5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc, columns={'external_ids': {'neutron:port_name': '', 'neutron:device_id': 'fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f', 'neutron:project_id': '6c0b876005f44086bd9bde5fef672dfe', 'neutron:cidrs': '10.0.0.51/26 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a/64', 'neutron:device_owner': 'compute:nova', 'neutron:network_name': 'neutron-2a1a4a3a-a47b-48bd-a7df-c90fc75a1c63', 'neutron:security_group_ids': 'ed75ab0b-d9a4-4023-9204-93038729f6d3', 'neutron:revision_number': '3'}, 'parent_name': [], 'tag': [], 'options': {'requested-chassis': 'localhost.localdomain', 'mcast_flood_reports': 'true'}, 'enabled': True, 'port_security': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'dhcpv4_options': [UUID('01cae053-612b-4c91-81e8-39884ef035dd')], 'dhcpv6_options': [], 'type': '', 'addresses': ['fa:16:3e:02:7a:6a 10.0.0.51 fd5b:4cf4:f7c1:0:f816:3eff:fe02:7a6a'], 'ha_chassis_group': []}, if_exists=False) {{(pid=20918) do_commit /usr/local/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90}}
>> Jun 10 10:18:13 localhost.localdomain nova-compute[20725]: DEBUG nova.compute.manager [None req-66324ba4-aeda-4bbd-9711-9eafbf9c7bb7 None None] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Skipping network cache update for instance because it is Building. {{(pid=20725) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9465}}
>> Jun 10 10:18:49 localhost.localdomain nova-compute[20725]: WARNING nova.compute.manager [None req-4a03d9e1-6053-4167-ab43-fba30072a79c demo admin] [instance: fc8c3558-e9f5-488f-96b6-8a2eeb1fbe4f] Timeout waiting for ['network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc'] for instance with vm_state building and task_state spawning. Event states are: network-vif-plugged-5ae4c0d6-2ad1-4f91-9a3b-5ed0d93239cc: timed out after 300.00 seconds: eventlet.timeout.Timeout: 300 seconds
>> Jun 08 15:24:48 localhost.localdomain nova-compute[87555]: ERROR nova.compute.manager nova.exception.ExternalNetworkAttachForbidden: It is not allowed to create an interface on external network 205fa7d7-7067-4f63-b3d9-6a9b37b>




More information about the openstack-discuss mailing list