Hi Hongbin, Thank you for your reply. Following are the log errors I am getting: LogFile: 2019-04-04 12:59:04.718 42324 INFO werkzeug [-] 127.0.0.1 - - [04/Apr/2019 12:59:04] "POST /IpamDriver.RequestAddress HTTP/1.1" 500 - 2019-04-04 13:16:34.652 42324 DEBUG kuryr_libnetwork.controllers [-] Received JSON data {u'Address': u'10.68.48.137', u'Options': {u'com.docker.network.endpoint.macaddress': u'fa:16:3e:20:9d:2c'}, u'PoolID': u'b130d090-9b7f-4c68-9f45-775a18bbb649'} for /IpamDriver.RequestAddress ipam_request_address /usr/local/lib/python2.7/dist-packages/kuryr_libnetwork/controllers.py:1646 2019-04-04 13:16:35.113 42324 ERROR kuryr_libnetwork.utils [-] Unexpected error happened: Requested ip address {'subnet_id': u'e684a17b-6b17-4ea2-bbc9-752a21304dcc', 'ip_address': u'10.68.48.137'} already belongs to a bound Neutron port: 045ff1a5-d20c-4299-9ffd-9c676af398b0: AddressInUseException: Requested ip address {'subnet_id': u'e684a17b-6b17-4ea2-bbc9-752a21304dcc', 'ip_address': u'10.68.48.137'} already belongs to a bound Neutron port: 045ff1a5-d20c-4299-9ffd-9c676af398b0 Journald: Apr 04 13:16:35 bedminster kuryr-server[42324]: Traceback (most recent call last): Apr 04 13:16:35 bedminster kuryr-server[42324]: File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1813, in full_dispatch_request Apr 04 13:16:35 bedminster kuryr-server[42324]: rv = self.dispatch_request() Apr 04 13:16:35 bedminster kuryr-server[42324]: File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1799, in dispatch_request Apr 04 13:16:35 bedminster kuryr-server[42324]: return self.view_functions[rule.endpoint](**req.view_args) Apr 04 13:16:35 bedminster kuryr-server[42324]: File "/usr/local/lib/python2.7/dist-packages/kuryr_libnetwork/controllers.py", line 1740, in ipam_request_address Apr 04 13:16:35 bedminster kuryr-server[42324]: existing_port, fixed_ip, req_mac_address) Apr 04 13:16:35 bedminster kuryr-server[42324]: File "/usr/local/lib/python2.7/dist-packages/kuryr_libnetwork/controllers.py", line 542, in _update_existing_port Apr 04 13:16:35 bedminster kuryr-server[42324]: existing_port['id'])) Apr 04 13:16:35 bedminster kuryr-server[42324]: AddressInUseException: Requested ip address {'subnet_id': u'e684a17b-6b17-4ea2-bbc9-752a21304dcc', 'ip_address': u'10.68.48.137'} already belongs to a bound Neutron port: 045ff1a5-d20c-4299-9ffd-9c676af398b0 Kind Regards, Navdeep From: Hongbin Lu <hongbin034@gmail.com> Sent: 04 April 2019 14:10 To: Navdeep Uniyal <navdeep.uniyal@bristol.ac.uk> Cc: openstack@lists.openstack.org Subject: Re: [Zun] Address already belongs to a bound Neutron port Error On Thu, Apr 4, 2019 at 5:14 AM Navdeep Uniyal <navdeep.uniyal@bristol.ac.uk<mailto:navdeep.uniyal@bristol.ac.uk>> wrote: Hi Hongbin, I could not see the logs. I have installed zun on the baremetal machine with full openstack (not devstack) -- Logs begin at Tue 2019-04-02 16:23:14 UTC, end at Thu 2019-04-04 09:05:59 UTC. -- -- No entries – My bad. If you were following the installation guide, you should run "sudo journalctl -u kuryr-libnetwork" Please advise me how can I enable logging for this. I can’t find anything in /etc/kuryr/kuryr.log In my kuryr.conf file I have set the value to debug and the log file as ‘/etc/kuryr/kuryr.log’ It is systemd process so uses journalctl to retrieve the log. No further configuration is needed, although it is always helpful to turn on debug on troubleshooting My flat network is: openstack network show 5guknet +---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2019-04-03T15:30:50Z | | description | | | dns_domain | None | | id | a9decbf0-e8c8-4574-86f8-242756fce63b | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | mtu | 1500 | | name | 5guknet | | port_security_enabled | True | | project_id | a8888997d0da4d3d80fe7ca833e74bb7 | | provider:network_type | flat | | provider:physical_network | 5guknet | | provider:segmentation_id | None | | qos_policy_id | None | | revision_number | 4 | | router:external | External | | segments | None | | shared | True | | status | ACTIVE | | subnets | e684a17b-6b17-4ea2-bbc9-752a21304dcc | | tags | kuryr.net.existing:7c34de9e89a3, kuryr.net.uuid.lh:7c34de9e89a3e86ee3561834520bc2ea, kuryr.net.uuid.uh:35c16ff10c1821c06a7098833e0bf131 | | updated_at | 2019-04-03T15:33:37Z | +---------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ Subnet: openstack subnet show e684a17b-6b17-4ea2-bbc9-752a21304dcc +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 10.68.48.131-10.68.48.150 | | cidr | 10.68.48.0/24<http://10.68.48.0/24> | | created_at | 2019-04-03T15:30:51Z | | description | | | dns_nameservers | 8.8.8.8 | | enable_dhcp | True | | gateway_ip | 10.68.48.1 | | host_routes | | | id | e684a17b-6b17-4ea2-bbc9-752a21304dcc | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | 5guknet | | network_id | a9decbf0-e8c8-4574-86f8-242756fce63b | | project_id | a8888997d0da4d3d80fe7ca833e74bb7 | | revision_number | 1 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | b130d090-9b7f-4c68-9f45-775a18bbb649 | | updated_at | 2019-04-03T15:33:37Z | +-------------------+--------------------------------------+ Port: openstack port show 8b203580-2051-4ad0-8057-c7885324d0b9 +-----------------------+-----------------------------------------------------------------------------+ | Field | Value | +-----------------------+-----------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | bedminster | | binding_profile | | | binding_vif_details | port_filter='True' | | binding_vif_type | bridge | | binding_vnic_type | normal | | created_at | 2019-04-04T09:05:33Z | | data_plane_status | None | | description | | | device_id | c6428670-2846-4862-8ecb-611293cabc2c | | device_owner | compute:kuryr | | dns_assignment | None | | dns_domain | None | | dns_name | None | | extra_dhcp_opts | | | fixed_ips | ip_address='10.68.48.143', subnet_id='e684a17b-6b17-4ea2-bbc9-752a21304dcc' | | id | 8b203580-2051-4ad0-8057-c7885324d0b9 | | mac_address | fa:16:3e:2e:07:ad | | name | | | network_id | a9decbf0-e8c8-4574-86f8-242756fce63b | | port_security_enabled | True | | project_id | a8888997d0da4d3d80fe7ca833e74bb7 | | qos_policy_id | None | | revision_number | 2 | | security_group_ids | 49839b6f-79c7-4db7-bb71-ffecbdd645d5 | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2019-04-04T09:05:33Z | +-----------------------+-----------------------------------------------------------------------------+ Thank you, Kind Regards, Navdeep From: Hongbin Lu <hongbin034@gmail.com<mailto:hongbin034@gmail.com>> Sent: 04 April 2019 02:49 To: Navdeep Uniyal <navdeep.uniyal@bristol.ac.uk<mailto:navdeep.uniyal@bristol.ac.uk>> Cc: openstack@lists.openstack.org<mailto:openstack@lists.openstack.org> Subject: Re: [Zun] Address already belongs to a bound Neutron port Error Hi Navdeep, It looks I cannot reproduce the problem using rocky with flat network. If possible, would you provide the following: * Your kuryr-libnetwork log (with sensitive information removed). You can get it by running "sudo journalctl -u devstack@kuryr-libnetwork". * Your flat network (openstack network show XXX) * Your subnet (openstack subnet show e684a17b-6b17-4ea2-bbc9-752a21304dcc) * Your port (openstack port show f0d6153a-7408-4cd5-82c3-9abed48ad056) Best regards, Hongbin On Wed, Apr 3, 2019 at 11:52 AM Navdeep Uniyal <navdeep.uniyal@bristol.ac.uk<mailto:navdeep.uniyal@bristol.ac.uk>> wrote: Dear All, I have just installed Zun and tried creating a container following the instructions as per the following webpage: https://docs.openstack.org/zun/rocky/install/index.html I am getting following error while starting my test container using ‘openstack appcontainer run --name container2 --net network=$NET_ID cirros ping 8.8.8.8’: status_reason | Docker internal error: 500 Server Error: Internal Server Error ("IpamDriver.RequestAddress: Requested ip address {'subnet_id': u'e684a17b-6b17-4ea2-bbc9-752a21304dcc', 'ip_address': u'10.68.48.147'} already belongs to a bound Neutron port: f0d6153a-7408-4cd5-82c3-9abed48ad056"). The network I selected is a flat network on openstack with external flag true. Please advise me how to solve the issue or if there is a specific requirement for this. Thank you, Kind Regards, Navdeep