[neutron] Cannot deploy instance in provider network openstack Ussuri -- network fails
Hello, i am new to Openstack. I have followed the instructions from https://docs.openstack.org/install-guide/ to install openstack Ussuri where i use a controller node and a compute node (i can ping each other and access the internet from both nodes through the mgmt iface). I use ubuntu 18.04 for all components. I have configured the network option 2, seems to work fine: openstack network agent list +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ | 2d8c3a89-32c4-4b97-aa4f-ca19db53b24f | L3 agent | controller | nova | :-) | UP | neutron-l3-agent | | 413cd13d-88d7-45ce-8b2e-26fdb265740f | Metadata agent | controller | None | :-) | UP | neutron-metadata-agent | | 42f57bee-63b3-44e6-9392-939ece98719d | Linux bridge agent | compute | None | :-) | UP | neutron-linuxbridge-agent | | 4a787a09-04aa-4350-bd32-0c0177ed06a1 | DHCP agent | controller | nova | :-) | UP | neutron-dhcp-agent | | fdafc337-7581-4ecd-b175-810713a25e1f | Linux bridge agent | controller | None | :-) | UP | neutron-linuxbridge-agent | +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ If i try to bring up instances connected by an internal network, they are successfully deployed. When i use the provider network to bring up an instance with the command: openstack server create --flavor m1.tiny --image cirros034 --nic net-id=4463e4a7-3b06-46d9-b24f-3e2e7867ee96 --security-group=5cf380e5-efb3-48b3-b8e9-095b1ebd817d provider-instance the instance fails to be instantiated because the network cannot be deployed. I noticed two points from the neutron-linuxbridge-agent.log file at the compute node (the logs at the controller seem fine): 1) ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied') 2) INFO neutron.plugins.ml2.drivers.agent._common_agent [req-816902d4-6581-49c1-939a-6ea69b437c99 - - - - -] Linux bridge agent Agent out of sync with plugin! -- not sure if this one is a problem. One part that i am not sure is the following configuration since i am on ubuntu 18 and not ubuntu 16: # The provider network interface auto INTERFACE_NAME iface INTERFACE_NAME inet manual up ip link set dev $IFACE up down ip link set dev $IFACE down I did not include this part in my file since its ubuntu 18, and netplan does not have an equivalent (i think) option. I have this netplan config enp0s8: --> mgmt dhcp4: false addresses: [10.0.0.31/24] gateway4: 10.0.0.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] enp0s9: --> provider dhcp4: false any advice? all the best, Pavlos Basaras
Hi, On Mon, Nov 23, 2020 at 12:11:55PM +0200, Pavlos Basaras wrote:
Hello,
i am new to Openstack.
I have followed the instructions from https://docs.openstack.org/install-guide/ to install openstack Ussuri where i use a controller node and a compute node (i can ping each other and access the internet from both nodes through the mgmt iface).
I use ubuntu 18.04 for all components.
I have configured the network option 2, seems to work fine:
openstack network agent list +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+ | 2d8c3a89-32c4-4b97-aa4f-ca19db53b24f | L3 agent | controller | nova | :-) | UP | neutron-l3-agent | | 413cd13d-88d7-45ce-8b2e-26fdb265740f | Metadata agent | controller | None | :-) | UP | neutron-metadata-agent | | 42f57bee-63b3-44e6-9392-939ece98719d | Linux bridge agent | compute | None | :-) | UP | neutron-linuxbridge-agent | | 4a787a09-04aa-4350-bd32-0c0177ed06a1 | DHCP agent | controller | nova | :-) | UP | neutron-dhcp-agent | | fdafc337-7581-4ecd-b175-810713a25e1f | Linux bridge agent | controller | None | :-) | UP | neutron-linuxbridge-agent | +--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
If i try to bring up instances connected by an internal network, they are successfully deployed.
When i use the provider network to bring up an instance with the command: openstack server create --flavor m1.tiny --image cirros034 --nic net-id=4463e4a7-3b06-46d9-b24f-3e2e7867ee96 --security-group=5cf380e5-efb3-48b3-b8e9-095b1ebd817d provider-instance
the instance fails to be instantiated because the network cannot be deployed.
I noticed two points from the neutron-linuxbridge-agent.log file at the compute node (the logs at the controller seem fine):
1) ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied')
This sounds like maybe some issue with usage of sudo by user which runs neutron-linuxbridge-agent. Is it configured properly?
2) INFO neutron.plugins.ml2.drivers.agent._common_agent [req-816902d4-6581-49c1-939a-6ea69b437c99 - - - - -] Linux bridge agent Agent out of sync with plugin! -- not sure if this one is a problem.
This is probably just a result of above error. But to be sure I would need to see whole log. If You will still have such issue, please open LP bug and attach Your logs there. In such way it should be maybe easier to investigate what is going on there. In Ussuri we are running CI job neutron-tempest-plugin-scenario-linuxbridge which deploys neutron-linuxbridge-agent and all works fine there. You can check that e.g. on https://zuul.opendev.org/t/openstack/build/441182cb87fd47daa34d5018fc2eb9ab
One part that i am not sure is the following configuration since i am on ubuntu 18 and not ubuntu 16:
# The provider network interface auto INTERFACE_NAME iface INTERFACE_NAME inet manual up ip link set dev $IFACE up down ip link set dev $IFACE down
I did not include this part in my file since its ubuntu 18, and netplan does not have an equivalent (i think) option. I have this netplan config
enp0s8: --> mgmt dhcp4: false addresses: [10.0.0.31/24] gateway4: 10.0.0.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] enp0s9: --> provider dhcp4: false
any advice?
all the best, Pavlos Basaras
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hello, I double checked the configuration with: https://docs.openstack.org/neutron/ussuri/install/compute-install-ubuntu.htm... , it looks ok, i think. attached the log from the compute node. Do you need any other log files ? As i am a newbie i don't really know how to "open an LP bug " all the best Pavlos. On Mon, Nov 23, 2020 at 1:52 PM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
On Mon, Nov 23, 2020 at 12:11:55PM +0200, Pavlos Basaras wrote:
Hello,
i am new to Openstack.
I have followed the instructions from https://docs.openstack.org/install-guide/ to install openstack Ussuri where i use a controller node and a compute node (i can ping each other and access the internet from both nodes through the mgmt iface).
I use ubuntu 18.04 for all components.
I have configured the network option 2, seems to work fine:
openstack network agent list
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| 2d8c3a89-32c4-4b97-aa4f-ca19db53b24f | L3 agent | controller | nova | :-) | UP | neutron-l3-agent | | 413cd13d-88d7-45ce-8b2e-26fdb265740f | Metadata agent | controller | None | :-) | UP | neutron-metadata-agent | | 42f57bee-63b3-44e6-9392-939ece98719d | Linux bridge agent | compute | None | :-) | UP | neutron-linuxbridge-agent | | 4a787a09-04aa-4350-bd32-0c0177ed06a1 | DHCP agent | controller | nova | :-) | UP | neutron-dhcp-agent | | fdafc337-7581-4ecd-b175-810713a25e1f | Linux bridge agent | controller | None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
If i try to bring up instances connected by an internal network, they are successfully deployed.
When i use the provider network to bring up an instance with the command: openstack server create --flavor m1.tiny --image cirros034 --nic net-id=4463e4a7-3b06-46d9-b24f-3e2e7867ee96 --security-group=5cf380e5-efb3-48b3-b8e9-095b1ebd817d provider-instance
the instance fails to be instantiated because the network cannot be deployed.
I noticed two points from the neutron-linuxbridge-agent.log file at the compute node (the logs at the controller seem fine):
1) ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied')
This sounds like maybe some issue with usage of sudo by user which runs neutron-linuxbridge-agent. Is it configured properly?
2) INFO neutron.plugins.ml2.drivers.agent._common_agent [req-816902d4-6581-49c1-939a-6ea69b437c99 - - - - -] Linux bridge agent Agent out of sync with plugin! -- not sure if this one is a problem.
This is probably just a result of above error. But to be sure I would need to see whole log.
If You will still have such issue, please open LP bug and attach Your logs there. In such way it should be maybe easier to investigate what is going on there.
In Ussuri we are running CI job neutron-tempest-plugin-scenario-linuxbridge which deploys neutron-linuxbridge-agent and all works fine there. You can check that e.g. on https://zuul.opendev.org/t/openstack/build/441182cb87fd47daa34d5018fc2eb9ab
One part that i am not sure is the following configuration since i am on ubuntu 18 and not ubuntu 16:
# The provider network interface auto INTERFACE_NAME iface INTERFACE_NAME inet manual up ip link set dev $IFACE up down ip link set dev $IFACE down
I did not include this part in my file since its ubuntu 18, and netplan does not have an equivalent (i think) option. I have this netplan config
enp0s8: --> mgmt dhcp4: false addresses: [10.0.0.31/24] gateway4: 10.0.0.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] enp0s9: --> provider dhcp4: false
any advice?
all the best, Pavlos Basaras
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hello, adding sudo sysctl net.ipv6.conf.all.disable_ipv6=1 to the compute node solved the problem thanks, Pavlos. On Mon, Nov 23, 2020 at 3:52 PM Pavlos Basaras <pbasaras@gmail.com> wrote:
Hello,
I double checked the configuration with: https://docs.openstack.org/neutron/ussuri/install/compute-install-ubuntu.htm... , it looks ok, i think.
attached the log from the compute node. Do you need any other log files ?
As i am a newbie i don't really know how to "open an LP bug "
all the best Pavlos.
On Mon, Nov 23, 2020 at 1:52 PM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
On Mon, Nov 23, 2020 at 12:11:55PM +0200, Pavlos Basaras wrote:
Hello,
i am new to Openstack.
I have followed the instructions from https://docs.openstack.org/install-guide/ to install openstack Ussuri where i use a controller node and a compute node (i can ping each other and access the internet from both nodes through the mgmt iface).
I use ubuntu 18.04 for all components.
I have configured the network option 2, seems to work fine:
openstack network agent list
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| 2d8c3a89-32c4-4b97-aa4f-ca19db53b24f | L3 agent | controller | nova | :-) | UP | neutron-l3-agent | | 413cd13d-88d7-45ce-8b2e-26fdb265740f | Metadata agent | controller | None | :-) | UP | neutron-metadata-agent | | 42f57bee-63b3-44e6-9392-939ece98719d | Linux bridge agent | compute | None | :-) | UP | neutron-linuxbridge-agent | | 4a787a09-04aa-4350-bd32-0c0177ed06a1 | DHCP agent | controller | nova | :-) | UP | neutron-dhcp-agent | | fdafc337-7581-4ecd-b175-810713a25e1f | Linux bridge agent | controller | None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
If i try to bring up instances connected by an internal network, they
are
successfully deployed.
When i use the provider network to bring up an instance with the command: openstack server create --flavor m1.tiny --image cirros034 --nic net-id=4463e4a7-3b06-46d9-b24f-3e2e7867ee96 --security-group=5cf380e5-efb3-48b3-b8e9-095b1ebd817d provider-instance
the instance fails to be instantiated because the network cannot be deployed.
I noticed two points from the neutron-linuxbridge-agent.log file at the compute node (the logs at the controller seem fine):
1) ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied')
This sounds like maybe some issue with usage of sudo by user which runs neutron-linuxbridge-agent. Is it configured properly?
2) INFO neutron.plugins.ml2.drivers.agent._common_agent [req-816902d4-6581-49c1-939a-6ea69b437c99 - - - - -] Linux bridge agent Agent out of sync with plugin! -- not sure if this one is a problem.
This is probably just a result of above error. But to be sure I would need to see whole log.
If You will still have such issue, please open LP bug and attach Your logs there. In such way it should be maybe easier to investigate what is going on there.
In Ussuri we are running CI job neutron-tempest-plugin-scenario-linuxbridge which deploys neutron-linuxbridge-agent and all works fine there. You can check that e.g. on
https://zuul.opendev.org/t/openstack/build/441182cb87fd47daa34d5018fc2eb9ab
One part that i am not sure is the following configuration since i am on ubuntu 18 and not ubuntu 16:
# The provider network interface auto INTERFACE_NAME iface INTERFACE_NAME inet manual up ip link set dev $IFACE up down ip link set dev $IFACE down
I did not include this part in my file since its ubuntu 18, and netplan does not have an equivalent (i think) option. I have this netplan config
enp0s8: --> mgmt dhcp4: false addresses: [10.0.0.31/24] gateway4: 10.0.0.1 nameservers: addresses: [8.8.8.8, 8.8.4.4] enp0s9: --> provider dhcp4: false
any advice?
all the best, Pavlos Basaras
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hi, Dnia poniedziałek, 23 listopada 2020 16:20:49 CET Pavlos Basaras pisze:
Hello,
adding sudo sysctl net.ipv6.conf.all.disable_ipv6=1
to the compute node solved the problem
It's more like workaround not a solution really. If You will want to use IPv6 on the host, You will have the issue again.
thanks, Pavlos.
On Mon, Nov 23, 2020 at 3:52 PM Pavlos Basaras <pbasaras@gmail.com> wrote:
Hello,
I double checked the configuration with: https://docs.openstack.org/neutron/ussuri/install/compute-install-ubuntu.h tml , it looks ok, i think.
attached the log from the compute node. Do you need any other log files ?
As i am a newbie i don't really know how to "open an LP bug "
all the best Pavlos.
On Mon, Nov 23, 2020 at 1:52 PM Slawek Kaplonski <skaplons@redhat.com>
wrote:
Hi,
On Mon, Nov 23, 2020 at 12:11:55PM +0200, Pavlos Basaras wrote:
Hello,
i am new to Openstack.
I have followed the instructions from https://docs.openstack.org/install-guide/ to install openstack Ussuri
where
i use a controller node and a compute node (i can ping each other and access the internet from both nodes through the mgmt iface).
I use ubuntu 18.04 for all components.
I have configured the network option 2, seems to work fine:
openstack network agent list
+--------------------------------------+--------------------+------------ +-------------------+-------+-------+---------------------------+>>
| ID | Agent Type | Host
Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------------ +-------------------+-------+-------+---------------------------+>>
| 2d8c3a89-32c4-4b97-aa4f-ca19db53b24f | L3 agent |
controller |
nova | :-) | UP | neutron-l3-agent |
| 413cd13d-88d7-45ce-8b2e-26fdb265740f | Metadata agent |
controller |
None | :-) | UP | neutron-metadata-agent |
| 42f57bee-63b3-44e6-9392-939ece98719d | Linux bridge agent | compute
None | :-) | UP | neutron-linuxbridge-agent |
| 4a787a09-04aa-4350-bd32-0c0177ed06a1 | DHCP agent |
controller |
nova | :-) | UP | neutron-dhcp-agent |
| fdafc337-7581-4ecd-b175-810713a25e1f | Linux bridge agent |
controller |
None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+------------ +-------------------+-------+-------+---------------------------+>>
If i try to bring up instances connected by an internal network, they
are
successfully deployed.
When i use the provider network to bring up an instance with the
command:
openstack server create --flavor m1.tiny --image cirros034 --nic net-id=4463e4a7-3b06-46d9-b24f-3e2e7867ee96 --security-group=5cf380e5-efb3-48b3-b8e9-095b1ebd817d provider-instance
the instance fails to be instantiated because the network cannot be deployed.
I noticed two points from the neutron-linuxbridge-agent.log file at the compute node (the logs at the controller seem fine):
1) ERROR neutron.plugins.ml2.drivers.agent._common_agent pyroute2.netlink.exceptions.NetlinkError: (13, 'Permission denied')
This sounds like maybe some issue with usage of sudo by user which runs neutron-linuxbridge-agent. Is it configured properly?
2) INFO neutron.plugins.ml2.drivers.agent._common_agent [req-816902d4-6581-49c1-939a-6ea69b437c99 - - - - -] Linux bridge agent Agent out of sync with plugin! -- not sure if this one is a problem.
This is probably just a result of above error. But to be sure I would need to see whole log.
If You will still have such issue, please open LP bug and attach Your logs there. In such way it should be maybe easier to investigate what is going on there.
In Ussuri we are running CI job neutron-tempest-plugin-scenario-linuxbridge which deploys neutron-linuxbridge-agent and all works fine there. You can check that e.g. on
https://zuul.opendev.org/t/openstack/build/441182cb87fd47daa34d5018fc2eb9 ab
One part that i am not sure is the following configuration since i am on ubuntu 18 and not ubuntu 16:
# The provider network interface auto INTERFACE_NAME iface INTERFACE_NAME inet manual up ip link set dev $IFACE up down ip link set dev $IFACE down
I did not include this part in my file since its ubuntu 18, and netplan does not have an equivalent (i think) option. I have this netplan config
enp0s8: --> mgmt
dhcp4: false addresses: [10.0.0.31/24] gateway4: 10.0.0.1
nameservers: addresses: [8.8.8.8, 8.8.4.4]
enp0s9: --> provider
dhcp4: false
any advice?
all the best, Pavlos Basaras
-- Slawek Kaplonski Principal Software Engineer Red Hat
-- Slawek Kaplonski Principal Software Engineer Red Hat
participants (2)
-
Pavlos Basaras
-
Slawek Kaplonski