[openstack][neutron] it is good if we set fixed ip and set ip in instance netplan also
Hello guys. Sometimes I see that my instances won't get IP from Neutron DHCP Agent. Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also? Nguyen Huu Khoi
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry. Nguyen Huu Khoi On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
IIRC, when you disable dhcp on the network, modern cloud-init (like debian 12/Ubuntu 22.04) will configure the IP on your instance at boot time after doing a metadata network request. That just works, but it depends on OS. Cheers, Arnaud Le 11 août 2023 04:43:42 GMT+02:00, "Nguyễn Hữu Khôi" <nguyenhuukhoinw@gmail.com> a écrit :
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry.
Nguyen Huu Khoi
On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
On 2023-08-11 19:30:14 +0200 (+0200), Arnaud wrote:
IIRC, when you disable dhcp on the network, modern cloud-init (like debian 12/Ubuntu 22.04) will configure the IP on your instance at boot time after doing a metadata network request.
That just works, but it depends on OS. [...]
Cloud-init can also get network configuration from a mounted configdrive (if you pass the right server create options so it gets one), and skip requesting metadata over the network that way. -- Jeremy Stanley
Hi Jeremy, Thank you much for suggestions. I will consider it. :) Nguyen Huu Khoi On Sat, Aug 12, 2023 at 12:45 AM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2023-08-11 19:30:14 +0200 (+0200), Arnaud wrote:
IIRC, when you disable dhcp on the network, modern cloud-init (like debian 12/Ubuntu 22.04) will configure the IP on your instance at boot time after doing a metadata network request.
That just works, but it depends on OS. [...]
Cloud-init can also get network configuration from a mounted configdrive (if you pass the right server create options so it gets one), and skip requesting metadata over the network that way. -- Jeremy Stanley
Hi Arnaud. Thank you so much for your reply. I will verify it. Nguyen Huu Khoi On Sat, Aug 12, 2023 at 12:38 AM Arnaud <arnaud.morin@gmail.com> wrote:
IIRC, when you disable dhcp on the network, modern cloud-init (like debian 12/Ubuntu 22.04) will configure the IP on your instance at boot time after doing a metadata network request.
That just works, but it depends on OS. Cheers, Arnaud
Le 11 août 2023 04:43:42 GMT+02:00, "Nguyễn Hữu Khôi" < nguyenhuukhoinw@gmail.com> a écrit :
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry.
Nguyen Huu Khoi
On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
That's depending on your use-case. We have two types of VMs in our cloud, production VMs and the rest (self-service tenant networks). All production VMs are in provider networks, launched with config-drive and independent of the control plane. So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody. So if you have machines that are required to work independent of network nodes (control nodes) you should consider using such provider networks. Or move the VMs to those networks if you already have been using provider networks. Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com>:
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry.
Nguyen Huu Khoi
On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
Hello Eugen, I got what you mean, but "So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody." Can you explain this to me? Nguyen Huu Khoi On Sun, Aug 13, 2023 at 1:13 AM Eugen Block <eblock@nde.ag> wrote:
That's depending on your use-case. We have two types of VMs in our cloud, production VMs and the rest (self-service tenant networks). All production VMs are in provider networks, launched with config-drive and independent of the control plane. So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody. So if you have machines that are required to work independent of network nodes (control nodes) you should consider using such provider networks. Or move the VMs to those networks if you already have been using provider networks.
Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com>:
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry.
Nguyen Huu Khoi
On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
Which part is unclear? I added [1] where the physical network structure is explained a bit. The major advantage of using provider networks is that the compute nodes are themselves physically attached to the external (provider) network, so VM traffic doesn't have to be routed across control/network nodes. That mitigates a potential bottleneck if for example you only have on control node, lots of VMs and all of them send their traffic through that one control node. Spreading that load out across the compute nodes reduces the load on the control node(s). So if our control nodes fail the VMs in the provider networks are not affected. Apart from that, if your DHCP agents misbehave I definitely would look into that and fix it, independent of provider networks. The logs might reveal something, for example if the agents are overloaded. We saw that last year in a customer cloud where only one agent was available due to power outage on the second control node (third one was in maintenance). We also increased "dhcp_agents_per_network" to 2 (default 1) which seems to have helped there as well. We haven't had any issues since then. [1] https://docs.openstack.org/install-guide/launch-instance-networks-provider.h... Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com>:
Hello Eugen,
I got what you mean, but "So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody."
Can you explain this to me?
Nguyen Huu Khoi
On Sun, Aug 13, 2023 at 1:13 AM Eugen Block <eblock@nde.ag> wrote:
That's depending on your use-case. We have two types of VMs in our cloud, production VMs and the rest (self-service tenant networks). All production VMs are in provider networks, launched with config-drive and independent of the control plane. So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody. So if you have machines that are required to work independent of network nodes (control nodes) you should consider using such provider networks. Or move the VMs to those networks if you already have been using provider networks.
Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com>:
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry.
Nguyen Huu Khoi
On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
Hello Eugen, I got it know. Thank you very much. Nguyen Huu Khoi On Sun, Aug 13, 2023 at 3:09 PM Eugen Block <eblock@nde.ag> wrote:
Which part is unclear? I added [1] where the physical network structure is explained a bit. The major advantage of using provider networks is that the compute nodes are themselves physically attached to the external (provider) network, so VM traffic doesn't have to be routed across control/network nodes. That mitigates a potential bottleneck if for example you only have on control node, lots of VMs and all of them send their traffic through that one control node. Spreading that load out across the compute nodes reduces the load on the control node(s). So if our control nodes fail the VMs in the provider networks are not affected. Apart from that, if your DHCP agents misbehave I definitely would look into that and fix it, independent of provider networks. The logs might reveal something, for example if the agents are overloaded. We saw that last year in a customer cloud where only one agent was available due to power outage on the second control node (third one was in maintenance). We also increased "dhcp_agents_per_network" to 2 (default 1) which seems to have helped there as well. We haven't had any issues since then.
[1]
https://docs.openstack.org/install-guide/launch-instance-networks-provider.h...
Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com>:
Hello Eugen,
I got what you mean, but "So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody."
Can you explain this to me?
Nguyen Huu Khoi
On Sun, Aug 13, 2023 at 1:13 AM Eugen Block <eblock@nde.ag> wrote:
That's depending on your use-case. We have two types of VMs in our cloud, production VMs and the rest (self-service tenant networks). All production VMs are in provider networks, launched with config-drive and independent of the control plane. So as long as the compute nodes and the network infrastructure are running we can work, the less important VMs are not reachable during that time, of course, but it doesn't really hurt anybody. So if you have machines that are required to work independent of network nodes (control nodes) you should consider using such provider networks. Or move the VMs to those networks if you already have been using provider networks.
Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com>:
Yes. I agree with you but when all controller is down and some users need to reboot their instances then it won't have ip. This is my worry.
Nguyen Huu Khoi
On Fri, Aug 11, 2023 at 5:00 AM Clark Boylan <cboylan@sapwetik.org> wrote:
On Thu, Aug 10, 2023, at 12:00 AM, Nguyễn Hữu Khôi wrote:
Hello guys.
Sometimes I see that my instances won't get IP from Neutron DHCP Agent.
Is it good if we set a fixed port for instance and set a static ip in the instance's netplan configuration also?
We do this for IPv6 addresses on some of our servers as we've had problems with RAs. It seems to work fine in that case. Ideally, this shouldn't be necessary for any cloud instance though and it might be worth digging into why dhcp isn't working so that it can be corrected.
Nguyen Huu Khoi
participants (5)
-
Arnaud
-
Clark Boylan
-
Eugen Block
-
Jeremy Stanley
-
Nguyễn Hữu Khôi