[openstack][neutron] Using external network for another project
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant. I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly. Is it normal? Pls correct me if I am wrong Thank you. Nguyen Huu Khoi
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default. depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
Oh. This is because our tenants own their vlan and we want to use magnum so I do that. Could we do some changes? Who owns vlan networks can have full permission with this although external network? On Thu, May 1, 2025, 10:18 PM Sean Mooney <smooney@redhat.com> wrote:
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default.
depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was
always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
On 5/1/25 08:17, Sean Mooney wrote:
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default.
depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
I think I actually looked at this recently while re-triaging old bugs downstream and AFAICT it's still an issue: https://bugs.launchpad.net/nova/+bug/1675486 We have mentioned the idea of deferring to Neutron for this entirely (and I expect we can with no ill side effects) but I don't believe we formally discussed with the Neutron team to confirm whether it would be 100% OK to do. I think it would be nice to follow up on ^ and finally remove the external network policy check in Nova. -melwitt
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was
always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
On 01/05/2025 20:22, melanie witt wrote:
On 5/1/25 08:17, Sean Mooney wrote:
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default.
depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
I think I actually looked at this recently while re-triaging old bugs downstream and AFAICT it's still an issue:
https://bugs.launchpad.net/nova/+bug/1675486
We have mentioned the idea of deferring to Neutron for this entirely (and I expect we can with no ill side effects) but I don't believe we formally discussed with the Neutron team to confirm whether it would be 100% OK to do.
I think it would be nice to follow up on ^ and finally remove the external network policy check in Nova.
-melwitt
we merged https://review.opendev.org/c/openstack/nova/+/794360 3 years ago but looking at the code change it just removed the commen so we still have our own NETWORK_ATTACH_EXTERNAL polciy which is admin only. so i guess an operat can chagne that to project_member but your right we never actully removed the proejct. we probably should do that but ya we can see if neutron are still ok with this. im 99% sure that we talked about it with them in a past ptg but its definetly been a couple of years. one thing to keep in mind. external network do not have neutron router as there defautl gateway by default neutron implements the metadata proxy via the l3 agent if your ussing ml2/ovs so you need to enable the option to do that via the dhcp agent if you are still usign ml2/ovs or your vms wont have metadata. for ml2/ovn i think it will just work but perhaps not.
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was
always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
On 5/1/25 15:11, Sean Mooney wrote:
On 01/05/2025 20:22, melanie witt wrote:
On 5/1/25 08:17, Sean Mooney wrote:
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default.
depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
I think I actually looked at this recently while re-triaging old bugs downstream and AFAICT it's still an issue:
https://bugs.launchpad.net/nova/+bug/1675486
We have mentioned the idea of deferring to Neutron for this entirely (and I expect we can with no ill side effects) but I don't believe we formally discussed with the Neutron team to confirm whether it would be 100% OK to do.
I think it would be nice to follow up on ^ and finally remove the external network policy check in Nova.
-melwitt
we merged https://review.opendev.org/c/openstack/nova/+/794360 3 years ago
but looking at the code change it just removed the commen so we still have our own
NETWORK_ATTACH_EXTERNAL polciy which is admin only.
so i guess an operat can chagne that to project_member but your right we never actully removed the proejct.
we probably should do that but ya we can see if neutron are still ok with this.
im 99% sure that we talked about it with them in a past ptg but its definetly been a couple of years.
Good to know, it is highly likely that I missed it or forgot given the time that has passed since then. I'm supportive of going ahead and removing it now and give it a lot of bake time this cycle to uncover potential issues.
one thing to keep in mind. external network do not have neutron router as there defautl gateway
by default neutron implements the metadata proxy via the l3 agent if your ussing ml2/ovs so you need
to enable the option to do that via the dhcp agent if you are still usign ml2/ovs or your vms wont have
metadata. for ml2/ovn i think it will just work but perhaps not.
I didn't understand any of that unfortunately :) but if it's a potential problem with deferring to Neutron it's good to look out for it. -melwitt
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was
always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
On 01/05/2025 23:23, melanie witt wrote:
On 5/1/25 15:11, Sean Mooney wrote:
On 01/05/2025 20:22, melanie witt wrote:
On 5/1/25 08:17, Sean Mooney wrote:
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default.
depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
I think I actually looked at this recently while re-triaging old bugs downstream and AFAICT it's still an issue:
https://bugs.launchpad.net/nova/+bug/1675486
We have mentioned the idea of deferring to Neutron for this entirely (and I expect we can with no ill side effects) but I don't believe we formally discussed with the Neutron team to confirm whether it would be 100% OK to do.
I think it would be nice to follow up on ^ and finally remove the external network policy check in Nova.
-melwitt
we merged https://review.opendev.org/c/openstack/nova/+/794360 3 years ago
but looking at the code change it just removed the commen so we still have our own
NETWORK_ATTACH_EXTERNAL polciy which is admin only.
so i guess an operat can chagne that to project_member but your right we never actully removed the proejct.
we probably should do that but ya we can see if neutron are still ok with this.
im 99% sure that we talked about it with them in a past ptg but its definetly been a couple of years.
Good to know, it is highly likely that I missed it or forgot given the time that has passed since then.
I'm supportive of going ahead and removing it now and give it a lot of bake time this cycle to uncover potential issues.
one thing to keep in mind. external network do not have neutron router as there defautl gateway
by default neutron implements the metadata proxy via the l3 agent if your ussing ml2/ovs so you need
to enable the option to do that via the dhcp agent if you are still usign ml2/ovs or your vms wont have
metadata. for ml2/ovn i think it will just work but perhaps not.
I didn't understand any of that unfortunately :) but if it's a potential problem with deferring to Neutron it's good to look out for it.
-melwitt
sorry i didnt explain it well. the way the nova metadata api is made aviable to vm is via a neutron proxy, that proxy when not using ovn is provide by a iptables port forwading rule combinde with a proxy service that translates the users request without any auth into a authitcated request to novas metadata api. the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or by the neutron dhcp agent in in the dhcp servers namespace. the default when not using ovn is for it to be in the router namespace. because the derault route when you place a port directly on an external network is your datacenter router not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box unless you do metadata via the dhcp server instead. if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work. on the flip side port on an external network wont have there traffic nat'd when going to the internet external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data center routeres to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network. while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center. that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup initally.
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was
always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks. Nguyen Huu Khoi On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com> wrote:
On 01/05/2025 23:23, melanie witt wrote:
On 5/1/25 15:11, Sean Mooney wrote:
On 01/05/2025 20:22, melanie witt wrote:
On 5/1/25 08:17, Sean Mooney wrote:
On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote:
Hello. I created an external network for a specified project but I cannot create instances with the external network on this tenant.
I must create RBAC Policies with shared and external policies then my instances can get IP addresses and run properly.
booting to a external network is admin only by default.
depending on your release nova used to enforce this too but in newer release we defer to neutron to enforce that policy
I think I actually looked at this recently while re-triaging old bugs downstream and AFAICT it's still an issue:
https://bugs.launchpad.net/nova/+bug/1675486
We have mentioned the idea of deferring to Neutron for this entirely (and I expect we can with no ill side effects) but I don't believe we formally discussed with the Neutron team to confirm whether it would be 100% OK to do.
I think it would be nice to follow up on ^ and finally remove the external network policy check in Nova.
-melwitt
we merged https://review.opendev.org/c/openstack/nova/+/794360 3 years ago
but looking at the code change it just removed the commen so we still have our own
NETWORK_ATTACH_EXTERNAL polciy which is admin only.
so i guess an operat can chagne that to project_member but your right we never actully removed the proejct.
we probably should do that but ya we can see if neutron are still ok with this.
im 99% sure that we talked about it with them in a past ptg but its definetly been a couple of years.
Good to know, it is highly likely that I missed it or forgot given the time that has passed since then.
I'm supportive of going ahead and removing it now and give it a lot of bake time this cycle to uncover potential issues.
one thing to keep in mind. external network do not have neutron router as there defautl gateway
by default neutron implements the metadata proxy via the l3 agent if your ussing ml2/ovs so you need
to enable the option to do that via the dhcp agent if you are still usign ml2/ovs or your vms wont have
metadata. for ml2/ovn i think it will just work but perhaps not.
I didn't understand any of that unfortunately :) but if it's a potential problem with deferring to Neutron it's good to look out for it.
-melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a neutron proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service that translates the users request without any auth into a authitcated request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there traffic nat'd when going to the internet
external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data center routeres
to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup
initally.
Is it normal? Pls correct me if I am wrong
yes its normal, neutron may have change the defautl to allow non admins but booting direclty to an extenal network was
always considerd privaldged in the past.
Thank you.
Nguyen Huu Khoi
Hi, On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network? Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next week. Thanks, -Brian
On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com <mailto:smooney@redhat.com>> wrote:
On 01/05/2025 23:23, melanie witt wrote: > On 5/1/25 15:11, Sean Mooney wrote: >> >> On 01/05/2025 20:22, melanie witt wrote: >>> On 5/1/25 08:17, Sean Mooney wrote: >>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>> Hello. >>>>> I created an external network for a specified project but I cannot >>>>> create instances with the external network on this tenant. >>>>> >>>>> I must create RBAC Policies with shared and external policies then >>>>> my instances can get IP addresses and run properly. >>>>> >>>> booting to a external network is admin only by default. >>>> >>>> depending on your release nova used to enforce this too but in >>>> newer release we defer to neutron to enforce that policy >>> >>> I think I actually looked at this recently while re-triaging old >>> bugs downstream and AFAICT it's still an issue: >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> We have mentioned the idea of deferring to Neutron for this entirely >>> (and I expect we can with no ill side effects) but I don't believe >>> we formally discussed with the Neutron team to confirm whether it >>> would be 100% OK to do. >>> >>> I think it would be nice to follow up on ^ and finally remove the >>> external network policy check in Nova. >>> >>> -melwitt >> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 <https://review.opendev.org/c/openstack/nova/+/794360> 3 >> years ago >> >> but looking at the code change it just removed the commen so we still >> have our own >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> so i guess an operat can chagne that to project_member but your >> right we never actully removed the proejct. >> >> we probably should do that but ya we can see if neutron are still ok >> with this. >> >> im 99% sure that we talked about it with them in a past ptg but its >> definetly been a couple of years. > > Good to know, it is highly likely that I missed it or forgot given the > time that has passed since then. > > I'm supportive of going ahead and removing it now and give it a lot of > bake time this cycle to uncover potential issues. > >> one thing to keep in mind. external network do not have neutron >> router as there defautl gateway >> >> by default neutron implements the metadata proxy via the l3 agent if >> your ussing ml2/ovs so you need >> >> to enable the option to do that via the dhcp agent if you are still >> usign ml2/ovs or your vms wont have >> >> metadata. for ml2/ovn i think it will just work but perhaps not. > > I didn't understand any of that unfortunately :) but if it's a > potential problem with deferring to Neutron it's good to look out for it. > > -melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a neutron proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service that translates the users request without any auth into a authitcated request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there traffic nat'd when going to the internet
external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data center routeres
to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup
initally.
> >> >>> >>>>> Is it normal? Pls correct me if I am wrong >>>> >>>> yes its normal, neutron may have change the defautl to allow non >>>> admins but booting direclty to an extenal network was >>>> >>>> always considerd privaldged in the past. >>>> >>>>> >>>>> Thank you. >>>>> >>>>> Nguyen Huu Khoi >>>> >>> >> >
On 5/2/25 11:23, Brian Haley wrote:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next week.
Thanks for replying. In Nova, we would like to remove a RBAC policy check we currently have for permission to attach to an external network [1]: def _check_external_network_attach(self, context, nets): """Check if attaching to external network is permitted.""" if not context.can(servers_policies.NETWORK_ATTACH_EXTERNAL, fatal=False): for net in nets: # Perform this check here rather than in validate_networks to # ensure the check is performed every time # allocate_for_instance is invoked if net.get('router:external') and not net.get('shared'): raise exception.ExternalNetworkAttachForbidden( network_uuid=net['id']) The permission defaults to admin-only. This creates what seems like an unnecessary extra requirement for operators to configure RBAC policy in Nova to let non-admins use external networks. The ask from Neutron is, "can you advise if it is safe for Nova to remove this RBAC policy check and defer all permission decisions about external networks to Neutron?" We have expected the answer to this question is "yes" but wanted to make sure in case there are any gotchas that would happen if we remove it. -melwitt [1] https://github.com/openstack/nova/blob/a5bcaf69b1a80d4d02fe092900471a6e7a28e...
[...]
Hi, W dniu 2.05.2025 o 20:23, Brian Haley pisze:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... - so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it) Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next week.
Thanks,
-Brian
On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com <mailto:smooney@redhat.com>> wrote:
On 01/05/2025 23:23, melanie witt wrote: > On 5/1/25 15:11, Sean Mooney wrote: >> >> On 01/05/2025 20:22, melanie witt wrote: >>> On 5/1/25 08:17, Sean Mooney wrote: >>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>> Hello. >>>>> I created an external network for a specified project but I cannot >>>>> create instances with the external network on this tenant. >>>>> >>>>> I must create RBAC Policies with shared and external policies then >>>>> my instances can get IP addresses and run properly. >>>>> >>>> booting to a external network is admin only by default. >>>> >>>> depending on your release nova used to enforce this too but in >>>> newer release we defer to neutron to enforce that policy >>> >>> I think I actually looked at this recently while re-triaging old >>> bugs downstream and AFAICT it's still an issue: >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> We have mentioned the idea of deferring to Neutron for this entirely >>> (and I expect we can with no ill side effects) but I don't believe >>> we formally discussed with the Neutron team to confirm whether it >>> would be 100% OK to do. >>> >>> I think it would be nice to follow up on ^ and finally remove the >>> external network policy check in Nova. >>> >>> -melwitt >> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 <https://review.opendev.org/c/openstack/nova/+/794360> 3 >> years ago >> >> but looking at the code change it just removed the commen so we still >> have our own >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> so i guess an operat can chagne that to project_member but your >> right we never actully removed the proejct. >> >> we probably should do that but ya we can see if neutron are still ok >> with this. >> >> im 99% sure that we talked about it with them in a past ptg but its >> definetly been a couple of years. > > Good to know, it is highly likely that I missed it or forgot given the > time that has passed since then. > > I'm supportive of going ahead and removing it now and give it a lot of > bake time this cycle to uncover potential issues. > >> one thing to keep in mind. external network do not have neutron >> router as there defautl gateway >> >> by default neutron implements the metadata proxy via the l3 agent if >> your ussing ml2/ovs so you need >> >> to enable the option to do that via the dhcp agent if you are still >> usign ml2/ovs or your vms wont have >> >> metadata. for ml2/ovn i think it will just work but perhaps not. > > I didn't understand any of that unfortunately :) but if it's a > potential problem with deferring to Neutron it's good to look out for it. > > -melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a neutron proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service that translates the users request without any auth into a authitcated request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there traffic nat'd when going to the internet
external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data center routeres
to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup
initally.
> >> >>> >>>>> Is it normal? Pls correct me if I am wrong >>>> >>>> yes its normal, neutron may have change the defautl to allow non >>>> admins but booting direclty to an extenal network was >>>> >>>> always considerd privaldged in the past. >>>> >>>>> >>>>> Thank you. >>>>> >>>>> Nguyen Huu Khoi >>>> >>> >> >
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud. On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
W dniu 2.05.2025 o 20:23, Brian Haley pisze:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... - so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it)
Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next week.
Thanks,
-Brian
On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com <mailto:smooney@redhat.com>> wrote:
On 01/05/2025 23:23, melanie witt wrote: > On 5/1/25 15:11, Sean Mooney wrote: >> >> On 01/05/2025 20:22, melanie witt wrote: >>> On 5/1/25 08:17, Sean Mooney wrote: >>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>> Hello. >>>>> I created an external network for a specified project but I cannot >>>>> create instances with the external network on this tenant. >>>>> >>>>> I must create RBAC Policies with shared and external policies then >>>>> my instances can get IP addresses and run properly. >>>>> >>>> booting to a external network is admin only by default. >>>> >>>> depending on your release nova used to enforce this too but in >>>> newer release we defer to neutron to enforce that policy >>> >>> I think I actually looked at this recently while re-triaging old >>> bugs downstream and AFAICT it's still an issue: >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> We have mentioned the idea of deferring to Neutron for this entirely >>> (and I expect we can with no ill side effects) but I don't believe >>> we formally discussed with the Neutron team to confirm whether it >>> would be 100% OK to do. >>> >>> I think it would be nice to follow up on ^ and finally remove the >>> external network policy check in Nova. >>> >>> -melwitt >> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 <https://review.opendev.org/c/openstack/nova/+/794360> 3 >> years ago >> >> but looking at the code change it just removed the commen so we still >> have our own >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> so i guess an operat can chagne that to project_member but your >> right we never actully removed the proejct. >> >> we probably should do that but ya we can see if neutron are still ok >> with this. >> >> im 99% sure that we talked about it with them in a past ptg but its >> definetly been a couple of years. > > Good to know, it is highly likely that I missed it or forgot given the > time that has passed since then. > > I'm supportive of going ahead and removing it now and give it a lot of > bake time this cycle to uncover potential issues. > >> one thing to keep in mind. external network do not have neutron >> router as there defautl gateway >> >> by default neutron implements the metadata proxy via the l3 agent if >> your ussing ml2/ovs so you need >> >> to enable the option to do that via the dhcp agent if you are still >> usign ml2/ovs or your vms wont have >> >> metadata. for ml2/ovn i think it will just work but perhaps not. > > I didn't understand any of that unfortunately :) but if it's a > potential problem with deferring to Neutron it's good to look out for it. > > -melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a neutron proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service that translates the users request without any auth into a authitcated request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there traffic nat'd when going to the internet
external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data center routeres
to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup
initally.
> >> >>> >>>>> Is it normal? Pls correct me if I am wrong >>>> >>>> yes its normal, neutron may have change the defautl to allow non >>>> admins but booting direclty to an extenal network was >>>> >>>> always considerd privaldged in the past. >>>> >>>>> >>>>> Thank you. >>>>> >>>>> Nguyen Huu Khoi >>>> >>> >> >
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hello. Could we approve this feature in future releases? Nguyen Huu Khoi On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud.
On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
W dniu 2.05.2025 o 20:23, Brian Haley pisze:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... - so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it)
Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next week.
Thanks,
-Brian
On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com <mailto:smooney@redhat.com>> wrote:
On 01/05/2025 23:23, melanie witt wrote: > On 5/1/25 15:11, Sean Mooney wrote: >> >> On 01/05/2025 20:22, melanie witt wrote: >>> On 5/1/25 08:17, Sean Mooney wrote: >>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>> Hello. >>>>> I created an external network for a specified project but I cannot >>>>> create instances with the external network on this tenant. >>>>> >>>>> I must create RBAC Policies with shared and external
then >>>>> my instances can get IP addresses and run properly. >>>>> >>>> booting to a external network is admin only by default. >>>> >>>> depending on your release nova used to enforce this too but
in
>>>> newer release we defer to neutron to enforce that policy >>> >>> I think I actually looked at this recently while re-triaging old >>> bugs downstream and AFAICT it's still an issue: >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> We have mentioned the idea of deferring to Neutron for this entirely >>> (and I expect we can with no ill side effects) but I don't believe >>> we formally discussed with the Neutron team to confirm whether it >>> would be 100% OK to do. >>> >>> I think it would be nice to follow up on ^ and finally remove the >>> external network policy check in Nova. >>> >>> -melwitt >> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 <https://review.opendev.org/c/openstack/nova/+/794360> 3 >> years ago >> >> but looking at the code change it just removed the commen so we still >> have our own >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> so i guess an operat can chagne that to project_member but
your
>> right we never actully removed the proejct. >> >> we probably should do that but ya we can see if neutron are still ok >> with this. >> >> im 99% sure that we talked about it with them in a past ptg but its >> definetly been a couple of years. > > Good to know, it is highly likely that I missed it or forgot given the > time that has passed since then. > > I'm supportive of going ahead and removing it now and give it a lot of > bake time this cycle to uncover potential issues. > >> one thing to keep in mind. external network do not have neutron >> router as there defautl gateway >> >> by default neutron implements the metadata proxy via the l3 agent if >> your ussing ml2/ovs so you need >> >> to enable the option to do that via the dhcp agent if you are still >> usign ml2/ovs or your vms wont have >> >> metadata. for ml2/ovn i think it will just work but perhaps
not.
> > I didn't understand any of that unfortunately :) but if it's a > potential problem with deferring to Neutron it's good to look
out
for it. > > -melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a
neutron
proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service
translates the users request without any auth into a authitcated request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there
nat'd when going to the internet
external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data
policies that traffic center
routeres
to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup
initally.
> >> >>> >>>>> Is it normal? Pls correct me if I am wrong >>>> >>>> yes its normal, neutron may have change the defautl to allow non >>>> admins but booting direclty to an extenal network was >>>> >>>> always considerd privaldged in the past. >>>> >>>>> >>>>> Thank you. >>>>> >>>>> Nguyen Huu Khoi >>>> >>> >> >
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hi, Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello.
Could we approve this feature in future releases?
What feature you are exactly asking for?
Nguyen Huu Khoi
On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud.
On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
W dniu 2.05.2025 o 20:23, Brian Haley pisze:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... - so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it)
Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next week.
Thanks,
-Brian
On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com <mailto:smooney@redhat.com>> wrote:
On 01/05/2025 23:23, melanie witt wrote: > On 5/1/25 15:11, Sean Mooney wrote: >> >> On 01/05/2025 20:22, melanie witt wrote: >>> On 5/1/25 08:17, Sean Mooney wrote: >>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>> Hello. >>>>> I created an external network for a specified project but I cannot >>>>> create instances with the external network on this tenant. >>>>> >>>>> I must create RBAC Policies with shared and external
then >>>>> my instances can get IP addresses and run properly. >>>>> >>>> booting to a external network is admin only by default. >>>> >>>> depending on your release nova used to enforce this too but
in
>>>> newer release we defer to neutron to enforce that policy >>> >>> I think I actually looked at this recently while re-triaging old >>> bugs downstream and AFAICT it's still an issue: >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> We have mentioned the idea of deferring to Neutron for this entirely >>> (and I expect we can with no ill side effects) but I don't believe >>> we formally discussed with the Neutron team to confirm whether it >>> would be 100% OK to do. >>> >>> I think it would be nice to follow up on ^ and finally remove the >>> external network policy check in Nova. >>> >>> -melwitt >> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 <https://review.opendev.org/c/openstack/nova/+/794360> 3 >> years ago >> >> but looking at the code change it just removed the commen so we still >> have our own >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> so i guess an operat can chagne that to project_member but
your
>> right we never actully removed the proejct. >> >> we probably should do that but ya we can see if neutron are still ok >> with this. >> >> im 99% sure that we talked about it with them in a past ptg but its >> definetly been a couple of years. > > Good to know, it is highly likely that I missed it or forgot given the > time that has passed since then. > > I'm supportive of going ahead and removing it now and give it a lot of > bake time this cycle to uncover potential issues. > >> one thing to keep in mind. external network do not have neutron >> router as there defautl gateway >> >> by default neutron implements the metadata proxy via the l3 agent if >> your ussing ml2/ovs so you need >> >> to enable the option to do that via the dhcp agent if you are still >> usign ml2/ovs or your vms wont have >> >> metadata. for ml2/ovn i think it will just work but perhaps
not.
> > I didn't understand any of that unfortunately :) but if it's a > potential problem with deferring to Neutron it's good to look
out
for it. > > -melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a
neutron
proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service
translates the users request without any auth into a authitcated request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either the neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil not be proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a metadta proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there
nat'd when going to the internet
external network in neutron are expected to have subnets that are routeable to the WAN so operator need to confiugre there data
policies that traffic center
routeres
to perform nat if that is not the case and they will also need to provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external vlan network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to your datacenter but it need a lot of hand holding from the operator to get setup
initally.
> >> >>> >>>>> Is it normal? Pls correct me if I am wrong >>>> >>>> yes its normal, neutron may have change the defautl to allow non >>>> admins but booting direclty to an extenal network was >>>> >>>> always considerd privaldged in the past. >>>> >>>>> >>>>> Thank you. >>>>> >>>>> Nguyen Huu Khoi >>>> >>> >> >
-- Slawek Kaplonski Principal Software Engineer Red Hat
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hello. As I mentioned earlier, We have multi tenants and each has their own vlan networks. When I create vlan networks for them then I mask it as an external network(with this tenant not * in RABC Policy) and they cannot use this network for launching instances. If I want to use this network I must create a RABC policy shared with this project. Nguyen Huu Khoi On Tue, May 13, 2025 at 2:30 PM Sławek Kapłoński <skaplons@redhat.com> wrote:
Hi,
Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello.
Could we approve this feature in future releases?
What feature you are exactly asking for?
Nguyen Huu Khoi
On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi <
wrote:
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud.
On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
W dniu 2.05.2025 o 20:23, Brian Haley pisze:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote:
Could we find a solution for this case? Because they own vlan networks Currently, I can create RBAC Policies to allow tenants using external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128...
- so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it)
Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next
week.
Thanks,
-Brian
On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com <mailto:smooney@redhat.com>> wrote:
On 01/05/2025 23:23, melanie witt wrote: > On 5/1/25 15:11, Sean Mooney wrote: >> >> On 01/05/2025 20:22, melanie witt wrote: >>> On 5/1/25 08:17, Sean Mooney wrote: >>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>> Hello. >>>>> I created an external network for a specified project
cannot >>>>> create instances with the external network on this
tenant.
>>>>> >>>>> I must create RBAC Policies with shared and external
but I policies
then >>>>> my instances can get IP addresses and run properly. >>>>> >>>> booting to a external network is admin only by default. >>>> >>>> depending on your release nova used to enforce this too
but in
>>>> newer release we defer to neutron to enforce that policy >>> >>> I think I actually looked at this recently while
re-triaging
old >>> bugs downstream and AFAICT it's still an issue: >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> We have mentioned the idea of deferring to Neutron for
entirely >>> (and I expect we can with no ill side effects) but I don't believe >>> we formally discussed with the Neutron team to confirm whether it >>> would be 100% OK to do. >>> >>> I think it would be nice to follow up on ^ and finally remove the >>> external network policy check in Nova. >>> >>> -melwitt >> >> we merged
https://review.opendev.org/c/openstack/nova/+/794360
<https://review.opendev.org/c/openstack/nova/+/794360> 3 >> years ago >> >> but looking at the code change it just removed the commen
so we
still >> have our own >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> so i guess an operat can chagne that to project_member but
your
>> right we never actully removed the proejct. >> >> we probably should do that but ya we can see if neutron are still ok >> with this. >> >> im 99% sure that we talked about it with them in a past ptg but its >> definetly been a couple of years. > > Good to know, it is highly likely that I missed it or forgot given the > time that has passed since then. > > I'm supportive of going ahead and removing it now and give
it a
lot of > bake time this cycle to uncover potential issues. > >> one thing to keep in mind. external network do not have
neutron
>> router as there defautl gateway >> >> by default neutron implements the metadata proxy via the l3 agent if >> your ussing ml2/ovs so you need >> >> to enable the option to do that via the dhcp agent if you
still >> usign ml2/ovs or your vms wont have >> >> metadata. for ml2/ovn i think it will just work but perhaps not. > > I didn't understand any of that unfortunately :) but if it's a > potential problem with deferring to Neutron it's good to look out for it. > > -melwitt
sorry i didnt explain it well.
the way the nova metadata api is made aviable to vm is via a neutron proxy, that proxy when not using ovn is provide
by a iptables port forwading rule combinde with a proxy service
are that
translates the users request without any auth into a
authitcated
request
to novas metadata api.
the metadtat proxy and iptables rule can be deploy be either
neutron l3 agent in the router network namespace or
by the neutron dhcp agent in in the dhcp servers namespace.
the default when not using ovn is for it to be in the router namespace.
because the derault route when you place a port directly on an external network is your datacenter router
not a neutron one a request form teh vm to 169.whatever wil
not be
proxied to nova so metadta in the vm wont work out of the box
unless you do metadata via the dhcp server instead.
if your using ovn i think this just work because ovn has a
nguyenhuukhoinw@gmail.com> this the metadta
proxy on every compute node and it isntalls an openflow rule
that matches on the destination ip to redirect it to that local proxy. so for ovn i belive it will just work.
on the flip side port on an external network wont have there
traffic
nat'd when going to the internet
external network in neutron are expected to have subnets that
are
routeable to the WAN so operator need to confiugre there data
center
routeres
to perform nat if that is not the case and they will also need
to
provide the eant with an ip pool or precreated subnet+network.
while not un heard of providign tenant with there own external
vlan
network like this is not very clody in the normal self service workflow but it can have some advantages by allowing the routing to be delgated to a hardware router in your data center.
that can allow your tenats to bring there own ip block to to
your
datacenter but it need a lot of hand holding from the operator
to
get setup
initally.
> >> >>> >>>>> Is it normal? Pls correct me if I am wrong >>>> >>>> yes its normal, neutron may have change the defautl to allow non >>>> admins but booting direclty to an extenal network was >>>> >>>> always considerd privaldged in the past. >>>> >>>>> >>>>> Thank you. >>>>> >>>>> Nguyen Huu Khoi >>>> >>> >> >
-- Slawek Kaplonski Principal Software Engineer Red Hat
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hi, Dnia wtorek, 13 maja 2025 13:41:47 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello. As I mentioned earlier, We have multi tenants and each has their own vlan networks. When I create vlan networks for them then I mask it as an external network(with this tenant not * in RABC Policy) and they cannot use this network for launching instances. If I want to use this network I must create a RABC policy shared with this project.
Yes, that is how it works. You can also share network with all tenants (use "*" as target). If you think about any other new functionality, feel free to propose new RFE [1] which we can discuss later during the drivers meeting. [1] https://docs.openstack.org/neutron/latest/contributor/policies/blueprints.ht...
Nguyen Huu Khoi
On Tue, May 13, 2025 at 2:30 PM Sławek Kapłoński <skaplons@redhat.com> wrote:
Hi,
Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello.
Could we approve this feature in future releases?
What feature you are exactly asking for?
Nguyen Huu Khoi
On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi <
wrote:
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud.
On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
W dniu 2.05.2025 o 20:23, Brian Haley pisze:
Hi,
On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote: > Could we find a solution for this case? Because they own vlan > networks Currently, I can create RBAC Policies to allow tenants using > external networks.
I've read this thread a couple of times and I still don't exactly understand what the ask is from Neutron. This tenant "owns" a VLAN network and wants to integrate it and a range of IP addresses into Neutron as an external network?
I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128...
- so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it)
Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
Also, I don't recall any previous discussions that took place, maybe someone else on the Neutron team does though and can comment next
week.
Thanks,
-Brian
> On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com > <mailto:smooney@redhat.com>> wrote: > > > On 01/05/2025 23:23, melanie witt wrote: > > On 5/1/25 15:11, Sean Mooney wrote: > >> > >> On 01/05/2025 20:22, melanie witt wrote: > >>> On 5/1/25 08:17, Sean Mooney wrote: > >>>> > >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: > >>>>> Hello. > >>>>> I created an external network for a specified project
> cannot > >>>>> create instances with the external network on this tenant. > >>>>> > >>>>> I must create RBAC Policies with shared and external
but I policies
> then > >>>>> my instances can get IP addresses and run properly. > >>>>> > >>>> booting to a external network is admin only by default. > >>>> > >>>> depending on your release nova used to enforce this too but in > >>>> newer release we defer to neutron to enforce that policy > >>> > >>> I think I actually looked at this recently while re-triaging > old > >>> bugs downstream and AFAICT it's still an issue: > >>> > >>> https://bugs.launchpad.net/nova/+bug/1675486 > <https://bugs.launchpad.net/nova/+bug/1675486> > >>> > >>> We have mentioned the idea of deferring to Neutron for
> entirely > >>> (and I expect we can with no ill side effects) but I don't > believe > >>> we formally discussed with the Neutron team to confirm > whether it > >>> would be 100% OK to do. > >>> > >>> I think it would be nice to follow up on ^ and finally > remove the > >>> external network policy check in Nova. > >>> > >>> -melwitt > >> > >> we merged https://review.opendev.org/c/openstack/nova/+/794360 > <https://review.opendev.org/c/openstack/nova/+/794360> 3 > >> years ago > >> > >> but looking at the code change it just removed the commen so we > still > >> have our own > >> > >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. > >> > >> so i guess an operat can chagne that to project_member but your > >> right we never actully removed the proejct. > >> > >> we probably should do that but ya we can see if neutron are > still ok > >> with this. > >> > >> im 99% sure that we talked about it with them in a past ptg > but its > >> definetly been a couple of years. > > > > Good to know, it is highly likely that I missed it or forgot > given the > > time that has passed since then. > > > > I'm supportive of going ahead and removing it now and give it a > lot of > > bake time this cycle to uncover potential issues. > > > >> one thing to keep in mind. external network do not have neutron > >> router as there defautl gateway > >> > >> by default neutron implements the metadata proxy via the l3 > agent if > >> your ussing ml2/ovs so you need > >> > >> to enable the option to do that via the dhcp agent if you are > still > >> usign ml2/ovs or your vms wont have > >> > >> metadata. for ml2/ovn i think it will just work but perhaps not. > > > > I didn't understand any of that unfortunately :) but if it's a > > potential problem with deferring to Neutron it's good to look out > for it. > > > > -melwitt > > sorry i didnt explain it well. > > the way the nova metadata api is made aviable to vm is via a neutron > proxy, that proxy when not using ovn is provide > > by a iptables port forwading rule combinde with a proxy service that > translates the users request without any auth into a authitcated > request > > to novas metadata api. > > the metadtat proxy and iptables rule can be deploy be either
> neutron > l3 agent in the router network namespace or > > by the neutron dhcp agent in in the dhcp servers namespace. > > the default when not using ovn is for it to be in the router > namespace. > > because the derault route when you place a port directly on an > external > network is your datacenter router > > not a neutron one a request form teh vm to 169.whatever wil not be > proxied to nova so metadta in the vm wont work out of the box > > unless you do metadata via the dhcp server instead. > > > if your using ovn i think this just work because ovn has a
nguyenhuukhoinw@gmail.com> this the metadta
> proxy > on every compute node and it isntalls an openflow rule > > that matches on the destination ip to redirect it to that local > proxy. > so for ovn i belive it will just work. > > on the flip side port on an external network wont have there traffic > nat'd when going to the internet > > external network in neutron are expected to have subnets that are > routeable to the WAN so operator need to confiugre there data center > routeres > > to perform nat if that is not the case and they will also need to > provide the eant with an ip pool or precreated subnet+network. > > while not un heard of providign tenant with there own external vlan > network like this is not very clody in the normal self service > workflow but > it can have some advantages by allowing the routing to be > delgated to a > hardware router in your data center. > > that can allow your tenats to bring there own ip block to to your > datacenter but it need a lot of hand holding from the operator to > get setup > > initally. > > > > >> > >>> > >>>>> Is it normal? Pls correct me if I am wrong > >>>> > >>>> yes its normal, neutron may have change the defautl to > allow non > >>>> admins but booting direclty to an extenal network was > >>>> > >>>> always considerd privaldged in the past. > >>>> > >>>>> > >>>>> Thank you. > >>>>> > >>>>> Nguyen Huu Khoi > >>>> > >>> > >> > > >
-- Slawek Kaplonski Principal Software Engineer Red Hat
-- Slawek Kaplonski Principal Software Engineer Red Hat
-- Slawek Kaplonski Principal Software Engineer Red Hat
On 13/05/2025 15:56, Sławek Kapłoński wrote:
Hi,
Dnia wtorek, 13 maja 2025 13:41:47 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello. As I mentioned earlier, We have multi tenants and each has their own vlan networks. When I create vlan networks for them then I mask it as an external network(with this tenant not * in RABC Policy) and they cannot use this network for launching instances.
just to be clear you are creating the vlans provider network for them because you cannot use neutron support for vlan tenant network to have them create the network themselves either because of business constraints or because in your deployment you need to assign specific vlan id to the tenant and your using data center routers to provide them with external connectivtiy correct. precreating an external network for a tenant is not uncommon but normally the tenant will create an additional network and a nutron router and then expose there vms via floating ips if you have not configured something liek netowrkign-bgp to make there network routable. the current issues you are having are beign caused because you are trying to allow unprivladge end user to add ports to an external network which by default required admin permissions. you can adjust that in your nova and neutron policy if desired.
If I want to use this network I must create a RABC policy shared with this project. Yes, that is how it works. You can also share network with all tenants (use "*" as target). If you think about any other new functionality, feel free to propose new RFE [1] which we can discuss later during the drivers meeting.
[1] https://docs.openstack.org/neutron/latest/contributor/policies/blueprints.ht...
that wont be enough because nova by default only allow admins to use external networks. https://github.com/openstack/nova/blob/023be4f5615255d06f61b393605da1308f045... the question we (nova) had to neturon was is it ok if we relax that to base.PROJECT_MEMBER_OR_ADMIN and then rely on neutron RBAC to determin if 1 the network should be visible and 2 if the attachment to it is allowed. baiscly is it ok for nova to delegate that policy enforcement to neutron. note we dont curretnly have anyone working on that although it was something that was stared a few release ago and never got completed. so if neutron is ok with the change and folk want to work on it we can review.
Nguyen Huu Khoi
On Tue, May 13, 2025 at 2:30 PM Sławek Kapłoński <skaplons@redhat.com> wrote:
Hi,
Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello.
Could we approve this feature in future releases? What feature you are exactly asking for?
Nguyen Huu Khoi
On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi < nguyenhuukhoinw@gmail.com> wrote:
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud.
On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote:
Hi,
W dniu 2.05.2025 o 20:23, Brian Haley pisze: > Hi, > > On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote: >> Could we find a solution for this case? Because they own vlan >> networks Currently, I can create RBAC Policies to allow tenants using >> external networks. > I've read this thread a couple of times and I still don't exactly > understand what the ask is from Neutron. This tenant "owns" a VLAN > network and wants to integrate it and a range of IP addresses into > Neutron as an external network? I also have some problem to understand what the issue is really. First of all I just checked in our code, and policy for port creation don't tell anything about external network. It is allowed for the ADMIN_OR_PROJECT_MEMBER or SERVICE - see
- so if tenant owns this vlan network, it should be doable to create port in such network IMO (but I don't have env up now to check it)
Another thing which I am not sure is why this vlan network needs to be marked as external - are You using it also as external gateway for routers and have floating ips from it? If not (and that's what I understood from the thread so far) you don't need to set `router:external` to `True`.
> Also, I don't recall any previous discussions that took place, maybe > someone else on the Neutron team does though and can comment next week. > Thanks, > > -Brian > >> On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com >> <mailto:smooney@redhat.com>> wrote: >> >> >> On 01/05/2025 23:23, melanie witt wrote: >> > On 5/1/25 15:11, Sean Mooney wrote: >> >> >> >> On 01/05/2025 20:22, melanie witt wrote: >> >>> On 5/1/25 08:17, Sean Mooney wrote: >> >>>> >> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >> >>>>> Hello. >> >>>>> I created an external network for a specified project but I >> cannot >> >>>>> create instances with the external network on this tenant. >> >>>>> >> >>>>> I must create RBAC Policies with shared and external policies >> then >> >>>>> my instances can get IP addresses and run properly. >> >>>>> >> >>>> booting to a external network is admin only by default. >> >>>> >> >>>> depending on your release nova used to enforce this too but in >> >>>> newer release we defer to neutron to enforce that policy >> >>> >> >>> I think I actually looked at this recently while re-triaging >> old >> >>> bugs downstream and AFAICT it's still an issue: >> >>> >> >>> https://bugs.launchpad.net/nova/+bug/1675486 >> <https://bugs.launchpad.net/nova/+bug/1675486> >> >>> >> >>> We have mentioned the idea of deferring to Neutron for
>> entirely >> >>> (and I expect we can with no ill side effects) but I don't >> believe >> >>> we formally discussed with the Neutron team to confirm >> whether it >> >>> would be 100% OK to do. >> >>> >> >>> I think it would be nice to follow up on ^ and finally >> remove the >> >>> external network policy check in Nova. >> >>> >> >>> -melwitt >> >> >> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 >> <https://review.opendev.org/c/openstack/nova/+/794360> 3 >> >> years ago >> >> >> >> but looking at the code change it just removed the commen so we >> still >> >> have our own >> >> >> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >> >> >> >> so i guess an operat can chagne that to project_member but your >> >> right we never actully removed the proejct. >> >> >> >> we probably should do that but ya we can see if neutron are >> still ok >> >> with this. >> >> >> >> im 99% sure that we talked about it with them in a past ptg >> but its >> >> definetly been a couple of years. >> > >> > Good to know, it is highly likely that I missed it or forgot >> given the >> > time that has passed since then. >> > >> > I'm supportive of going ahead and removing it now and give it a >> lot of >> > bake time this cycle to uncover potential issues. >> > >> >> one thing to keep in mind. external network do not have neutron >> >> router as there defautl gateway >> >> >> >> by default neutron implements the metadata proxy via the l3 >> agent if >> >> your ussing ml2/ovs so you need >> >> >> >> to enable the option to do that via the dhcp agent if you are >> still >> >> usign ml2/ovs or your vms wont have >> >> >> >> metadata. for ml2/ovn i think it will just work but perhaps not. >> > >> > I didn't understand any of that unfortunately :) but if it's a >> > potential problem with deferring to Neutron it's good to look out >> for it. >> > >> > -melwitt >> >> sorry i didnt explain it well. >> >> the way the nova metadata api is made aviable to vm is via a neutron >> proxy, that proxy when not using ovn is provide >> >> by a iptables port forwading rule combinde with a proxy service that >> translates the users request without any auth into a authitcated >> request >> >> to novas metadata api. >> >> the metadtat proxy and iptables rule can be deploy be either
>> neutron >> l3 agent in the router network namespace or >> >> by the neutron dhcp agent in in the dhcp servers namespace. >> >> the default when not using ovn is for it to be in the router >> namespace. >> >> because the derault route when you place a port directly on an >> external >> network is your datacenter router >> >> not a neutron one a request form teh vm to 169.whatever wil not be >> proxied to nova so metadta in the vm wont work out of the box >> >> unless you do metadata via the dhcp server instead. >> >> >> if your using ovn i think this just work because ovn has a
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... this the metadta
>> proxy >> on every compute node and it isntalls an openflow rule >> >> that matches on the destination ip to redirect it to that local >> proxy. >> so for ovn i belive it will just work. >> >> on the flip side port on an external network wont have there traffic >> nat'd when going to the internet >> >> external network in neutron are expected to have subnets that are >> routeable to the WAN so operator need to confiugre there data center >> routeres >> >> to perform nat if that is not the case and they will also need to >> provide the eant with an ip pool or precreated subnet+network. >> >> while not un heard of providign tenant with there own external vlan >> network like this is not very clody in the normal self service >> workflow but >> it can have some advantages by allowing the routing to be >> delgated to a >> hardware router in your data center. >> >> that can allow your tenats to bring there own ip block to to your >> datacenter but it need a lot of hand holding from the operator to >> get setup >> >> initally. >> >> > >> >> >> >>> >> >>>>> Is it normal? Pls correct me if I am wrong >> >>>> >> >>>> yes its normal, neutron may have change the defautl to >> allow non >> >>>> admins but booting direclty to an extenal network was >> >>>> >> >>>> always considerd privaldged in the past. >> >>>> >> >>>>> >> >>>>> Thank you. >> >>>>> >> >>>>> Nguyen Huu Khoi >> >>>> >> >>> >> >> >> > >> -- Slawek Kaplonski Principal Software Engineer Red Hat
-- Slawek Kaplonski Principal Software Engineer Red Hat
Hi, It is very detailed. Thank you very much @Sean Mooney <smooney@redhat.com> . From an operator perspective, It will be useful if we can allow this. Nguyen Huu Khoi On Wed, May 14, 2025 at 12:38 AM Sean Mooney <smooney@redhat.com> wrote:
On 13/05/2025 15:56, Sławek Kapłoński wrote:
Hi,
Dnia wtorek, 13 maja 2025 13:41:47 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello. As I mentioned earlier, We have multi tenants and each has their own vlan networks. When I create vlan networks for them then I mask it as an external network(with this tenant not * in RABC Policy) and they cannot use this network for launching instances.
just to be clear you are creating the vlans provider network for them
because you cannot use neutron support for vlan tenant network to have them create the network
themselves either because of business constraints or because in your deployment you need to
assign specific vlan id to the tenant and your using data center routers to provide them with external connectivtiy correct.
precreating an external network for a tenant is not uncommon but normally the tenant will create an additional network
and a nutron router and then expose there vms via floating ips if you have not configured something liek netowrkign-bgp
to make there network routable.
the current issues you are having are beign caused because you are trying to allow unprivladge end user to add ports
to an external network which by default required admin permissions.
you can adjust that in your nova and neutron policy if desired.
If I want to use this network I must create a RABC policy shared with this project. Yes, that is how it works. You can also share network with all tenants (use "*" as target). If you think about any other new functionality, feel free to propose new RFE [1] which we can discuss later during the drivers meeting.
[1] https://docs.openstack.org/neutron/latest/contributor/policies/blueprints.ht...
that wont be enough because nova by default only allow admins to use external networks.
https://github.com/openstack/nova/blob/023be4f5615255d06f61b393605da1308f045...
the question we (nova) had to neturon was is it ok if we relax that to
base.PROJECT_MEMBER_OR_ADMIN
and then rely on neutron RBAC to determin if
1 the network should be visible and 2 if the attachment to it is allowed.
baiscly is it ok for nova to delegate that policy enforcement to neutron.
note we dont curretnly have anyone working on that although it was something that was stared a few release ago and
never got completed. so if neutron is ok with the change and folk want to work on it we can review.
Nguyen Huu Khoi
On Tue, May 13, 2025 at 2:30 PM Sławek Kapłoński <skaplons@redhat.com> wrote:
Hi,
Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze:
Hello.
Could we approve this feature in future releases? What feature you are exactly asking for?
Nguyen Huu Khoi
On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi < nguyenhuukhoinw@gmail.com> wrote:
Hello. Because other tenants need external network to use with routers, float ips as well as magnum. I think it will be need for private cloud.
On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> wrote: > Hi, > > W dniu 2.05.2025 o 20:23, Brian Haley pisze: >> Hi, >> >> On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote: >>> Could we find a solution for this case? Because they own vlan >>> networks Currently, I can create RBAC Policies to allow tenants using >>> external networks. >> I've read this thread a couple of times and I still don't exactly >> understand what the ask is from Neutron. This tenant "owns" a VLAN >> network and wants to integrate it and a range of IP addresses into >> Neutron as an external network? > I also have some problem to understand what the issue is really. First > of all I just checked in our code, and policy for port creation don't > tell anything about external network. It is allowed for the > ADMIN_OR_PROJECT_MEMBER or SERVICE - see > >
> - so if tenant owns this vlan network, it should be doable to create > port in such network IMO (but I don't have env up now to check it) > > Another thing which I am not sure is why this vlan network needs to be > marked as external - are You using it also as external gateway for > routers and have floating ips from it? If not (and that's what I > understood from the thread so far) you don't need to set > `router:external` to `True`. > > >> Also, I don't recall any previous discussions that took place, maybe >> someone else on the Neutron team does though and can comment next week. >> Thanks, >> >> -Brian >> >>> On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com >>> <mailto:smooney@redhat.com>> wrote: >>> >>> >>> On 01/05/2025 23:23, melanie witt wrote: >>> > On 5/1/25 15:11, Sean Mooney wrote: >>> >> >>> >> On 01/05/2025 20:22, melanie witt wrote: >>> >>> On 5/1/25 08:17, Sean Mooney wrote: >>> >>>> >>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>> >>>>> Hello. >>> >>>>> I created an external network for a specified project but I >>> cannot >>> >>>>> create instances with the external network on this tenant. >>> >>>>> >>> >>>>> I must create RBAC Policies with shared and external > policies >>> then >>> >>>>> my instances can get IP addresses and run properly. >>> >>>>> >>> >>>> booting to a external network is admin only by default. >>> >>>> >>> >>>> depending on your release nova used to enforce this too but > in >>> >>>> newer release we defer to neutron to enforce that
>>> >>> >>> >>> I think I actually looked at this recently while re-triaging >>> old >>> >>> bugs downstream and AFAICT it's still an issue: >>> >>> >>> >>> https://bugs.launchpad.net/nova/+bug/1675486 >>> <https://bugs.launchpad.net/nova/+bug/1675486> >>> >>> >>> >>> We have mentioned the idea of deferring to Neutron for this >>> entirely >>> >>> (and I expect we can with no ill side effects) but I don't >>> believe >>> >>> we formally discussed with the Neutron team to confirm >>> whether it >>> >>> would be 100% OK to do. >>> >>> >>> >>> I think it would be nice to follow up on ^ and finally >>> remove the >>> >>> external network policy check in Nova. >>> >>> >>> >>> -melwitt >>> >> >>> >> we merged https://review.opendev.org/c/openstack/nova/+/794360 >>> <https://review.opendev.org/c/openstack/nova/+/794360> 3 >>> >> years ago >>> >> >>> >> but looking at the code change it just removed the commen so we >>> still >>> >> have our own >>> >> >>> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >>> >> >>> >> so i guess an operat can chagne that to project_member but > your >>> >> right we never actully removed the proejct. >>> >> >>> >> we probably should do that but ya we can see if neutron are >>> still ok >>> >> with this. >>> >> >>> >> im 99% sure that we talked about it with them in a past
>>> but its >>> >> definetly been a couple of years. >>> > >>> > Good to know, it is highly likely that I missed it or forgot >>> given the >>> > time that has passed since then. >>> > >>> > I'm supportive of going ahead and removing it now and give it a >>> lot of >>> > bake time this cycle to uncover potential issues. >>> > >>> >> one thing to keep in mind. external network do not have neutron >>> >> router as there defautl gateway >>> >> >>> >> by default neutron implements the metadata proxy via the l3 >>> agent if >>> >> your ussing ml2/ovs so you need >>> >> >>> >> to enable the option to do that via the dhcp agent if you are >>> still >>> >> usign ml2/ovs or your vms wont have >>> >> >>> >> metadata. for ml2/ovn i think it will just work but
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... policy ptg perhaps
> not. >>> > >>> > I didn't understand any of that unfortunately :) but if it's a >>> > potential problem with deferring to Neutron it's good to look > out >>> for it. >>> > >>> > -melwitt >>> >>> sorry i didnt explain it well. >>> >>> the way the nova metadata api is made aviable to vm is via a > neutron >>> proxy, that proxy when not using ovn is provide >>> >>> by a iptables port forwading rule combinde with a proxy service > that >>> translates the users request without any auth into a authitcated >>> request >>> >>> to novas metadata api. >>> >>> the metadtat proxy and iptables rule can be deploy be either the >>> neutron >>> l3 agent in the router network namespace or >>> >>> by the neutron dhcp agent in in the dhcp servers namespace. >>> >>> the default when not using ovn is for it to be in the router >>> namespace. >>> >>> because the derault route when you place a port directly on an >>> external >>> network is your datacenter router >>> >>> not a neutron one a request form teh vm to 169.whatever wil not be >>> proxied to nova so metadta in the vm wont work out of the box >>> >>> unless you do metadata via the dhcp server instead. >>> >>> >>> if your using ovn i think this just work because ovn has a metadta >>> proxy >>> on every compute node and it isntalls an openflow rule >>> >>> that matches on the destination ip to redirect it to that local >>> proxy. >>> so for ovn i belive it will just work. >>> >>> on the flip side port on an external network wont have there > traffic >>> nat'd when going to the internet >>> >>> external network in neutron are expected to have subnets that are >>> routeable to the WAN so operator need to confiugre there data > center >>> routeres >>> >>> to perform nat if that is not the case and they will also need to >>> provide the eant with an ip pool or precreated subnet+network. >>> >>> while not un heard of providign tenant with there own external vlan >>> network like this is not very clody in the normal self service >>> workflow but >>> it can have some advantages by allowing the routing to be >>> delgated to a >>> hardware router in your data center. >>> >>> that can allow your tenats to bring there own ip block to to your >>> datacenter but it need a lot of hand holding from the operator to >>> get setup >>> >>> initally. >>> >>> > >>> >> >>> >>> >>> >>>>> Is it normal? Pls correct me if I am wrong >>> >>>> >>> >>>> yes its normal, neutron may have change the defautl to >>> allow non >>> >>>> admins but booting direclty to an extenal network was >>> >>>> >>> >>>> always considerd privaldged in the past. >>> >>>> >>> >>>>> >>> >>>>> Thank you. >>> >>>>> >>> >>>>> Nguyen Huu Khoi >>> >>>> >>> >>> >>> >> >>> > >>> > -- > Slawek Kaplonski > Principal Software Engineer > Red Hat > >
-- Slawek Kaplonski Principal Software Engineer Red Hat
On 15/05/2025 00:28, Nguyễn Hữu Khôi wrote:
Hi,
It is very detailed. Thank you very much @Sean Mooney <mailto:smooney@redhat.com> .
From an operator perspective, It will be useful if we can allow this.
as an operator you can already do this via customer policy the real question is it safe fro nova to do that by default. there are some security and operational consideration in general so while it may be approcate for nova to allow it by default# neutron should likely not defautl to allowing attachemt directly to external networks without reviewign the upgrade impact with there users. i suspect this is likely not what public clouds would want out of the box but i may be wrong. i think what we should do is propsoa a patch to modify the default policy in nova and we can then ask the neutron team to review that when its available.
Nguyen Huu Khoi
On Wed, May 14, 2025 at 12:38 AM Sean Mooney <smooney@redhat.com> wrote:
On 13/05/2025 15:56, Sławek Kapłoński wrote: > Hi, > > Dnia wtorek, 13 maja 2025 13:41:47 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze: >> Hello. >> As I mentioned earlier, We have multi tenants and each has their own vlan >> networks. When I create vlan networks for them then I mask it as an >> external network(with this tenant not * in RABC Policy) and they cannot use >> this network for launching instances.
just to be clear you are creating the vlans provider network for them
because you cannot use neutron support for vlan tenant network to have them create the network
themselves either because of business constraints or because in your deployment you need to
assign specific vlan id to the tenant and your using data center routers to provide them with external connectivtiy correct.
precreating an external network for a tenant is not uncommon but normally the tenant will create an additional network
and a nutron router and then expose there vms via floating ips if you have not configured something liek netowrkign-bgp
to make there network routable.
the current issues you are having are beign caused because you are trying to allow unprivladge end user to add ports
to an external network which by default required admin permissions.
you can adjust that in your nova and neutron policy if desired.
>> If I want to use this network I must >> create a RABC policy shared with this project. > Yes, that is how it works. You can also share network with all tenants (use "*" as target). If you think about any other new functionality, feel free to propose new RFE [1] which we can discuss later during the drivers meeting. > > [1] https://docs.openstack.org/neutron/latest/contributor/policies/blueprints.ht...
that wont be enough because nova by default only allow admins to use external networks.
https://github.com/openstack/nova/blob/023be4f5615255d06f61b393605da1308f045...
the question we (nova) had to neturon was is it ok if we relax that to
base.PROJECT_MEMBER_OR_ADMIN
and then rely on neutron RBAC to determin if
1 the network should be visible and 2 if the attachment to it is allowed.
baiscly is it ok for nova to delegate that policy enforcement to neutron.
note we dont curretnly have anyone working on that although it was something that was stared a few release ago and
never got completed. so if neutron is ok with the change and folk want to work on it we can review.
>> Nguyen Huu Khoi >> >> >> On Tue, May 13, 2025 at 2:30 PM Sławek Kapłoński <skaplons@redhat.com> >> wrote: >> >>> Hi, >>> >>> Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni >>> Nguyễn Hữu Khôi pisze: >>>> Hello. >>>> >>>> Could we approve this feature in future releases? >>> What feature you are exactly asking for? >>> >>>> Nguyen Huu Khoi >>>> >>>> >>>> On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi < >>> nguyenhuukhoinw@gmail.com> >>>> wrote: >>>> >>>>> Hello. Because other tenants need external network to use with routers, >>>>> float ips as well as magnum. I think it will be need for private cloud. >>>>> >>>>> On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> >>> wrote: >>>>>> Hi, >>>>>> >>>>>> W dniu 2.05.2025 o 20:23, Brian Haley pisze: >>>>>>> Hi, >>>>>>> >>>>>>> On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote: >>>>>>>> Could we find a solution for this case? Because they own vlan >>>>>>>> networks Currently, I can create RBAC Policies to allow tenants >>> using >>>>>>>> external networks. >>>>>>> I've read this thread a couple of times and I still don't exactly >>>>>>> understand what the ask is from Neutron. This tenant "owns" a VLAN >>>>>>> network and wants to integrate it and a range of IP addresses into >>>>>>> Neutron as an external network? >>>>>> I also have some problem to understand what the issue is really. First >>>>>> of all I just checked in our code, and policy for port creation don't >>>>>> tell anything about external network. It is allowed for the >>>>>> ADMIN_OR_PROJECT_MEMBER or SERVICE - see >>>>>> >>>>>> >>> https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128... >>>>>> - so if tenant owns this vlan network, it should be doable to create >>>>>> port in such network IMO (but I don't have env up now to check it) >>>>>> >>>>>> Another thing which I am not sure is why this vlan network needs to be >>>>>> marked as external - are You using it also as external gateway for >>>>>> routers and have floating ips from it? If not (and that's what I >>>>>> understood from the thread so far) you don't need to set >>>>>> `router:external` to `True`. >>>>>> >>>>>> >>>>>>> Also, I don't recall any previous discussions that took place, maybe >>>>>>> someone else on the Neutron team does though and can comment next >>> week. >>>>>>> Thanks, >>>>>>> >>>>>>> -Brian >>>>>>> >>>>>>>> On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com >>>>>>>> <mailto:smooney@redhat.com>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 01/05/2025 23:23, melanie witt wrote: >>>>>>>> > On 5/1/25 15:11, Sean Mooney wrote: >>>>>>>> >> >>>>>>>> >> On 01/05/2025 20:22, melanie witt wrote: >>>>>>>> >>> On 5/1/25 08:17, Sean Mooney wrote: >>>>>>>> >>>> >>>>>>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>>>>> >>>>> Hello. >>>>>>>> >>>>> I created an external network for a specified project >>> but I >>>>>>>> cannot >>>>>>>> >>>>> create instances with the external network on this >>> tenant. >>>>>>>> >>>>> >>>>>>>> >>>>> I must create RBAC Policies with shared and external >>>>>> policies >>>>>>>> then >>>>>>>> >>>>> my instances can get IP addresses and run properly. >>>>>>>> >>>>> >>>>>>>> >>>> booting to a external network is admin only by default. >>>>>>>> >>>> >>>>>>>> >>>> depending on your release nova used to enforce this too >>> but >>>>>> in >>>>>>>> >>>> newer release we defer to neutron to enforce that policy >>>>>>>> >>> >>>>>>>> >>> I think I actually looked at this recently while >>> re-triaging >>>>>>>> old >>>>>>>> >>> bugs downstream and AFAICT it's still an issue: >>>>>>>> >>> >>>>>>>> >>> https://bugs.launchpad.net/nova/+bug/1675486 >>>>>>>> <https://bugs.launchpad.net/nova/+bug/1675486> >>>>>>>> >>> >>>>>>>> >>> We have mentioned the idea of deferring to Neutron for >>> this >>>>>>>> entirely >>>>>>>> >>> (and I expect we can with no ill side effects) but I don't >>>>>>>> believe >>>>>>>> >>> we formally discussed with the Neutron team to confirm >>>>>>>> whether it >>>>>>>> >>> would be 100% OK to do. >>>>>>>> >>> >>>>>>>> >>> I think it would be nice to follow up on ^ and finally >>>>>>>> remove the >>>>>>>> >>> external network policy check in Nova. >>>>>>>> >>> >>>>>>>> >>> -melwitt >>>>>>>> >> >>>>>>>> >> we merged >>> https://review.opendev.org/c/openstack/nova/+/794360 >>>>>>>> <https://review.opendev.org/c/openstack/nova/+/794360> 3 >>>>>>>> >> years ago >>>>>>>> >> >>>>>>>> >> but looking at the code change it just removed the commen >>> so we >>>>>>>> still >>>>>>>> >> have our own >>>>>>>> >> >>>>>>>> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >>>>>>>> >> >>>>>>>> >> so i guess an operat can chagne that to project_member but >>>>>> your >>>>>>>> >> right we never actully removed the proejct. >>>>>>>> >> >>>>>>>> >> we probably should do that but ya we can see if neutron are >>>>>>>> still ok >>>>>>>> >> with this. >>>>>>>> >> >>>>>>>> >> im 99% sure that we talked about it with them in a past ptg >>>>>>>> but its >>>>>>>> >> definetly been a couple of years. >>>>>>>> > >>>>>>>> > Good to know, it is highly likely that I missed it or forgot >>>>>>>> given the >>>>>>>> > time that has passed since then. >>>>>>>> > >>>>>>>> > I'm supportive of going ahead and removing it now and give >>> it a >>>>>>>> lot of >>>>>>>> > bake time this cycle to uncover potential issues. >>>>>>>> > >>>>>>>> >> one thing to keep in mind. external network do not have >>> neutron >>>>>>>> >> router as there defautl gateway >>>>>>>> >> >>>>>>>> >> by default neutron implements the metadata proxy via the l3 >>>>>>>> agent if >>>>>>>> >> your ussing ml2/ovs so you need >>>>>>>> >> >>>>>>>> >> to enable the option to do that via the dhcp agent if you >>> are >>>>>>>> still >>>>>>>> >> usign ml2/ovs or your vms wont have >>>>>>>> >> >>>>>>>> >> metadata. for ml2/ovn i think it will just work but perhaps >>>>>> not. >>>>>>>> > >>>>>>>> > I didn't understand any of that unfortunately :) but if >>> it's a >>>>>>>> > potential problem with deferring to Neutron it's good to >>> look >>>>>> out >>>>>>>> for it. >>>>>>>> > >>>>>>>> > -melwitt >>>>>>>> >>>>>>>> sorry i didnt explain it well. >>>>>>>> >>>>>>>> the way the nova metadata api is made aviable to vm is via a >>>>>> neutron >>>>>>>> proxy, that proxy when not using ovn is provide >>>>>>>> >>>>>>>> by a iptables port forwading rule combinde with a proxy service >>>>>> that >>>>>>>> translates the users request without any auth into a >>> authitcated >>>>>>>> request >>>>>>>> >>>>>>>> to novas metadata api. >>>>>>>> >>>>>>>> the metadtat proxy and iptables rule can be deploy be either >>> the >>>>>>>> neutron >>>>>>>> l3 agent in the router network namespace or >>>>>>>> >>>>>>>> by the neutron dhcp agent in in the dhcp servers namespace. >>>>>>>> >>>>>>>> the default when not using ovn is for it to be in the router >>>>>>>> namespace. >>>>>>>> >>>>>>>> because the derault route when you place a port directly on an >>>>>>>> external >>>>>>>> network is your datacenter router >>>>>>>> >>>>>>>> not a neutron one a request form teh vm to 169.whatever wil >>> not be >>>>>>>> proxied to nova so metadta in the vm wont work out of the box >>>>>>>> >>>>>>>> unless you do metadata via the dhcp server instead. >>>>>>>> >>>>>>>> >>>>>>>> if your using ovn i think this just work because ovn has a >>> metadta >>>>>>>> proxy >>>>>>>> on every compute node and it isntalls an openflow rule >>>>>>>> >>>>>>>> that matches on the destination ip to redirect it to that local >>>>>>>> proxy. >>>>>>>> so for ovn i belive it will just work. >>>>>>>> >>>>>>>> on the flip side port on an external network wont have there >>>>>> traffic >>>>>>>> nat'd when going to the internet >>>>>>>> >>>>>>>> external network in neutron are expected to have subnets that >>> are >>>>>>>> routeable to the WAN so operator need to confiugre there data >>>>>> center >>>>>>>> routeres >>>>>>>> >>>>>>>> to perform nat if that is not the case and they will also need >>> to >>>>>>>> provide the eant with an ip pool or precreated subnet+network. >>>>>>>> >>>>>>>> while not un heard of providign tenant with there own external >>> vlan >>>>>>>> network like this is not very clody in the normal self service >>>>>>>> workflow but >>>>>>>> it can have some advantages by allowing the routing to be >>>>>>>> delgated to a >>>>>>>> hardware router in your data center. >>>>>>>> >>>>>>>> that can allow your tenats to bring there own ip block to to >>> your >>>>>>>> datacenter but it need a lot of hand holding from the operator >>> to >>>>>>>> get setup >>>>>>>> >>>>>>>> initally. >>>>>>>> >>>>>>>> > >>>>>>>> >> >>>>>>>> >>> >>>>>>>> >>>>> Is it normal? Pls correct me if I am wrong >>>>>>>> >>>> >>>>>>>> >>>> yes its normal, neutron may have change the defautl to >>>>>>>> allow non >>>>>>>> >>>> admins but booting direclty to an extenal network was >>>>>>>> >>>> >>>>>>>> >>>> always considerd privaldged in the past. >>>>>>>> >>>> >>>>>>>> >>>>> >>>>>>>> >>>>> Thank you. >>>>>>>> >>>>> >>>>>>>> >>>>> Nguyen Huu Khoi >>>>>>>> >>>> >>>>>>>> >>> >>>>>>>> >> >>>>>>>> > >>>>>>>> >>>>>> -- >>>>>> Slawek Kaplonski >>>>>> Principal Software Engineer >>>>>> Red Hat >>>>>> >>>>>> >>> >>> -- >>> Slawek Kaplonski >>> Principal Software Engineer >>> Red Hat >
Hello. Yes, i can do it via policies but if there are hundred networks so we need x2 policies. I will try with nova policies. Would you like give us some ideas on it, neutron team? On Thu, May 15, 2025, 8:31 PM Sean Mooney <smooney@redhat.com> wrote:
On 15/05/2025 00:28, Nguyễn Hữu Khôi wrote:
Hi,
It is very detailed. Thank you very much @Sean Mooney <mailto:smooney@redhat.com> .
From an operator perspective, It will be useful if we can allow this.
as an operator you can already do this via customer policy the real question is
it safe fro nova to do that by default.
there are some security and operational consideration in general so while it may be approcate for nova to allow it by default#
neutron should likely not defautl to allowing attachemt directly to external networks without reviewign the upgrade impact with there users.
i suspect this is likely not what public clouds would want out of the box but i may be wrong.
i think what we should do is propsoa a patch to modify the default policy in nova and we can then ask the neutron team to review that when its
available.
Nguyen Huu Khoi
On Wed, May 14, 2025 at 12:38 AM Sean Mooney <smooney@redhat.com> wrote:
On 13/05/2025 15:56, Sławek Kapłoński wrote: > Hi, > > Dnia wtorek, 13 maja 2025 13:41:47 czas środkowoeuropejski letni Nguyễn Hữu Khôi pisze: >> Hello. >> As I mentioned earlier, We have multi tenants and each has their own vlan >> networks. When I create vlan networks for them then I mask it as
an
>> external network(with this tenant not * in RABC Policy) and they cannot use >> this network for launching instances.
just to be clear you are creating the vlans provider network for them
because you cannot use neutron support for vlan tenant network to have them create the network
themselves either because of business constraints or because in your deployment you need to
assign specific vlan id to the tenant and your using data center routers to provide them with external connectivtiy correct.
precreating an external network for a tenant is not uncommon but normally the tenant will create an additional network
and a nutron router and then expose there vms via floating ips if you have not configured something liek netowrkign-bgp
to make there network routable.
the current issues you are having are beign caused because you are trying to allow unprivladge end user to add ports
to an external network which by default required admin permissions.
you can adjust that in your nova and neutron policy if desired.
>> If I want to use this network I must >> create a RABC policy shared with this project. > Yes, that is how it works. You can also share network with all tenants (use "*" as target). If you think about any other new functionality, feel free to propose new RFE [1] which we can discuss later during the drivers meeting. > > [1]
https://docs.openstack.org/neutron/latest/contributor/policies/blueprints.ht...
that wont be enough because nova by default only allow admins to use external networks.
https://github.com/openstack/nova/blob/023be4f5615255d06f61b393605da1308f045...
the question we (nova) had to neturon was is it ok if we relax that
to
base.PROJECT_MEMBER_OR_ADMIN
and then rely on neutron RBAC to determin if
1 the network should be visible and 2 if the attachment to it is allowed.
baiscly is it ok for nova to delegate that policy enforcement to neutron.
note we dont curretnly have anyone working on that although it was something that was stared a few release ago and
never got completed. so if neutron is ok with the change and folk want to work on it we can review.
>> Nguyen Huu Khoi >> >> >> On Tue, May 13, 2025 at 2:30 PM Sławek Kapłoński <skaplons@redhat.com> >> wrote: >> >>> Hi, >>> >>> Dnia poniedziałek, 12 maja 2025 18:19:05 czas środkowoeuropejski letni >>> Nguyễn Hữu Khôi pisze: >>>> Hello. >>>> >>>> Could we approve this feature in future releases? >>> What feature you are exactly asking for? >>> >>>> Nguyen Huu Khoi >>>> >>>> >>>> On Sat, May 3, 2025 at 7:21 PM Nguyễn Hữu Khôi < >>> nguyenhuukhoinw@gmail.com> >>>> wrote: >>>> >>>>> Hello. Because other tenants need external network to use with routers, >>>>> float ips as well as magnum. I think it will be need for private cloud. >>>>> >>>>> On Sat, May 3, 2025, 2:31 AM Slawek Kaplonski <skaplons@redhat.com> >>> wrote: >>>>>> Hi, >>>>>> >>>>>> W dniu 2.05.2025 o 20:23, Brian Haley pisze: >>>>>>> Hi, >>>>>>> >>>>>>> On 5/2/25 8:36 AM, Nguyễn Hữu Khôi wrote: >>>>>>>> Could we find a solution for this case? Because they own
vlan
>>>>>>>> networks Currently, I can create RBAC Policies to allow tenants >>> using >>>>>>>> external networks. >>>>>>> I've read this thread a couple of times and I still don't exactly >>>>>>> understand what the ask is from Neutron. This tenant "owns" a VLAN >>>>>>> network and wants to integrate it and a range of IP addresses into >>>>>>> Neutron as an external network? >>>>>> I also have some problem to understand what the issue is really. First >>>>>> of all I just checked in our code, and policy for port creation don't >>>>>> tell anything about external network. It is allowed for the >>>>>> ADMIN_OR_PROJECT_MEMBER or SERVICE - see >>>>>> >>>>>> >>>
https://github.com/openstack/neutron/blob/7c0aac975a67b5f49bef7b2fb6f643d128...
>>>>>> - so if tenant owns this vlan network, it should be doable to create >>>>>> port in such network IMO (but I don't have env up now to check it) >>>>>> >>>>>> Another thing which I am not sure is why this vlan network needs to be >>>>>> marked as external - are You using it also as external gateway for >>>>>> routers and have floating ips from it? If not (and that's what I >>>>>> understood from the thread so far) you don't need to set >>>>>> `router:external` to `True`. >>>>>> >>>>>> >>>>>>> Also, I don't recall any previous discussions that took place, maybe >>>>>>> someone else on the Neutron team does though and can comment next >>> week. >>>>>>> Thanks, >>>>>>> >>>>>>> -Brian >>>>>>> >>>>>>>> On Fri, May 2, 2025 at 5:39 AM Sean Mooney <smooney@redhat.com >>>>>>>> <mailto:smooney@redhat.com>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 01/05/2025 23:23, melanie witt wrote: >>>>>>>> > On 5/1/25 15:11, Sean Mooney wrote: >>>>>>>> >> >>>>>>>> >> On 01/05/2025 20:22, melanie witt wrote: >>>>>>>> >>> On 5/1/25 08:17, Sean Mooney wrote: >>>>>>>> >>>> >>>>>>>> >>>> On 01/05/2025 15:56, Nguyễn Hữu Khôi wrote: >>>>>>>> >>>>> Hello. >>>>>>>> >>>>> I created an external network for a specified project >>> but I >>>>>>>> cannot >>>>>>>> >>>>> create instances with the external network on this >>> tenant. >>>>>>>> >>>>> >>>>>>>> >>>>> I must create RBAC Policies with shared and external >>>>>> policies >>>>>>>> then >>>>>>>> >>>>> my instances can get IP addresses and run properly. >>>>>>>> >>>>> >>>>>>>> >>>> booting to a external network is admin only by default. >>>>>>>> >>>> >>>>>>>> >>>> depending on your release nova used to enforce this too >>> but >>>>>> in >>>>>>>> >>>> newer release we defer to neutron to enforce that policy >>>>>>>> >>> >>>>>>>> >>> I think I actually looked at this recently while >>> re-triaging >>>>>>>> old >>>>>>>> >>> bugs downstream and AFAICT it's still an issue: >>>>>>>> >>> >>>>>>>> >>> https://bugs.launchpad.net/nova/+bug/1675486 >>>>>>>> <https://bugs.launchpad.net/nova/+bug/1675486> >>>>>>>> >>> >>>>>>>> >>> We have mentioned the idea of deferring to Neutron for >>> this >>>>>>>> entirely >>>>>>>> >>> (and I expect we can with no ill side effects) but I don't >>>>>>>> believe >>>>>>>> >>> we formally discussed with the Neutron team to confirm >>>>>>>> whether it >>>>>>>> >>> would be 100% OK to do. >>>>>>>> >>> >>>>>>>> >>> I think it would be nice to follow up on ^ and finally >>>>>>>> remove the >>>>>>>> >>> external network policy check in Nova. >>>>>>>> >>> >>>>>>>> >>> -melwitt >>>>>>>> >> >>>>>>>> >> we merged >>> https://review.opendev.org/c/openstack/nova/+/794360 >>>>>>>> <https://review.opendev.org/c/openstack/nova/+/794360>
3
>>>>>>>> >> years ago >>>>>>>> >> >>>>>>>> >> but looking at the code change it just removed the commen >>> so we >>>>>>>> still >>>>>>>> >> have our own >>>>>>>> >> >>>>>>>> >> NETWORK_ATTACH_EXTERNAL polciy which is admin only. >>>>>>>> >> >>>>>>>> >> so i guess an operat can chagne that to project_member but >>>>>> your >>>>>>>> >> right we never actully removed the proejct. >>>>>>>> >> >>>>>>>> >> we probably should do that but ya we can see if neutron are >>>>>>>> still ok >>>>>>>> >> with this. >>>>>>>> >> >>>>>>>> >> im 99% sure that we talked about it with them in a past ptg >>>>>>>> but its >>>>>>>> >> definetly been a couple of years. >>>>>>>> > >>>>>>>> > Good to know, it is highly likely that I missed it or forgot >>>>>>>> given the >>>>>>>> > time that has passed since then. >>>>>>>> > >>>>>>>> > I'm supportive of going ahead and removing it now and give >>> it a >>>>>>>> lot of >>>>>>>> > bake time this cycle to uncover potential issues. >>>>>>>> > >>>>>>>> >> one thing to keep in mind. external network do not have >>> neutron >>>>>>>> >> router as there defautl gateway >>>>>>>> >> >>>>>>>> >> by default neutron implements the metadata proxy via the l3 >>>>>>>> agent if >>>>>>>> >> your ussing ml2/ovs so you need >>>>>>>> >> >>>>>>>> >> to enable the option to do that via the dhcp agent if you >>> are >>>>>>>> still >>>>>>>> >> usign ml2/ovs or your vms wont have >>>>>>>> >> >>>>>>>> >> metadata. for ml2/ovn i think it will just work but perhaps >>>>>> not. >>>>>>>> > >>>>>>>> > I didn't understand any of that unfortunately :) but if >>> it's a >>>>>>>> > potential problem with deferring to Neutron it's good to >>> look >>>>>> out >>>>>>>> for it. >>>>>>>> > >>>>>>>> > -melwitt >>>>>>>> >>>>>>>> sorry i didnt explain it well. >>>>>>>> >>>>>>>> the way the nova metadata api is made aviable to vm is via a >>>>>> neutron >>>>>>>> proxy, that proxy when not using ovn is provide >>>>>>>> >>>>>>>> by a iptables port forwading rule combinde with a proxy service >>>>>> that >>>>>>>> translates the users request without any auth into a >>> authitcated >>>>>>>> request >>>>>>>> >>>>>>>> to novas metadata api. >>>>>>>> >>>>>>>> the metadtat proxy and iptables rule can be deploy be either >>> the >>>>>>>> neutron >>>>>>>> l3 agent in the router network namespace or >>>>>>>> >>>>>>>> by the neutron dhcp agent in in the dhcp servers namespace. >>>>>>>> >>>>>>>> the default when not using ovn is for it to be in the router >>>>>>>> namespace. >>>>>>>> >>>>>>>> because the derault route when you place a port directly on an >>>>>>>> external >>>>>>>> network is your datacenter router >>>>>>>> >>>>>>>> not a neutron one a request form teh vm to 169.whatever wil >>> not be >>>>>>>> proxied to nova so metadta in the vm wont work out of the box >>>>>>>> >>>>>>>> unless you do metadata via the dhcp server instead. >>>>>>>> >>>>>>>> >>>>>>>> if your using ovn i think this just work because ovn has a >>> metadta >>>>>>>> proxy >>>>>>>> on every compute node and it isntalls an openflow rule >>>>>>>> >>>>>>>> that matches on the destination ip to redirect it to that local >>>>>>>> proxy. >>>>>>>> so for ovn i belive it will just work. >>>>>>>> >>>>>>>> on the flip side port on an external network wont have there >>>>>> traffic >>>>>>>> nat'd when going to the internet >>>>>>>> >>>>>>>> external network in neutron are expected to have subnets that >>> are >>>>>>>> routeable to the WAN so operator need to confiugre there data >>>>>> center >>>>>>>> routeres >>>>>>>> >>>>>>>> to perform nat if that is not the case and they will also need >>> to >>>>>>>> provide the eant with an ip pool or precreated subnet+network. >>>>>>>> >>>>>>>> while not un heard of providign tenant with there own external >>> vlan >>>>>>>> network like this is not very clody in the normal self service >>>>>>>> workflow but >>>>>>>> it can have some advantages by allowing the routing to be >>>>>>>> delgated to a >>>>>>>> hardware router in your data center. >>>>>>>> >>>>>>>> that can allow your tenats to bring there own ip block to to >>> your >>>>>>>> datacenter but it need a lot of hand holding from the operator >>> to >>>>>>>> get setup >>>>>>>> >>>>>>>> initally. >>>>>>>> >>>>>>>> > >>>>>>>> >> >>>>>>>> >>> >>>>>>>> >>>>> Is it normal? Pls correct me if I am wrong >>>>>>>> >>>> >>>>>>>> >>>> yes its normal, neutron may have change the defautl to >>>>>>>> allow non >>>>>>>> >>>> admins but booting direclty to an extenal network was >>>>>>>> >>>> >>>>>>>> >>>> always considerd privaldged in the past. >>>>>>>> >>>> >>>>>>>> >>>>> >>>>>>>> >>>>> Thank you. >>>>>>>> >>>>> >>>>>>>> >>>>> Nguyen Huu Khoi >>>>>>>> >>>> >>>>>>>> >>> >>>>>>>> >> >>>>>>>> > >>>>>>>> >>>>>> -- >>>>>> Slawek Kaplonski >>>>>> Principal Software Engineer >>>>>> Red Hat >>>>>> >>>>>> >>> >>> -- >>> Slawek Kaplonski >>> Principal Software Engineer >>> Red Hat >
participants (6)
-
Brian Haley
-
melanie witt
-
Nguyễn Hữu Khôi
-
Sean Mooney
-
Slawek Kaplonski
-
Sławek Kapłoński