Open Stack

Hi,

Dnia sobota, 6 marca 2021 13:53:02 CET Sean Mooney pisze:
> On Sat, 2021-03-06 at 08:37 +0100, Slawek Kaplonski wrote:
> > Hi,
> > 
> > Dnia piątek, 5 marca 2021 17:26:19 CET melanie witt pisze:
> > > Hello all,
> > > 
> > > I'm seeking input from the neutron and nova teams regarding policy
> > > enforcement for allowing attachment to external networks. Details below.
> > > 
> > > Recently we've been looking at an issue that was reported quite a long
> > > time ago (2017) [1] where we have a policy check in nova-compute that
> > > controls whether to allow users to attach an external network to their
> > > instances.
> > > 
> > > This has historically been a pain point for operators as (1) it goes
> > > against convention of having policy checks in nova-api only and (2)
> > > setting the policy to anything other than the default requires deploying
> > > a policy file change to all of the compute hosts in the deployment.
> > > 
> > > The launchpad bug report mentions neutron refactoring work that was
> > > happening at the time, which was thought might make the
> > > 'network:attach_external_network' policy check on the nova side
> > > redundant.
> > > 
> > > Years have passed since then and customers are still running into this
> > > problem, so we are thinking, can this policy check be removed on the
> > > nova-compute side now?
> > > 
> > > I did a local test with devstack to verify what the behavior is if we
> > > were to remove the 'network:attach_external_network' policy check
> > > entirely [2] and found that neutron appears to properly enforce
> > > permission to attach to external networks itself. It appears that the
> > > enforcement on the neutron side makes the nova policy check redundant.
> > > 
> > > When I tried to boot an instance to attach to an external network,
> > > neutron API returned the following:
> > > 
> > > INFO neutron.pecan_wsgi.hooks.translation
> > > [req-58fdb103-cd20-48c9-b73b-c9074061998c
> > > req-4d68df7e-e0fd-4b1e-9b57-733731123d46 demo demo] POST failed (client
> > > error): Tenant 7c60976c662a414cb2661831ff41ee30 not allowed to create
> > > port on this network
> > > [...]
> > > INFO neutron.wsgi [req-58fdb103-cd20-48c9-b73b-c9074061998c
> > > req-4d68df7e-e0fd-4b1e-9b57-733731123d46 demo demo] 127.0.0.1 "POST
> > > /v2.0/ports HTTP/1.1" status: 403  len: 360 time: 0.1582518
> > 
> > I just checked in Neutron code and we don't have any policy rule related
> > directly to the creation of ports on the external network.
> > Probably what You had there is the fact that Your router:external network
> > was owned by other tenant and due to that You wasn't able to create port
> > directly on it. If as an admin You would create external network which
> > would belong to Your tenant, You would be allowed to create port there.
> > 
> > > Can anyone from the neutron team confirm whether it would be OK for us
> > > to remove our nova-compute policy check for external network attach
> > > permission and let neutron take care of the check?
> > 
> > I don't know exactly the reasons why it is forbiden on Nova's side but TBH
> > I don't see any reason why we should forbid pluging instances directly to
> > the network marked as router:external=True.
> 
> i have listed the majority of my consers in
> https://bugzilla.redhat.com/show_bug.cgi?id=1933047#c6 which is one of the
> downstream bug related to this.
> there are a number of issue sthat i was concerd about but tl;dr
> - booting ip form external network consumes ip from the floating ip subnet
> withtout using quota - by default neutron upstream and downstream is
> configured to provide nova metadata api access via the neutron router not
> the dhcp server so by default the metadata api will not work with external
> network. that would require neueton to be configre to use the dhcp server
> for metadta or config driver or else insance wont get ssh keys ingject by
> cloud init.
> - there might be security considertaions. typeically external networks are
> vlan or flat networks and in some cases operators may not want tenats to be
> able to boot on such networks expsially with vnic-type=driect-physical
> since that might allow them to violate tenant isolation if the top of rack
> switch was not configured by a heracical port binding driver to provide
> adiquite isolation in that case. this is not so much because this is an
> external network and more a concern anytime you do PF passtough but there
> may be other implication to allowing this by default. that said if neutron
> has a way to express policy in this regard nova does not have too.

Those are all valid points, true. But TBH, if administrator created such 
network as pool of FIPs for the user, then users will not be able to plug vms 
directly to that network as they aren't owners of the network so neutron will 
forbid that.

> 
> router:external=True is really used to mark a network as providing
> connectivity such that it can be used for the gateway port of neutron
> routers. the workaroud that i have come up with currently is to mark the
> network as shared and then use neturon rbac to only share it with the teant
> that owns it.
> 
> i assigning external network to speficic tenat being useful when you want to
> provde a specific ip allocation pool to them or just a set of ips. i
> understand that the current motivation for this request is commign form
> some edge deployments. in general i dont thinkthis would be widely used but
> for those that need its better ux then marking it as shared.
> 
> > > And on the nova side, I assume we would need a deprecation cycle before
> > > removing the 'network:attach_external_network' policy. If we can get
> > > confirmation from the neutron team, is anyone opposed to the idea of
> > > deprecating the 'network:attach_external_network' policy in the Wallaby
> > > cycle, to be removed in the Xena release?
> > > 
> > > I would appreciate your thoughts.
> > > 
> > > Cheers,
> > > -melanie
> > > 
> > > [1] https://bugs.launchpad.net/nova/+bug/1675486
> > > [2] https://bugs.launchpad.net/nova/+bug/1675486/comments/4


-- 
Slawek Kaplonski
Principal Software Engineer
Red Hat
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210306/a3fa650f/attachment.sig>