Open Stack

On Wed, 2021-12-22 at 22:36 +0900, Adam Harwell wrote:
> +1 from Yahoo.
> We have a number of compute AZs in each of our clouds, and networks that
> are specifically crafted for those AZs (mostly due to security zones). Our
> users very commonly pick incompatible combinations. We have a user manual
> with a table that shows valid combinations in an attempt to educate them,
> but this is really something that would be better handled directly in the
> tooling somehow. Maybe this kind of setup is more common than people think,
> at least for large private cloud operators?
it might be but you have to keep in mind that an AZ in openstack is just a metadata tag on a nova host aggreate.

it is not a concept that really spans serivce and its not a top level resoruece in the api.

yes cinder and neuton can havce sudo AZ by  setting some data in there config but resouce exposed by
cinder, neutron and other service dont really have a real az affintiy.

the cinder backed can be labled as bing intend for use with a given AZ but a volume does not actully have an AZ assocatied with it.
it may have a volume type assocaiated with it and that volume type may only be supported by a backedn in a specific AZ
but if you made the same volume type avaibale in a backend in a different AZ both would be equally valide to create the voluem.

by default nova allows cross az attach 
https://docs.openstack.org/nova/latest/configuration/config.html#cinder.cross_az_attach
it will only be prevented if you explcitly set that to false. that is a nova feature not a cinder one.
from a cinder point of view it only cares about volumes volume types and backends.

the same is pretty much true of neutron it really does not know much about aviablity zones nor should it really as desigend today.

to really be able to model what is being discussed we woudl need to have a neutorn extention to tag a netork with a az so that you could query
the networks by az and map them to azs. without that there is not much you can do in horizon and you can get the correct behavior in nova.
for network az affintiy to work nova woudl have to be enhacned to take the network az request into accoutn when schdulign an instance if you dont
also spcirfy it manually on server create and we woudl also want to ensure we done allow instance to be create with networks form differnt AZ.

if there is a grop of peole that want to work on this i would suggest propsoing a pair of nova and neutron specs starting with the neturon spec
to add az affintiy. you would have to think carfully about if the affinity shoudl exsit at the network, segments or subents level and how it affect other
resouces like subnet pools,qos policies and floating ips. once you have the rudemnts of a desgin for how to model this in the neutron api then 
an nova spec should created desciribing how this will impact the port resoucxe request and scheduling. 

with the neutron sepec in place the horizon and OSC comunities could also start working on enbiling the ui and cli to filter network resouces by avaiablity zone
while the nova work was done to make the end to end usecase work for network avaiableity zone aware scheduling.


a lot of the ground work is alreay avaiable by the way for this in placment but neutron would have to be enhanced to consume it.
the same is true of cinder. for cinder case it would need to start modeling storage backedn as shareing resouce providers of disk_gb with the volume type model as a trait.
if they plance that shareing resouce provider in nova host aggrate for a give AZ then that would cleanly model the affintiy of that backend to a given az and nova
could use the volumes az as an input to schduling when one is not speficed to ensure it only considers hosts in that az.

to make this work transparnetly is a lot of work and to date no one has really come forward to do it.
> 
>     —Adam
> 
> PS: I think routed networks may fix part of our configuration woes, but
> certainly not all. I wonder if there is a sane way to link networks to AZs
> like this…
> 
> On Wed, Dec 22, 2021 at 1:08 Jonathan Mills <jonmills at gmail.com> wrote:
> 
> > Thank you, Hai Wu, for raising the visibility of this issue.  We at NASA
> > Goddard want to echo this sentiment wholeheartedly.  It’s a serious pain
> > point!
> > 
> > Our production OpenStack (Wallaby) cloud spans two AZs (different
> > datacenters) right now, and before we’re done we may have 3 or 4 AZs.  We
> > use non-overlapping VLAN-based networks in different datacenters.
> > Moreover, we have different kinds of server hardware in each datacenter
> > (often recycled HPC compute nodes).  We also have different Cinder storage
> > backends in each datacenter.
> > 
> > What we end up having is AZs in Nova, AZs in Neutron, AZs in Cinder, and
> > Image Flavors for Glance that are rather AZ dependent (in order to best fit
> > the node geometry).  We instantiate these objects with explicit AZs when
> > they support it.  In the case of Neutron networks, we can use scheduler
> > hints.
> > 
> > The real crux of the problem is the end-user education though.  Because
> > OpenStack clients (Horizon, OSC) are perfectly happy to let the end-user
> > try to build impossible combinations.  We strongly feel that if the
> > individual services aren’t going to communicate AZ data to each other via
> > RPC, that the clients at least should do some kind of filtering on AZs or
> > scheduler hints about AZs.  I'm not entirely sure how you'd solve this
> > problem on CLI easily without RPC communication, but Horizon could
> > (should?) dynamically filter the different menus as you select options:
> > e.g. the first selection screen ‘Details’ has you choose an AZ. Once the
> > user makes a selection there, the flavors, storage, and network options
> > should adjust accordingly.
> > 
> > 
> > Jonathan
> > 
> > 
> > > On Dec 18, 2021, at 9:44 PM, hai wu <haiwu.us at gmail.com> wrote:
> > > 
> > > Not using vxlan, so there's no network span across availability zones.
> > > It seems there's no built-in way to configure Horizon to do so. The
> > > problem with this is that sometimes if user picks the wrong network
> > > and availability zone combo, the VM will end up in the wrong state.
> > > This is why it would be ideal to only show available networks for the
> > > availability zone that the user already picked.
> > > 
> > > I am wondering how to modify the horizon source code to achieve this,
> > > but I am not familiar with Angular, and the workflow code is pretty
> > > complex for newbie ..
> > > 
> > > On Sat, Dec 18, 2021 at 6:31 PM Sean Mooney <smooney at redhat.com> wrote:
> > > > 
> > > > On Sat, 2021-12-18 at 13:45 -0600, hai wu wrote:
> > > > > In Horizon "Launch Instance" workflow, in its 4th step of selecting
> > > > > networks for the new instance, it would always show all the available
> > > > > networks from all availability zones, regardless of which
> > > > > "Availability Zone" got picked in 1st step of "Details".
> > > > > 
> > > > > I tried to update some DB field for availability zone hint for the
> > > > > relevant network, and that did not help.
> > > > > 
> > > > > Any way to ensure in Horizon "Launch Instance" GUI workflow, after a
> > > > > user picking one availability zone in step 1, it would only show the
> > > > > related networks in that availability zone as available networks in
> > > > > step 4?
> > > > 
> > > > networks do not have any affinity to Avaiablity Zones.
> > > > there is no mapping between neutron physnets and nova host aggreates
> > which are use to
> > > > model aviablityitz zones.
> > > > 
> > > > when using tunneled networks like vxlan it is assuems that all hosts in
> > a cloud acorss all avaiablity zones can access
> > > > any tenant vxlan network. the same is also ture of vlan or flat network
> > the only excption being that htey have physnet
> > > > assocaited with them. physnets may not be aviable on all host but there
> > is corralation between phsynets and aviaablity zones.
> > > > unless you manually algin them.
> > > > > 
> > > > 
> > > 
> > 
> > 
> > --
> Thanks,
>     --Adam