Open Stack

Hello, Matt,

See inline comments with [joehuang]

Best Regards
Chaoyi Huang (joehuang)

________________________________________
From: Matt Riedemann [mriedemos at gmail.com]
Sent: 07 June 2017 10:09
To: openstack-dev at lists.openstack.org; openstack-operators at lists.openstack.org
Subject: Re: [openstack-dev] [nova][neutron]Nova cells v2+Neutron+Tricircle, it works

On 5/28/2017 6:58 PM, joehuang wrote:
> Hello,
>
> There is one session in OpenStack Boston summit about Neutron
> multi-site, and discuss how to make Neutron cells-aware[1].
>
> We have done experiment how Nova cells v2 + Neutron + Tricircle can work
> very well:

Thanks for testing this out Joe. It's great to get some feedback about
people experimenting with cells v2, especially this close to master - we
generally seem to have a 12-18 month latency on feedback for big things
like this so it's hard to react quickly to something a year old. :)

>
> Follow the guide[2], you will have one two cells environment: cell1 in
> node1 and cell2 in node2, and Nova-api/scheduler/conductor/placement in
> node1. To simplify the integration, the region for Nova-api is
> registered as CentralRegion in KeyStone.

This sounds sort of like what Dan Smith is doing in devstack for
upstream CI:

https://review.openstack.org/#/c/436094/

That's a 2-node job so we have to turn it into multi-cell with just
those two nodes, and one has to be running the controller services.

>
> At the same time, the tricircle devstack plugin will also install
> RegionOne Neutron in node1 to work with cell1, RegionTwo Neutron in
> node2, i,e, we'll have one local Neutron with one cell, the neutron
> endpoint url in cell's compute node will be configured to local Neutron.
> each local Neutron will be configured with Tricircle local Neutron plugin.
>
> We just mentioned that Nova-API is registered as CentralRegion, the
> tricircle devstack plugin will also start a central Neutron server and
> register it in CentralRegion( same region as Nova-API), the central
> Neutron server will be installed with Tricircle central Neutron plugin.
> In Nova-api's configuration nova.conf, the neutron endpoint url is
> configured to the central Neutron server's url.

If I'm understanding this correctly, nova-api will talk to Tricircle
which is federating the networking API for Neutron across the two
regions for "local" Neutron, right?

[joehuang] nova-api talk to central Neutron server, but not Tricircle. 
The central neutron server is installed with Tricircle central plugin

But what about nova-compute running within the cells? Are those talking
*only* to the Neutron deployed local to the same cell that nova-compute
is in? Or can each nova-compute talk to the central Neutron server?

[joehuang] nova-compute talking *only* to the Neutron deployed local to the
same cell that nova-compute is in


I think most of us in Nova have been considering external services as
global to Nova, so Neutron, Glance, Placement, Keystone and Cinder.

We don't support move operations, e.g. live migration, for server
instances across cells yet, but when we do we'd need to know the Neutron
topology to know which hosts we can send an instance to. This is sort of
touched on in the Neutron routed networks spec for Nova, which involves
using the Placement service for defining aggregate associations between
compute hosts and pools of network resources:

https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html

[joehuang] routed network can address neutron scalability to some extent, but it just
leaves the tenant network isolation and management to the outside world. The complexity
is still there, but just outside Neutron/Nova, if we don't care about tenant's network
isolation, then it's ok. For tricircle support VxLAN, the overlay network can be extended to 
the host you want to migrate to. In routed network , You have to know the network topology, because
the network connectivity is defined outside Neutron, but not inside
Neutron, some host may be not connected to this network. 
So compared to routed network, Tricircle provides more flexible and tenant isolated networking capability
through Neutron.

>
> In both central Neutron server and local Neutron, the nova endpoint url
> configuration will be pointed to central Nova-api url, it's for
> call-back message to Nova-API.

Yup, so either Neutron sends an event to the central nova-api endpoint
which knows which cell an instance is in and can route the message
appropriately. I'm thinking of the os-server-external-events API in this
case.

>
> After the installation, now you have one CentralRegion with one Nova API
> and one Neutron Server, and regard to Nova multi-cells, each cell is
> accompanied with one local Neutron. ( It's not necessary for 1:1 mapping
> between cell and local Neutron, may multi-cells mapped to one local
> Neutron).
>
> If you create network/router without availability-zone specified, global
> network/router is applied, all instances from any cell can be attached
> to. If you create network/router with availability-zone specified, you
> can get scoped network/router, i.e, the network/router can only be
> presented in regarding cells.

OK, this part about network AZs is probably related to what I was asking
about above. I think Nova is pretty minimal when it comes to dealing
with AZ and Neutron, i.e. when Nova creates a port it creates the port
in the same AZ that the instance is in. I'm not sure what happens if
that AZ does not exist in Neutron. I do know that managing AZs between
Nova and Cinder has been a constant source of bugs and frustration:

http://lists.openstack.org/pipermail/openstack-operators/2017-May/013622.html

[joehuang] This is an issue without community wide sync of Avaialbity Zone: how it should
be used, designed and deployment. At lease Nova/Cinder/Neutron should have
a consensus understanding and design, this is a problem existing many years.

>
> Note:
> 1. Because Nova multi-cells is still under development, there will be
> some issue in deployment and experience, some typical trouble shooting
> has been provided in the document[2].
> 2. For the RegionOne and RegionTwo name which is registered by local
> Neutron, you can change it to other better name if needed, for example
> "cell1-subregion", "cell2-subregion", etc.
>
> Feedback and contribution are welcome to make this mode works.
>
> [1]http://lists.openstack.org/pipermail/openstack-dev/2017-May/116614.html
> [2]https://docs.openstack.org/developer/tricircle/installation-guide.html#work-with-nova-cell-v2-experiment
>
> Best Regards
> Chaoyi Huang (joehuang)
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


--

Thanks,

Matt

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev