Open Stack

On 5/28/2017 6:58 PM, joehuang wrote:
> Hello,
> 
> There is one session in OpenStack Boston summit about Neutron 
> multi-site, and discuss how to make Neutron cells-aware[1].
> 
> We have done experiment how Nova cells v2 + Neutron + Tricircle can work 
> very well:

Thanks for testing this out Joe. It's great to get some feedback about 
people experimenting with cells v2, especially this close to master - we 
generally seem to have a 12-18 month latency on feedback for big things 
like this so it's hard to react quickly to something a year old. :)

> 
> Follow the guide[2], you will have one two cells environment: cell1 in 
> node1 and cell2 in node2, and Nova-api/scheduler/conductor/placement in 
> node1. To simplify the integration, the region for Nova-api is 
> registered as CentralRegion in KeyStone.

This sounds sort of like what Dan Smith is doing in devstack for 
upstream CI:

https://review.openstack.org/#/c/436094/

That's a 2-node job so we have to turn it into multi-cell with just 
those two nodes, and one has to be running the controller services.

> 
> At the same time, the tricircle devstack plugin will also install 
> RegionOne Neutron in node1 to work with cell1, RegionTwo Neutron in 
> node2, i,e, we'll have one local Neutron with one cell, the neutron 
> endpoint url in cell's compute node will be configured to local Neutron. 
> each local Neutron will be configured with Tricircle local Neutron plugin.
> 
> We just mentioned that Nova-API is registered as CentralRegion, the 
> tricircle devstack plugin will also start a central Neutron server and 
> register it in CentralRegion( same region as Nova-API), the central 
> Neutron server will be installed with Tricircle central Neutron plugin. 
> In Nova-api's configuration nova.conf, the neutron endpoint url is 
> configured to the central Neutron server's url.

If I'm understanding this correctly, nova-api will talk to Tricircle 
which is federating the networking API for Neutron across the two 
regions for "local" Neutron, right?

But what about nova-compute running within the cells? Are those talking 
*only* to the Neutron deployed local to the same cell that nova-compute 
is in? Or can each nova-compute talk to the central Neutron server?

I think most of us in Nova have been considering external services as 
global to Nova, so Neutron, Glance, Placement, Keystone and Cinder.

We don't support move operations, e.g. live migration, for server 
instances across cells yet, but when we do we'd need to know the Neutron 
topology to know which hosts we can send an instance to. This is sort of 
touched on in the Neutron routed networks spec for Nova, which involves 
using the Placement service for defining aggregate associations between 
compute hosts and pools of network resources:

https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html

> 
> In both central Neutron server and local Neutron, the nova endpoint url 
> configuration will be pointed to central Nova-api url, it's for 
> call-back message to Nova-API.

Yup, so either Neutron sends an event to the central nova-api endpoint 
which knows which cell an instance is in and can route the message 
appropriately. I'm thinking of the os-server-external-events API in this 
case.

> 
> After the installation, now you have one CentralRegion with one Nova API 
> and one Neutron Server, and regard to Nova multi-cells, each cell is 
> accompanied with one local Neutron. ( It's not necessary for 1:1 mapping 
> between cell and local Neutron, may multi-cells mapped to one local 
> Neutron).
> 
> If you create network/router without availability-zone specified, global 
> network/router is applied, all instances from any cell can be attached 
> to. If you create network/router with availability-zone specified, you 
> can get scoped network/router, i.e, the network/router can only be 
> presented in regarding cells.

OK, this part about network AZs is probably related to what I was asking 
about above. I think Nova is pretty minimal when it comes to dealing 
with AZ and Neutron, i.e. when Nova creates a port it creates the port 
in the same AZ that the instance is in. I'm not sure what happens if 
that AZ does not exist in Neutron. I do know that managing AZs between 
Nova and Cinder has been a constant source of bugs and frustration:

http://lists.openstack.org/pipermail/openstack-operators/2017-May/013622.html

> 
> Note:
> 1. Because Nova multi-cells is still under development, there will be 
> some issue in deployment and experience, some typical trouble shooting 
> has been provided in the document[2].
> 2. For the RegionOne and RegionTwo name which is registered by local 
> Neutron, you can change it to other better name if needed, for example 
> "cell1-subregion", "cell2-subregion", etc.
> 
> Feedback and contribution are welcome to make this mode works.
> 
> [1]http://lists.openstack.org/pipermail/openstack-dev/2017-May/116614.html
> [2]https://docs.openstack.org/developer/tricircle/installation-guide.html#work-with-nova-cell-v2-experiment
> 
> Best Regards
> Chaoyi Huang (joehuang)
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 

Thanks,

Matt