Open Stack

Sha,

As you noticed the vif_plug_notification  does not work with cells under the stock confiuguration.  In icehouse/juno we simply ran with vif_plugging_is_fatal is false and set the timeout value to 5 or 10 seconds iirc.

Sam Morrison and made a patch and Mathieu Gagné help updated it, to fix this.  You can find it here: https://review.openstack.org/#/c/215459/

We are running this patch under kilo without any issues in production.

From experience there are a number of other things that are broken in a cells setup:
1.) Flavor creation
2.) Availability zones
3.) Host aggregates creation for the API Cell
4.) Instance-name (doesn't match between api-cell/child-cell/HV)
5.) vif_plug_notification
6.) metadata with x509-keys (x509 key is updated in api cell and not pushed to child cell, but metadata tries to reference it from the child cell)

We (LDT) are in the process of documenting the patches that we use with Cells and trying to get those upstreamed, we are hoping to have a complete list by Tokyo:
https://etherpad.openstack.org/p/PAO-LDT-cells-patches

NeCTAR maintains a large number of patches that fix the above broken ness in cells.  You can find most of their patches by looking at the commit history on their kilo branch:
https://github.com/NeCTAR-RC/nova/commits/nectar/kilo?page=2 (mainly August 12/13/17/24) - We are running ~10 of those patches in production without any issues.
___________________________________________________________________
Kris Lindgren
Senior Linux Systems Engineer
GoDaddy

From: Sha Li
Date: Thursday, September 17, 2015 at 1:38 AM
To: "OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>"
Cc: "sam.morrison at unimelb.edu.au<mailto:sam.morrison at unimelb.edu.au>", "belmiro.moreira at cern.ch<mailto:belmiro.moreira at cern.ch>"
Subject: [Openstack-operators] deploy nova cells with neutron network


Hi,

I am try to test the nova cells function.
My test deployment consits of one api-cell node, one child-cell node and one compute node.

api-cell node:  nova-api, nova-cells, nova-cert, nova-condoleauth, nova-novncproxy
child-cell node: nova-cells, nova-conductor, nova-scheduler
compute node: nova-compute

I found most deployment example is using nova-network with nova-cells. I want to use neutron. So I had keystone , glance, and neutron-server, neutron-dhcp, neutron-l3  shared between all cells and deployed all on the api-cell node.

I encounterd similar problem as described in this bug report
https://bugs.launchpad.net/nova/+bug/1348103

When boot a new instance, nova-compute fails to get the network-vif-plugged  notification and get time out waiting for the call back.
But on the neutron server side, it looks like the notification had been successfully sent and get the 200 response code from nova-api server

I had to set
vif_plugging_is_fatal = False
Then the instnace can be spawned normally

I am wondering how people use neutron with nova-cells, is this going to cause any trouble in large scale production deployment.


Cheers,
Sha



--- neutron server log file

2015-08-22 00:20:35.464 16812 DEBUG neutron.notifiers.nova [-] Sending events: [{'status': 'completed', 'tag': u'2839ca4d-b632-4d64-a174-ecfe34a7a746', 'name': 'network-vif-plugged', 'server_uuid': u'092c8bc4-3643-44c0-b79e-ad5caac18b3d'}] send_events /usr/lib/python2.7/site-packages/neutron/notifiers/nova.py:232

2015-08-22 00:20:35.468 16812 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.81.221

2015-08-22 00:20:35.548 16812 DEBUG urllib3.connectionpool [-] "POST /v2/338aad513c604880a6a0dcc58b88b905/os-server-external-events HTTP/1.1" 200 183 _make_request /usr/lib/python2.7/site-packages/urllib3/connectionpool.py:357

2015-08-22 00:20:35.550 16812 INFO neutron.notifiers.nova [-] Nova event response: {u'status': u'completed', u'tag': u'2839ca4d-b632-4d64-a174-ecfe34a7a746', u'name': u'network-vif-plugged', u'server_uuid': u'092c8bc4-3643-44c0-b79e-ad5caac18b3d', u'code': 200}




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150917/a1335e96/attachment.html>