[Openstack-operators] Neutron crashed hard

Jay Pipes jaypipes at gmail.com
Thu Dec 19 05:48:51 UTC 2013


On 12/19/2013 12:36 AM, Joe Topjian wrote:
> Hi Jay,
>
>     What version of OVS did you deploy? There's a bad bug/behavior in
>     OVS 1.04 that can result in circular routes in the GRE mesh, which
>     we saw entirely take down an entire deployment zone with tenant
>     traffic swamping the bonded NIC that was housing the GRE overlay
>     network. Upgrading to OVS 1.10 and then 1.11 solved that issue along
>     with some scripting...
>
>
> OVS 1.10 is available in the Ubuntu havana repo. I'm using the Raring
> 3.8 kernel available in the standard 12.04 repo. The standard 12.04 repo
> also includes OVS 1.9 kernel module to compile with the 3.8 kernel.
>
> So, long story short: OVS 1.10 with 1.9 kernel module.

OK, then I think you are clear of the circular routing issue we saw.

>     Are you using a multi-plexed neutron server (workers config option > 1)?
>
>
> I haven't explicitly set this option as I didn't know it existed. Do you
> have a reference for this option? I did a quick scan/grep of the neutron
> config files and didn't see a reference to workers.

Sure. So this patch added in the multiplexed Neutron server functionality:

https://review.openstack.org/#/c/37131/

It's not in Havana, but we backport that sucker into our Grizzly and 
Havana deployments:

https://gist.github.com/alanmeadows/7770570
https://review.openstack.org/#/c/63020/

I would highly suggest you do the same, set workers=10 or so and retry 
your concurrent launch scenario...

Best,
-jay



More information about the OpenStack-operators mailing list