[Openstack-operators] instances floating IPs not reachable while restarting nova-network

Matt Fischer matt at mattfischer.com
Thu Nov 26 17:36:31 UTC 2015


For reference, neutron has similar issues when restarting some neutron
services, for example the ovs-agent plugin, The delay in coming back up
scales on the number of routers you are hosting. For this reason we don't
let puppet restart the senstitive services and our "rabbit connections are
broken reboot everything" ansible script specifically also avoids these
too. This is supposed to be fixed in Liberty and is probably the biggest
thing I'm looking forward to. Perhaps nova-network is not as granular, but
neutron has about 87 different services (+/- 65) and ones like
neutron-server are okay to bounce.



On Thu, Nov 26, 2015 at 9:23 AM, Joe Topjian <joe at topjian.net> wrote:

> Yup, this is expected. It happens for both single-host and multi-host.
> With the former, we have an older environment where it takes around 10
> minutes for all network access to resume. That's with a few hundred
> tenants, a few hundred vlans, and a few hundred floating IPs all on one
> host, though.
>
> From your list of reasons for restarting, we only need to restart
> nova-network for config changes. If you're running into odd issues that you
> think nova-network might be causing, definitely feel free to describe some
> symptoms :)
>
> Joe
> On Nov 26, 2015 8:49 AM, "Gustavo Randich" <gustavo.randich at gmail.com>
> wrote:
>
>> Hi everybody, (still using nova-network in production... :)
>>
>> Using nova-network (icehouse), multi-host, FlatDHCPManager
>>
>> Is it expectable to experience an interruption of various seconds in
>> instances' floating IP reachability when nova-network is restarted and
>> repopulates iptables' NAT output/prerouting/float-snat tables?  (IP packets
>> are not delivered to VMs until iptables forwarding rules are setup)
>>
>> We don't restart nova-network often, but we have certain cases when we
>> need(ed) to:
>>
>>   * nova-network not reconnecting to RabbitMQ (latest oslo messaging
>> patch mitigates this)
>>   * configuration changes in nova.conf from time to time
>>   * sanitary periodic (weekly or monthly) restarts to prevent poorly
>> understood problems of the past (resource leaks?); will stop doing this due
>> to NAT downtime
>>
>> Thanks!
>>
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20151126/1d8575e5/attachment.html>


More information about the OpenStack-operators mailing list