[Openstack] Long delays applying security group changes in nova Liberty
abogott at wikimedia.org
Tue Aug 9 19:50:31 UTC 2016
Since upgrading to liberty, we've noticed some very dramatic lags in the
application of security group updates. Experience shows that it takes
somewhere between 15 minutes and forever for changes to take effect.
For example, I just now added a source group rule to a project:
Ingress - TCP 1 - 65535 - default
(In other words, allow access from anything that's also in this project
and has the default security group.) We sat and watched the iptables
for 20-30 minutes and waited for the new rules to appear, but they
didn't. Finally I restarted nova-compute, at which point compute went
through all the 'Ensuring static filters' steps and the iptables rules
I removed the rule to see if the restart resolved something but, nope,
still waiting for the change to apply.
- inter-service communication is otherwise working fine; For example, I
can create new instances on this host and it talks to designate.
- The project in question has hundreds of instances. When I experiment
with smaller projects things are typically more responsive.
Is this a known issue, or something others have run into?
More information about the Openstack