[openstack-dev] [all] Gate still backed up - need assistance with nova-network logging enhancements
Michael Still
mikal at stillhq.com
Tue Jun 10 10:36:27 UTC 2014
https://review.openstack.org/99002 adds more logging to
nova/network/manager.py, but I think you're not going to love the
debug log level. Was this the sort of thing you were looking for
though?
Michael
On Mon, Jun 9, 2014 at 11:45 PM, Sean Dague <sean at dague.net> wrote:
> Based on some back of envelope math the gate is basically processing 2
> changes an hour, failing one of them. So if you want to know how long
> the gate is, take the length / 2 in hours.
>
> Right now we're doing a lot of revert roulette, trying to revert things
> that we think landed about the time things went bad. I call this
> roulette because in many cases the actual issue isn't well understood. A
> key reason for this is:
>
> *nova network is a blackhole*
>
> There is no work unit logging in nova-network, and no attempted
> verification that the commands it ran did a thing. Most of these
> failures that we don't have good understanding of are the network not
> working under nova-network.
>
> So we could *really* use a volunteer or two to prioritize getting that
> into nova-network. Without it we might manage to turn down the failure
> rate by reverting things (or we might not) but we won't really know why,
> and we'll likely be here again soon.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
--
Rackspace Australia
More information about the OpenStack-dev
mailing list