[openstack-dev] [qa] [heat] [neutron] - Status of Heat and Neutron tempest blueprints?

Steve Baker sbaker at redhat.com
Thu Mar 6 20:52:06 UTC 2014

On 07/03/14 01:53, Sean Dague wrote:
> We're at Freeze, so I want to pick up and understand where we currently
> stand with both Neutron and Heat actually getting tested fully in the gate.
> First Neutron -
> https://blueprints.launchpad.net/tempest/+spec/fix-gate-tempest-devstack-vm-quantum-full
> We know that this is *close* as the full job is running non voting
> everywhere, and typically passing. How close are we? Or should we be
> defering this until Juno (which would be unfortunate).
> Second Heat -
> https://blueprints.launchpad.net/tempest/+spec/tempest-heat-integration
> The Heat tests that are in a normal Tempest job are relatively trivial
> surface verification, and in no way actually make sure that Heat is
> operating at a real level. This fact is a contributing factor to why
> Heat was broken in i2.
> The first real verification for Heat is in the Heat slow job (which we
> created to give Heat a separate time budget, because doing work that
> requires real guests takes time).
> The heat slow job looks like it is finally passing much of the time -
> http://logstash.openstack.org/#eyJzZWFyY2giOiIobWVzc2FnZTpcIkZpbmlzaGVkOiBTVUNDRVNTXCIgT1IgbWVzc2FnZTpcIkZpbmlzaGVkOiBGQUlMVVJFXCIpIEFORCBidWlsZF9uYW1lOmNoZWNrLXRlbXBlc3QtZHN2bS1uZXV0cm9uLWhlYXQtc2xvdyIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDEwOTg3NDQ4OH0=
> It's seeing a 78% pass rate in check. Can anyone in the Heat team
> confirm that the Failures in this job are actually real failures on
> patches that should have been blocked?
> I'd like to get that turned on (and on all the projects) as soon as the
> Heat team is confident on it so that Heat actually participates in the
> tempest/devstack gate in a material way and we can prevent future issues
> where a keystone, nova, neutron or whatever change would break Heat in git.
I've raised https://bugs.launchpad.net/tempest/+bug/1288970 to track the
most common error ( NeutronResourcesTestJSON f ailed to reach
CREATE_COMPLETE status within the required time (300 s))

tl;dr sometimes booting alone is taking 244s, so I think this test would
be significantly more reliable if the timeout was raised.

I would propose raising the default orchestration build_timeout to 600s
for now, but it may need to go up to 1200s when the autoscaling scenario
is enabled again.


Once this change is in heat-slow should be reliable enough to make it
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140307/dfb32c64/attachment-0001.html>

More information about the OpenStack-dev mailing list