On 07/03/14 01:53, Sean Dague wrote: > We're at Freeze, so I want to pick up and understand where we currently > stand with both Neutron and Heat actually getting tested fully in the gate. > > First Neutron - > https://blueprints.launchpad.net/tempest/+spec/fix-gate-tempest-devstack-vm-quantum-full > > > We know that this is *close* as the full job is running non voting > everywhere, and typically passing. How close are we? Or should we be > defering this until Juno (which would be unfortunate). > > Second Heat - > https://blueprints.launchpad.net/tempest/+spec/tempest-heat-integration > > The Heat tests that are in a normal Tempest job are relatively trivial > surface verification, and in no way actually make sure that Heat is > operating at a real level. This fact is a contributing factor to why > Heat was broken in i2. > > The first real verification for Heat is in the Heat slow job (which we > created to give Heat a separate time budget, because doing work that > requires real guests takes time). > > The heat slow job looks like it is finally passing much of the time - > http://logstash.openstack.org/#eyJzZWFyY2giOiIobWVzc2FnZTpcIkZpbmlzaGVkOiBTVUNDRVNTXCIgT1IgbWVzc2FnZTpcIkZpbmlzaGVkOiBGQUlMVVJFXCIpIEFORCBidWlsZF9uYW1lOmNoZWNrLXRlbXBlc3QtZHN2bS1uZXV0cm9uLWhlYXQtc2xvdyIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDEwOTg3NDQ4OH0= > > It's seeing a 78% pass rate in check. Can anyone in the Heat team > confirm that the Failures in this job are actually real failures on > patches that should have been blocked? > > I'd like to get that turned on (and on all the projects) as soon as the > Heat team is confident on it so that Heat actually participates in the > tempest/devstack gate in a material way and we can prevent future issues > where a keystone, nova, neutron or whatever change would break Heat in git. > > I've raised https://bugs.launchpad.net/tempest/+bug/1288970 to track the most common error ( NeutronResourcesTestJSON f ailed to reach CREATE_COMPLETE status within the required time (300 s)) tl;dr sometimes booting alone is taking 244s, so I think this test would be significantly more reliable if the timeout was raised. I would propose raising the default orchestration build_timeout to 600s for now, but it may need to go up to 1200s when the autoscaling scenario is enabled again. https://review.openstack.org/#/c/78756/ Once this change is in heat-slow should be reliable enough to make it voting. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140307/dfb32c64/attachment-0001.html>