Open Stack

On 10/13/2016 11:59 PM, Steve Martinelli wrote:
> On Thu, Oct 13, 2016 at 10:30 PM, Davanum Srinivas <davanum at gmail.com
> <mailto:davanum at gmail.com>> wrote:
>
>
>     Matt, Emilien,
>
>     there's one more option, run jobs daily and add the output to the
>     openstack health dashboard.
>     example -
>     http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master
>     <http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master>
>
>
> But that only runs against what is merged on master right? I think
> Emilien wants to catch changes that break before they are merged.
>
> Emilien, I think it's also worth noting which projects you intend to
> make these changes to? Just "core" projects + projects that tripleO uses
> (heat + ironic) + projects that have burned you in the past (OSC)? Or do
> you plan on covering all projects?
>
> Finding a balance between not enough testing, and overusing infra
> resources is tricky, we should only aim for high value targets like the
> ones listed above. I can't imagine this being run on all check jobs
> everywhere.

It's not just overuse of infra resources. Anything that's more 
complicated than unit tests has > 0% chance of failure. So every new 
complex full stack test is going to -1 some set of good patches.

The best thing to do when there is a break is to figure out what the 
root cause was, and push testing back as low in the stack as possible. 
Was there a good unit test that could have prevented it? That's super 
cheap, and helps a ton going forward. Or realize that projects are using 
undefined behavior of other tools, and we need to define that behavior.

The deployment teams are releasing their final version of things weeks 
after the newton release hits. By nature there is some following time.

I kind of wonder if that hints to a better model here instead of the 
deployments running services from master. Instead running periodics and 
moving forward reference hashes every day if tests pass, and not if they 
fail. That would let deployment tools automatically move forward, quite 
close to master, but not get tightly coupled into every change.

Because I do get concerned for deciding that every developer needs to 
understand tripleo, and ansible, and fuel and the configuration 
decisions they made, the bugs they have, the way their services are 
deployed, to be able to land any patches. Because I think that will 
further reduce the pool of developers that can contribute.

	-Sean

-- 
Sean Dague
http://dague.net