Open Stack

On Fri, Oct 14, 2016 at 7:33 AM, Sean Dague <sean at dague.net> wrote:
> On 10/13/2016 11:59 PM, Steve Martinelli wrote:
>>
>> On Thu, Oct 13, 2016 at 10:30 PM, Davanum Srinivas <davanum at gmail.com
>> <mailto:davanum at gmail.com>> wrote:
>>
>>
>>     Matt, Emilien,
>>
>>     there's one more option, run jobs daily and add the output to the
>>     openstack health dashboard.
>>     example -
>>
>> http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master
>>
>> <http://status.openstack.org/openstack-health/#/g/build_name/periodic-ironic-py35-with-oslo-master>
>>
>>
>> But that only runs against what is merged on master right? I think
>> Emilien wants to catch changes that break before they are merged.
>>
>> Emilien, I think it's also worth noting which projects you intend to
>> make these changes to? Just "core" projects + projects that tripleO uses
>> (heat + ironic) + projects that have burned you in the past (OSC)? Or do
>> you plan on covering all projects?
>>
>> Finding a balance between not enough testing, and overusing infra
>> resources is tricky, we should only aim for high value targets like the
>> ones listed above. I can't imagine this being run on all check jobs
>> everywhere.
>
>
> It's not just overuse of infra resources. Anything that's more complicated
> than unit tests has > 0% chance of failure. So every new complex full stack
> test is going to -1 some set of good patches.
>
> The best thing to do when there is a break is to figure out what the root
> cause was, and push testing back as low in the stack as possible. Was there
> a good unit test that could have prevented it? That's super cheap, and helps
> a ton going forward. Or realize that projects are using undefined behavior
> of other tools, and we need to define that behavior.
>
> The deployment teams are releasing their final version of things weeks after
> the newton release hits. By nature there is some following time.
>
> I kind of wonder if that hints to a better model here instead of the
> deployments running services from master. Instead running periodics and
> moving forward reference hashes every day if tests pass, and not if they
> fail. That would let deployment tools automatically move forward, quite
> close to master, but not get tightly coupled into every change.
>
> Because I do get concerned for deciding that every developer needs to
> understand tripleo, and ansible, and fuel and the configuration decisions
> they made, the bugs they have, the way their services are deployed, to be
> able to land any patches. Because I think that will further reduce the pool
> of developers that can contribute.

I explicitly mentioned the opposite in the proposal, let me copy/paste it again:

"We don't expect developers to investigate why new CI jobs would fail
(ex: a Nova developer to look at TripleO CI job), it would be unfair.
Just some horizontal communication would be enough, IRC or email, to
inform that a patch might break a CI job outside devstack."

If a CI job is breaking because of a patch, deployment tools folks
would look at the CI job and tell to the project why it fails, who
would decide if whether or not it's a blocker (ie: indeed, the patch
is breaking backward compatibility, or no, the patch is good,
installer has to be updated, etc).

I hope it's clear that we don't expect project to spend time on this
thing and of course we don't want to reduce the pool of devs here.

-- 
Emilien Macchi