Open Stack

On Thu, Nov 24, 2016 at 11:08 AM, Juan Antonio Osorio
<jaosorior at gmail.com> wrote:
> I don't have a strong opinion about any option, as long as we have something
> in place I'm happy.
>
> But regarding option 1.A: what would be done for newton once these templates
> are moved to t-h-t. Would they be backported? What about mitaka?

Newton: yes: we'll have to backport
https://review.openstack.org/#/c/402119/ and that's it!
Mitaka: we never supported and ran scenarios, so no problem here.

I started a first iteration:
https://review.openstack.org/#/q/topic:tripleo-ci/scenarios

Feel free to review and give any feedback.
Thanks,

>
> On 24 Nov 2016 17:55, "Carlos Camacho Gonzalez" <ccamacho at redhat.com> wrote:
>>
>> I think would be cool to go with option, +1 to 1.A
>>
>> IMHO,
>> - Easier to read.
>> - Easier to maintain.
>> - We don't make backports, instead we guarantee backwards compatibility.
>> - We'll re-use experience from Puppet OpenStack CI.
>>
>> On Wed, Nov 23, 2016 at 10:13 PM, Giulio Fidente <gfidente at redhat.com>
>> wrote:
>> > hi Emilien,
>> >
>> > thanks for putting some thought into this. We have a similar problem to
>> > test
>> > RGW which was only added in Newton.
>> >
>> >
>> > On 11/23/2016 03:02 AM, Emilien Macchi wrote:
>> >>
>> >> == Context
>> >>
>> >> In Newton we added new multinode jobs called "scenarios".
>> >> The challenged we tried to solve was "how to test the maximum of
>> >> services without overloading the nodes that run tests".
>> >>
>> >> Each scenarios deploys a set of services, which allows us to
>> >> horizontally scale the number of scenarios to increase the service
>> >> testing coverage.
>> >> See the result here:
>> >> https://github.com/openstack-infra/tripleo-ci#service-testing-matrix
>> >>
>> >> To implement this model, we took example of Puppet OpenStack CI:
>> >> https://github.com/openstack/puppet-openstack-integration#description
>> >> We even tried to keep consistent the services/scenarios relations, so
>> >> it's consistent and easier to maintain.
>> >>
>> >> Everything was fine until we had to add new services during Ocata
>> >> cycles.
>> >> Because tripleo-ci repository is not branched, adding Barbican service
>> >> in the TripleO environment for scenario002 would break Newton CI jobs.
>> >> During my vacations, the team created a new scenario, scenario004,
>> >> that deploys Barbican and that is only run for Ocata jobs.
>> >> I don't think we should proceed this way, and let me explain why.
>> >>
>> >> == Problem
>> >>
>> >> How to scale the number of services that we test without increasing
>> >> the number of scenarios and therefore the complexity of maintaining
>> >> them on long-term.
>> >>
>> >>
>> >> == Solutions
>> >>
>> >> The list is not exhaustive, feel free to add more.
>> >>
>> >> 1) Re-use experience from Puppet OpenStack CI and have environments
>> >> that are in a branched repository.
>> >> environments.
>> >> In Puppet OpenStack CI, the repository that deploys environments
>> >> (puppet-openstack-integration) is branched. So if puppet-barbican is
>> >> ready to be tested in Ocata, we'll patch
>> >> puppet-openstack-integration/master to start testing it and it won't
>> >> break stable jobs.
>> >> Like this, we were successfully able to maintain a fair number of
>> >> scenarios and keep our coverage increasing over each cycle.
>> >>
>> >> I see 2 sub-options here:
>> >>
>> >> a) Move CI environments and pingtest into
>> >> tripleo-heat-templates/environments/ci/(scenarios|pingtest). This repo
>> >> is branched and we could add a README to explain these files are used
>> >> in CI and we don't guarantee they would work outside TripleO CI tools.
>> >> b) Branch tripleo-ci repository. Personally I don't like this solution
>> >> because a lot of patches in this repo are not related to OpenStack
>> >> versions, which means we would need to backport most of the things
>> >> from master.
>> >
>> >
>> > I'd +1 this idea. It sounds like we could make the scenarios generic
>> > enough
>> > to be usable also outside CI? Maybe they can serve as samples?
>> > --
>> > Giulio Fidente
>> > GPG KEY: 08D733BA
>> >
>> >
>> >
>> > __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi