Open Stack

On Wed, Nov 8, 2017 at 5:10 PM, Wesley Hayutin <whayutin at redhat.com> wrote:

>
>
> On Wed, Nov 8, 2017 at 5:00 PM, Alex Schultz <aschultz at redhat.com> wrote:
>
>> On Tue, Nov 7, 2017 at 2:59 PM, Emilien Macchi <emilien at redhat.com>
>> wrote:
>> > On Wed, Nov 8, 2017 at 3:30 AM, James Slagle <james.slagle at gmail.com>
>> wrote:
>> >> On Sun, Nov 5, 2017 at 7:01 PM, Emilien Macchi <emilien at redhat.com>
>> wrote:
>> >>> On Mon, Oct 2, 2017 at 5:02 AM, Dan Prince <dprince at redhat.com>
>> wrote:
>> >>> [...]
>> >>>
>> >>>>  -CI resources: better use of CI resources. At the PTG we received
>> >>>> feedback from the OpenStack infrastructure team that our upstream CI
>> >>>> resource usage is quite high at times (even as high as 50% of the
>> >>>> total). Because of the shared framework and single node capabilities
>> we
>> >>>> can re-architecture much of our upstream CI matrix around single
>> node.
>> >>>> We no longer require multinode jobs to be able to test many of the
>> >>>> services in tripleo-heat-templates... we can just use a single cloud
>> VM
>> >>>> instead. We'll still want multinode undercloud -> overcloud jobs for
>> >>>> testing things like HA and baremetal provisioning. But we can cover a
>> >>>> large set of the services (in particular many of the new scenario
>> jobs
>> >>>> we added in Pike) with single node CI test runs in much less time.
>> >>>
>> >>> After the last (terrible) weeks in CI, it's pretty clear we need to
>> >>> find a solution to reduce and optimize our testing.
>> >>> I'm now really convinced by switching our current scenarios jobs to
>> >>> NOT deploy the overcloud, and just an undercloud with composable
>> >>> services & run tempest.
>> >>
>> >> +1 if you mean just the scenarios.
>> >
>> > Yes, just scenarios.
>> >
>> >> I think we need to keep at least 1 multinode job voting that deploys
>> >> the overcloud, probably containers-multinode.
>> >
>> > Yes, exactly, and also work on optimizing OVB jobs (maybe just keep
>> > one or 2 jobs, instead 3).
>> >
>> >>> Benefits:
>> >>> - deploy 1 node instead of 2 nodes, so we save nodepool resources
>> >>> - faster (no overcloud)
>> >>> - reduce gate queue time, faster development process, faster CI
>> >>>
>> >>> Challenges:
>> >>> - keep overcloud testing, with OVB
>> >>
>> >> This is why I'm not sure what you're proposing. Do you mean switch all
>> >> multinode jobs to be just an undercloud, or just the scenarios?
>> >
>> > Keep 1 or 2 OVB jobs, to test ironic + mistral + HA (HA could be
>> > tested with multinode though but well).
>> >
>> >>> - reduce OVB to strict minimum: Ironic, Nova, Mistral and basic
>> >>> containerized services on overcloud.
>> >>>
>> >>> I really want to get consensus on these points, please raise your
>> >>> voice now before we engage some work on that front.
>> >>
>> >> I'm fine to optimize the scenarios to be undercloud driven, but feel
>> >> we still need a multinode job that deploys the overcloud in the gate.
>> >> Otherwise, we'll have nothing that deploys an overcloud in the gate,
>> >> which is a step in the wrong direction imo. Primarily, b/c of the loss
>> >> of coverage around mistral and all of our workflows. Perhaps down the
>> >> road we could find ways to optimize that by using an ephemeral Mistral
>> >> (similar to the ephemeral Heat container), and then use a single node,
>> >> but we're not there yet.
>> >>
>> >> On the other hand, if the goal is just to test less upstream so that
>> >> we can more quickly merge code, then *not* deploying an overcloud in
>> >> the gate at all seems to fit that goal. Is that what you're after?
>> >
>> > Yes. Thanks for reformulate with better words.
>> > Just to be clear, I want to transform the scenarios into single-node
>> > jobs that deploy the SAME services (using composable services) from
>> > the undercloud, using the new ansible installer. I also want to keep
>> > running Tempest.
>> > And of course, like we said, keep one multinode job to test overcloud
>> > workflow, and OVB with some adjustments.
>> >
>>
>> So I'm ok with switching to use the containerized undercloud deploy to
>> smoke test functionality of more complex openstack service
>> deployments. What I would like to see prior to investing in this is
>> that the plain containerized undercloud deploy job reliability is on
>> par with the existing undercloud install.  We had to switch the
>> undercloud-containers back to non-voting due to higher failure rates
>> and it is still not voting.
>
>
> Agree, once we have a little success I'll update featureset027 ( the
> undercloud-containers )
> which is still non-voting to use this updated containerized deployment.
> Then we'll compare
> the undercloud-oooq to undercloud-containers (fs027) After a few weeks of
> running.
>
>
>> With the current state of CI being
>> questionable due to random failures which are not fully have resolved,
>> I would prefer that we ensure existing CI is stable and that what we
>> plan to move is as stable.
>>
>
> Agreed,
> There are times IMHO when one must strike when the iron is hot on certain
> parts of the work here.  I felt compelled to help bootstrap Ian with the
> containerized undercloud work or see old habits remain and have things
> running in tripleo.sh or other shell scripts.
>
> The priority is to help stablize the upstream CI for sure!
>
>
>>
>> Additionally I think we need to ensure that the ovb jobs that still do
>> full deployment process become voting by switching to 3rd party CI.
>>
>
> Yes, agreed.  I'll reach out to the rdo sf admins and see if we get it
> voting -2.
> That is what we discussed at summit.
>
>
>>
>> Thanks,
>> -Alex
>>
>> > Is it good?
>> >
>> > Thanks,
>> > --
>> > Emilien Macchi
>> >
>> > ____________________________________________________________
>> ______________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: OpenStack-dev-request at lists.op
>> enstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
FYI..  I thought it would be a good idea to allow folks to work on the
containerized undercloud w/o
using a bunch of upstream resources.

https://review.rdoproject.org/r/#/c/10468/

The above patch would allow us to recheck the containerized
undercloud install w/o running recheck.
You would need a change on instack-undercloud and to execute "check rdo
experimental"

Reviews are welcome
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171110/488464ba/attachment.html>