Open Stack

On Thu, Jul 6, 2017 at 6:07 AM, Honza Pokorny <honza at redhat.com> wrote:
> On 2017-07-05 12:42, Ben Nemec wrote:
>>
>>
>> On 07/05/2017 10:21 AM, Emilien Macchi wrote:
>> > On Wed, Jul 5, 2017 at 7:57 AM, Florian Fuchs <flfuchs at redhat.com> wrote:
>> > > On Tue, Jul 4, 2017 at 7:48 PM, Emilien Macchi <emilien at redhat.com> wrote:
>> > > > On Fri, Jun 30, 2017 at 10:51 AM, Honza Pokorny <honza at redhat.com> wrote:
>> > > > > I'd like to write some integration tests for the TripleO UI.  Simple
>> > > > > configuration checks, workflow tests, websocket tests, etc.
>> > > > >
>> > > > > I had a quick look at the tripleo-ci [1] repository, and it wasn't
>> > > > > immediately obvious to me how to extend the existing infrastructure to
>> > > > > add something like this.  It seems to me that the current way is just
>> > > > > "stick some shell script here and here".  The tripleo.sh script is about
>> > > > > 1,600 LOC and it look rather intimidating.
>> > > > >
>> > > > > I'm looking for help or pointers on how to do this.  Is there a standard
>> > > > > way of accomplishing this?  Are there any helpers I should be aware of?
>> > > > > Is there any documentation beyond what is in the git tree?
>> > > > >
>> > > > > Any help would be appreciated
>> > > >
>> > > > I think we have 2 options here (which could also be mixed):
>> > > >
>> > > > - Using tripleo-validatons and make sure we run it in CI.
>> > >
>> > > +1 for using ansible playbooks to write UI tests. I just don't think
>> > > we should host the test playbooks in the tripleo-validations
>> > > repository, because IMO testing the UI is different to what the
>> > > tripleo-validations are doing.
>> > > That said, the UI tests could use the ansible inventory from
>> > > tripleo-validations -- which ends up in
>> > > /usr/bin/tripleo-ansible-inventory and is already being used by
>> > > playbooks other than tripleo-validations.
>> >
>> > If tripleo-ui functional tests are written in Python, it makes a lot
>> > of sense to write a Tempest plugin instead of using Ansible.
>>
>> +1.  This seems like an obvious place to take advantage of the framework
>> Tempest already has in place.  These aren't intended as user-facing tests so
>> implementing them as validations would be a weird fit to me.
>
> OK, great, let's go with tempest.  Initially, I was intrigued with the
> idea of reusing some of the ansible code we already have in the
> validations repository.  But in the end, it seems like tempest is the
> better tool for what we want.

Let's give it a try and see how Tempest works for us. I'm ok with baby
steps and adapt if we realize it wasn't what we needed.

> Emilien, how do we go about setting up the new project (GitHub, gerrit,
> etc)?


Long story: https://docs.openstack.org/infra/manual/creators.html

Short story: https://review.openstack.org/#/c/459762/ /
https://review.openstack.org/#/c/459993/

>>
>> > The major reason to me is consistency with how we test other services
>> > in TripleO.
>> >
>> > Whatever option we'll take, we'll need a new repository I think that
>> > will contain the actual tests. There is a goal in Queens where Tempest
>> > plugins are out of tree:
>> > https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
>> > If we take that path, we'll create a new repo for that.
>> >
>> >
>> > The Ansible option would also be fine but we need to acknowledge the
>> > fact tripleo-ui testing wouldn't be possible without deploying the
>> > tripleo-validations bits.
>> >
>> > > Florian
>> > >
>> > >
>> > > > - Write a tempest plugin for tripleo-ui (like Horizon has one:
>> > > > https://github.com/openstack/tempest-horizon) (it could be
>> > > > openstack/tempest-triploe-ui) for example. We could run the tempest
>> > > > tests in the CI jobs (we're working on running Tempest on scenario
>> > > > jobs, see efforts made in
>> > > > https://trello.com/c/Z8jIillp/252-spec-out-the-work-required-to-run-tempest-on-the-tripleo-gating-jobs).
>> > > >
>> > > > Please do not implement tests in tripleo-ci or quickstart, I think we
>> > > > need a way to test it outside CI and include it directly in the
>> > > > product.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > > Thanks!
>> > > > >
>> > > > > Honza Pokorny
>> > > > >
>> > > > > [1]: https://github.com/openstack-infra/tripleo-ci
>> > > > >
>> > > > > __________________________________________________________________________
>> > > > > OpenStack Development Mailing List (not for usage questions)
>> > > > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Emilien Macchi
>> > > >
>> > > > __________________________________________________________________________
>> > > > OpenStack Development Mailing List (not for usage questions)
>> > > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> > >
>> > > __________________________________________________________________________
>> > > OpenStack Development Mailing List (not for usage questions)
>> > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Emilien Macchi