Open Stack

On 2017-07-05 12:42, Ben Nemec wrote:
> 
> 
> On 07/05/2017 10:21 AM, Emilien Macchi wrote:
> > On Wed, Jul 5, 2017 at 7:57 AM, Florian Fuchs <flfuchs at redhat.com> wrote:
> > > On Tue, Jul 4, 2017 at 7:48 PM, Emilien Macchi <emilien at redhat.com> wrote:
> > > > On Fri, Jun 30, 2017 at 10:51 AM, Honza Pokorny <honza at redhat.com> wrote:
> > > > > I'd like to write some integration tests for the TripleO UI.  Simple
> > > > > configuration checks, workflow tests, websocket tests, etc.
> > > > > 
> > > > > I had a quick look at the tripleo-ci [1] repository, and it wasn't
> > > > > immediately obvious to me how to extend the existing infrastructure to
> > > > > add something like this.  It seems to me that the current way is just
> > > > > "stick some shell script here and here".  The tripleo.sh script is about
> > > > > 1,600 LOC and it look rather intimidating.
> > > > > 
> > > > > I'm looking for help or pointers on how to do this.  Is there a standard
> > > > > way of accomplishing this?  Are there any helpers I should be aware of?
> > > > > Is there any documentation beyond what is in the git tree?
> > > > > 
> > > > > Any help would be appreciated
> > > > 
> > > > I think we have 2 options here (which could also be mixed):
> > > > 
> > > > - Using tripleo-validatons and make sure we run it in CI.
> > > 
> > > +1 for using ansible playbooks to write UI tests. I just don't think
> > > we should host the test playbooks in the tripleo-validations
> > > repository, because IMO testing the UI is different to what the
> > > tripleo-validations are doing.
> > > That said, the UI tests could use the ansible inventory from
> > > tripleo-validations -- which ends up in
> > > /usr/bin/tripleo-ansible-inventory and is already being used by
> > > playbooks other than tripleo-validations.
> > 
> > If tripleo-ui functional tests are written in Python, it makes a lot
> > of sense to write a Tempest plugin instead of using Ansible.
> 
> +1.  This seems like an obvious place to take advantage of the framework
> Tempest already has in place.  These aren't intended as user-facing tests so
> implementing them as validations would be a weird fit to me.

OK, great, let's go with tempest.  Initially, I was intrigued with the
idea of reusing some of the ansible code we already have in the
validations repository.  But in the end, it seems like tempest is the
better tool for what we want.

Emilien, how do we go about setting up the new project (GitHub, gerrit,
etc)?

> 
> > The major reason to me is consistency with how we test other services
> > in TripleO.
> > 
> > Whatever option we'll take, we'll need a new repository I think that
> > will contain the actual tests. There is a goal in Queens where Tempest
> > plugins are out of tree:
> > https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
> > If we take that path, we'll create a new repo for that.
> > 
> > 
> > The Ansible option would also be fine but we need to acknowledge the
> > fact tripleo-ui testing wouldn't be possible without deploying the
> > tripleo-validations bits.
> > 
> > > Florian
> > > 
> > > 
> > > > - Write a tempest plugin for tripleo-ui (like Horizon has one:
> > > > https://github.com/openstack/tempest-horizon) (it could be
> > > > openstack/tempest-triploe-ui) for example. We could run the tempest
> > > > tests in the CI jobs (we're working on running Tempest on scenario
> > > > jobs, see efforts made in
> > > > https://trello.com/c/Z8jIillp/252-spec-out-the-work-required-to-run-tempest-on-the-tripleo-gating-jobs).
> > > > 
> > > > Please do not implement tests in tripleo-ci or quickstart, I think we
> > > > need a way to test it outside CI and include it directly in the
> > > > product.
> > > > 
> > > > Thanks,
> > > > 
> > > > > Thanks!
> > > > > 
> > > > > Honza Pokorny
> > > > > 
> > > > > [1]: https://github.com/openstack-infra/tripleo-ci
> > > > > 
> > > > > __________________________________________________________________________
> > > > > OpenStack Development Mailing List (not for usage questions)
> > > > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > > > 
> > > > 
> > > > 
> > > > --
> > > > Emilien Macchi
> > > > 
> > > > __________________________________________________________________________
> > > > OpenStack Development Mailing List (not for usage questions)
> > > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > > 
> > > __________________________________________________________________________
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> > 
> > 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev