Open Stack

On 07/05/2017 10:21 AM, Emilien Macchi wrote:
> On Wed, Jul 5, 2017 at 7:57 AM, Florian Fuchs <flfuchs at redhat.com> wrote:
>> On Tue, Jul 4, 2017 at 7:48 PM, Emilien Macchi <emilien at redhat.com> wrote:
>>> On Fri, Jun 30, 2017 at 10:51 AM, Honza Pokorny <honza at redhat.com> wrote:
>>>> I'd like to write some integration tests for the TripleO UI.  Simple
>>>> configuration checks, workflow tests, websocket tests, etc.
>>>>
>>>> I had a quick look at the tripleo-ci [1] repository, and it wasn't
>>>> immediately obvious to me how to extend the existing infrastructure to
>>>> add something like this.  It seems to me that the current way is just
>>>> "stick some shell script here and here".  The tripleo.sh script is about
>>>> 1,600 LOC and it look rather intimidating.
>>>>
>>>> I'm looking for help or pointers on how to do this.  Is there a standard
>>>> way of accomplishing this?  Are there any helpers I should be aware of?
>>>> Is there any documentation beyond what is in the git tree?
>>>>
>>>> Any help would be appreciated
>>>
>>> I think we have 2 options here (which could also be mixed):
>>>
>>> - Using tripleo-validatons and make sure we run it in CI.
>>
>> +1 for using ansible playbooks to write UI tests. I just don't think
>> we should host the test playbooks in the tripleo-validations
>> repository, because IMO testing the UI is different to what the
>> tripleo-validations are doing.
>> That said, the UI tests could use the ansible inventory from
>> tripleo-validations -- which ends up in
>> /usr/bin/tripleo-ansible-inventory and is already being used by
>> playbooks other than tripleo-validations.
>
> If tripleo-ui functional tests are written in Python, it makes a lot
> of sense to write a Tempest plugin instead of using Ansible.

+1.  This seems like an obvious place to take advantage of the framework 
Tempest already has in place.  These aren't intended as user-facing 
tests so implementing them as validations would be a weird fit to me.

> The major reason to me is consistency with how we test other services
> in TripleO.
>
> Whatever option we'll take, we'll need a new repository I think that
> will contain the actual tests. There is a goal in Queens where Tempest
> plugins are out of tree:
> https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
> If we take that path, we'll create a new repo for that.
>
>
> The Ansible option would also be fine but we need to acknowledge the
> fact tripleo-ui testing wouldn't be possible without deploying the
> tripleo-validations bits.
>
>> Florian
>>
>>
>>> - Write a tempest plugin for tripleo-ui (like Horizon has one:
>>> https://github.com/openstack/tempest-horizon) (it could be
>>> openstack/tempest-triploe-ui) for example. We could run the tempest
>>> tests in the CI jobs (we're working on running Tempest on scenario
>>> jobs, see efforts made in
>>> https://trello.com/c/Z8jIillp/252-spec-out-the-work-required-to-run-tempest-on-the-tripleo-gating-jobs).
>>>
>>> Please do not implement tests in tripleo-ci or quickstart, I think we
>>> need a way to test it outside CI and include it directly in the
>>> product.
>>>
>>> Thanks,
>>>
>>>> Thanks!
>>>>
>>>> Honza Pokorny
>>>>
>>>> [1]: https://github.com/openstack-infra/tripleo-ci
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> --
>>> Emilien Macchi
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>