Open Stack

Dear all,

a quick update on the current status.

Zuul has been fixed to use the correct branch for roles coming from
different repositories [1].
The backport of the devstack patches to support multinode jobs is almost
complete. All stable/queens patches are merged, stable/pike patches are
almost all approved and going through the gate [2].

The two facts above mean that now the "devstack-tempest" base job defined
in Tempest can be switched to use the "orchestrate-devstack" role and thus
function as a base for multinode jobs [3].
It also means that work on writing grenade jobs in zuulv3 native format can
now be resumed [4].

Kind regards

Andrea Frittoli

[1]
http://lists.openstack.org/pipermail/openstack-dev/2018-April/129217.html
[2]
https://review.openstack.org/#/q/topic:multinode_zuulv3+(status:open+OR+status:merged
)
[3] https://review.openstack.org/#/c/545724/
[4]
https://review.openstack.org/#/q/status:open+branch:master+topic:grenade_zuulv3


On Mon, Mar 12, 2018 at 2:08 PM Andrea Frittoli <andrea.frittoli at gmail.com>
wrote:

> Dear all,
>
> post-PTG updates:
>
> - the devstack patches for multinode support are now merged on master. You
> can now build your multinode zuulv3 native devstack/tempest test jobs using
> the same base jobs as for single node, and setting a multinode nodeset.
> Documentation landed as well, so you can now find docs on roles [0], jobs
> [1] and a migration guide [2] which will show you which base jobs to start
> with and how to migrate those devstack-gate flags from legacy jobs to the
> zuul v3 jobs.
>
> - the multinode patches including switching of test-matrix (on master) and
> start including the list of devstack services in the base jobs. In doing so
> I used the new neutron service names. That may be causing issues to
> devstack-plugins looking for old service names, so if you encounter an
> issue please reach out in the openstack-qa / openstack-infra rooms. We
> could still roll back to the old names, however the beginning of the cycle
> is probably the best time to sort out issues related to the new names and
> new logic in the neutron - devstack code.
>
> Coming up next:
>
> - backport of devstack patches to stable (queens and pike), so we can
> switch the Tempest job devstack multinode mode and develop grenade zuulv3
> native jobs. I do not plan on backporting the new neutron names to any
> stable branch, let me know if there is any reason to do otherwise.
> - work on grenade is at very early stages [3], so far I got devstack
> running successfully on stable/queens from the /opt/stack/old folder using
> the zuulv3 roles. Next up is actually doing the migration and running all
> relevant checks.
>
> Andrea Frittoli (andreaf)
>
> [0] https://docs.openstack.org/devstack/latest/zuul_roles.html
> [1] https://docs.openstack.org/devstack/latest/zuul_jobs.html
> [2] https://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html
>
> [3]
> https://review.openstack.org/#/q/status:open+branch:master+topic:grenade_zuulv3
>
>
>
> On Tue, Feb 20, 2018 at 9:22 PM Andrea Frittoli <andrea.frittoli at gmail.com>
> wrote:
>
>> Dear all,
>>
>> updates:
>>
>> - host/group vars: zuul now supports declaring host and group vars in the
>> job definition [0][1] - thanks corvus and infra team!
>>   This is a great help towards writing the devstack and tempest base
>> multinode jobs [2][3]
>>   * NOTE: zuul merges dict variables through job inheritance. Variables
>> in host/group_vars override global ones. I will write some examples further
>> clarify this.
>>
>> - stable/pike: devstack ansible changes have been backported to
>> stable/pike, so we can now run zuulv3 jobs against stable/pike too - thank
>> you tosky!
>>   next change in progress related to pike is to provide tempest-full-pike
>> for branchless repositories [4]
>>
>> - documentation: devstack now publishes documentation on its ansible
>> roles [5].
>>   More devstack documentation patches are in progress to provide jobs
>> reference, examples and a job migration how-to [6].
>>
>>
>> Andrea Frittoli (andreaf)
>>
>> [0]
>> https://docs.openstack.org/infra/zuul/user/config.html#attr-job.host_vars
>>
>> [1]
>> https://docs.openstack.org/infra/zuul/user/config.html#attr-job.group_vars
>>
>> [2] https://review.openstack.org/#/c/545696/
>> [3] https://review.openstack.org/#/c/545724/
>> [4] https://review.openstack.org/#/c/546196/
>> [5] https://docs.openstack.org/devstack/latest/roles.html
>> [6] https://review.openstack.org/#/c/545992/
>>
>>
>> On Mon, Feb 19, 2018 at 2:46 PM Andrea Frittoli <
>> andrea.frittoli at gmail.com> wrote:
>>
>>> Dear all,
>>>
>>> updates:
>>> - tempest-full-queens and tempest-full-py3-queens are now available for
>>> testing of branchless repositories [0]. They are used for tempest and
>>> devstack-gate. If you own a tempest plugin in a branchless repo, you may
>>> consider adding similar jobs to your plugin if you use it for tests on
>>> stable/queen as well.
>>> - if you have migrated jobs based on devstack-tempest please let me
>>> know, I'm building reference docs and I'd like to include as many examples
>>> as possible
>>> - work on multi-node is in progress, but not ready still - you can
>>> follow the patches in the multinode branch [1]
>>> - updates on some of the points from my previous email are inline below
>>>
>>> Andrea Frittoli (andreaf)
>>>
>>> [0] http://git.openstack.org/cgit/openstack/tempest/tree/.zuul.yaml#n73
>>> [1]
>>> https://review.openstack.org/#/q/status:open++branch:master+topic:multinode
>>>
>>>
>>>
>>> On Thu, Feb 15, 2018 at 11:31 PM Andrea Frittoli <
>>> andrea.frittoli at gmail.com> wrote:
>>>
>>>> Dear all,
>>>>
>>>> this is the first or a series of ~regular updates on the migration of
>>>> Tempest / Grenade jobs to  Zuul v3 native.
>>>>
>>>> The QA team together with the infra team are working on providing the
>>>> OpenStack community with a set of base Tempest / Grenade jobs that can be
>>>> used as a basis to write new CI jobs / migrate existing legacy ones with a
>>>> minimal effort and very little or no Ansible knowledge as a precondition.
>>>>
>>>> The effort is tracked in an etherpad [0]; I'm trying to keep the
>>>> etherpad up to date but it may not always be a source of truth.
>>>>
>>>> Useful jobs available so far:
>>>> - devstack-tempest [0] is a simple tempest/devstack job that runs
>>>> keystone glance nova cinder neutron swift and tempest *smoke* filter
>>>> - tempest-full [1] is similar but runs a full test run - it replaces
>>>> the legacy tempest-dsvm-neutron-full from the integrated gate
>>>> - tempest-full-py3 [2] runs a full test run on python3 - it replaces
>>>> the legacy tempest-dsvm-py35
>>>>
>>>
>>> Some more details on this topic: what I did not mention in my previous
>>> email is that the autogenerated Tempest / Grenade CI jobs (legacy-*
>>> playbooks) are not meant to be used as a basis for Zuul V3 native jobs. To
>>> create Zuul V3 Tempest / Grenade native jobs for your projects you need to
>>> through away the legacy playbooks and defined new jobs in .zuul.yaml, as
>>> documented in the zuul v3 docs [2].
>>> The parent job for a single node Tempest job will usually be
>>> devstack-tempest. Example migrated jobs are avilable, for instance: [3] [4].
>>>
>>> [2]
>>> https://docs.openstack.org/infra/manual/zuulv3.html#howto-update-legacy-jobs
>>>
>>> [3]
>>> http://git.openstack.org/cgit/openstack/sahara-tests/tree/.zuul.yaml#n21
>>>
>>> [4] https://review.openstack.org/#/c/543048/5
>>>
>>>
>>>>
>>>> Both tempest-full and tempest-full-py3 are part of integrated-gate
>>>> templates, starting from stable/queens on.
>>>> The other stable branches still run the legacy jobs, since
>>>> devstack ansible changes have not been backported (yet). If we do backport
>>>> it will be up to pike maximum.
>>>>
>>>> Those jobs work in single node mode only at the moment. Enabling
>>>> multinode via job configuration only require a new Zuul feature [4][5] that
>>>> should be available soon; the new feature allows defining host/group
>>>> variables in the job definition, which means setting variables which are
>>>> specific to one host or a group of hosts.
>>>> Multinode DVR and Ironic jobs will require migration of the ovs-* roles
>>>> form devstack-gate to devstack as well.
>>>>
>>>> Grenade jobs (single and multinode) are still legacy, even if the
>>>> *legacy* word has been removed from the name.
>>>> They are currently temporarily hosted in the neutron repository. They
>>>> are going to be implemented as Zuul v3 native in the grenade repository.
>>>>
>>>> Roles are documented, and a couple of migration tips for DEVSTACK_GATE
>>>> flags is available in the etherpad [0]; more comprehensive examples /
>>>> docs will be available as soon as possible.
>>>>
>>>> Please let me know if you find this update useful and / or if you would
>>>> like to see different information in it.
>>>> I will send further updates as soon as significant changes / new
>>>> features become available.
>>>>
>>>> Andrea Frittoli (andreaf)
>>>>
>>>> [0]
>>>> https://etherpad.openstack.org/p/zuulv3-native-devstack-tempest-jobs
>>>> [1] http://git.openstack.org/cgit/openstack/tempest/tree/.zuul.yaml#n1
>>>> [2] http://git.openstack.org/cgit/openstack/tempest/tree/.zuul.yaml#n29
>>>>
>>>> [3] http://git.openstack.org/cgit/openstack/tempest/tree/.zuul.yaml#n47
>>>>
>>>> [4] https://etherpad.openstack.org/p/zuulv3-group-variables
>>>> [5] https://review.openstack.org/#/c/544562/
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180419/41dcce70/attachment.html>