[openstack-dev] [devstack][qa] Changes to devstack LIBS_FROM_GIT

James E. Blair corvus at inaugust.com
Wed Mar 28 20:21:38 UTC 2018


I've proposed a change to devstack which slightly alters the
LIBS_FROM_GIT behavior.  This shouldn't be a significant change for
those using legacy devstack jobs (but you may want to be aware of it).
It is more significant for new-style devstack jobs.

The change is at https://review.openstack.org/549252

In summary, when this change lands, new-style devstack jobs should no
longer need to set LIBS_FROM_GIT explicitly.  Existing legacy jobs
should be unaffected (but there is a change to the verification process
performed by devstack).

Currently devstack expects the contents of LIBS_FROM_GIT to be
exclusively a list of python packages which, obviously, should be
installed from git and not pypi.  It is used for two purposes:
determining whether an individual package should be installed from git,
and verifying that a package was installed from git.

In the old devstack-gate system, we prepared many of the common git
repos, whether they were used or not.  So LIBS_FROM_GIT was created to
indicate that in some cases devstack should ignore those repos and
install from pypi instead.  In other words, its original purpose was
purely as a method of selecting whether a devstack-gate prepared repo
should be used or ignored.

In Zuul v3, we have a good way to indicate whether a job is going to use
a repo or not -- add it to "required-projects".  Considering that, the
LIBS_FROM_GIT variable is redundant.  So my patch causes it to be
automatically generated based on the contents of required-projects.
This means that job authors don't need to list every required repository

However, a naïve implementation of that runs afoul of the second use of
LIBS_FROM_GIT -- verifying that python packages are installed from git.

This usage was added later, after a typographical error ("-" vs "_" in a
python package name) in a constraints file caused us not to install a
package from git.  Now devstack verifies that every package in
LIBS_FROM_GIT is installed.  However, Zuul doesn't know that devstack,
tempest, and other packages aren't installed.  So adding them
automatically to LIBS_FROM_GIT will cause devstack to fail.

My change modifies this verification to only check that packages
mentioned in LIBS_FROM_GIT that devstack tried to install were actually
installed.  I realize that stated as such this sounds tautological,
however, this check is still valid -- it would have caught the original
error that prompted the check in the first case.

What the revised check will no longer handle is a typo in a legacy job.
If someone enters a typo into LIBS_FROM_GIT, it will no longer fail.
However, I think the risk is worthwhile -- particularly since it is in
service of a system which eliminates the opportunity to introduce such
an error in the first place.

To see the result in action, take a look at this change which, in only a
few lines, implements what was a significantly more complex undertaking
in Zuul v2:


Finally, a note on the automatic generation of LIBS_FROM_GIT -- if, for
some reason, you require a new-style devstack job to manually set
LIBS_FROM_GIT, that will still work.  Simply define the variable as
normal, and the module which generates the devstack config will bypass
automatic generation if the variable is already set.


More information about the OpenStack-dev mailing list