Open Stack

On Thu, 2023-01-12 at 10:26 +0100, Gorka Eguileor wrote:
> On 10/01, Stephen Finucane wrote:
> > Another tox 4 PSA. It turns out the tox 3 was not using the command in '[tox]
> > install_command' when installing the package under test. tox 4 does, which means
> > overriding '[tox] install_command' to include a constraints file (-c) will
> > prevent you installing any requirement that is listed in the upper-constraints
> > file, even if said requirement is the thing you're currently working on. This
> > applies to all libraries (e.g. oslo.db, python-cinderclient) but not the
> > services (cinder, nova) since those aren't included in upper-constraints.
> > 
> > The "correct" way to respect upper-constraints is to provide them in 'deps'
> > alongside the requirements file(s), e.g.
> > 
> >   [testenv]
> >   deps =
> >     -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
> >     -r{toxinidir}/requirements.txt
> >     -r{toxinidir}/test-requirements.txt
> > 
> > This will cause tox to install all of the package's dependencies first *with*
> > constraints before installing the package itself *without constraints*. There is
> > a bug report open against pip to change this behaviour [1], but it's been sat
> > there for over two years with no activity so I wouldn't rely on this.
> > 
> > Stephen
> > 
> > [1] https://github.com/pypa/pip/issues/7839
> > 
> > 
> 
> Hi Stephen,
> 
> In my past experience with Cinder tox honored the "install_command" just
> fine, the problem was actually when using "usedevelop = True" and
> setting the constraints in deps.
> 
> We ended up adding a comment in our tox to prevent people from trying to
> move the constraints to "deps", as that created problems.  This is the
> comment present in Cinder's tox.ini:
> 
> # NOTE: Do not move the constraints from the install_command into deps, as that
> #       may result in tox using unconstrained/untested dependencies.
> #       We use "usedevelop = True" for tox jobs (except bindep), so tox does 2
> #       install calls, one for the deps and another for the cinder source code
> #       as editable (pip -e).
> #       Without the constraints in the install_command only the first
> #       installation will honor the upper constraints, and the second install
> #       for cinder itself will not know about the constraints which can result
> #       in installing versions we don't want.
> #       With constraints in the install_command tox will always honor our
> #       constraints.

Hmm, I tried to reproduce this this morning and tox 3 is using 'install_command'
as expected when installing the package under test (assuming 'skipsdist' isn't
set). I think I screwed up here 🙈

> Has this double requirement installation changed?  Will it work fine now
> in the scenario described in our note?

No, it sounds like what you've done is correct. Note that it'll only work for
services though (i.e. things that aren't listed in upper-constraints). You can't
do this for e.g. python-cinderclient. Nothing has changed there though.

Stephen

> Cheers,
> Gorka.