Open Stack

On 15/01/15 23:41, Sean Dague wrote:
> On 01/15/2015 06:25 PM, Joe Gordon wrote:
>> We can side step the dependency graphing and ordering issue by looking
>> at the list of curently installed packages via pip freeze and not
>> installing dependencies (pip install --no-deps)
>>
>> After looking into this further here are the known issues:
>>
>> * Partial capping won't work [0], so we need to pin all dependencies, we
>> can generate this list per file via "pip install -r" and "pip freeze",
>> but this doesn't address the issue of apt-get vs pip install. For
>> example in the stable gate we use suds 0.4.1 but only suds 0.4.0 is
>> available via pip.
>> * Not all packages are installed in are standard dsvm-tempest env, so
>> using pip-freeze from that isn't enough
>> * We need to run this per requirements file and move to using pip
>> install --no-deps everywhere. As the global-requirements sync wouldn't
>> work the first time since files don't list all transient dependencies yet.
> So that's basically writing our own dependency system entirely, and
> skipping pip's (vs. fudging around pip's issues). I expect that's going
> to go poorly in the OpenStack ecosystem realm because a lot of very
> repetitive manual analysis will need to be done on each project. And if
> we want to bump a cap, regenerating all the graphs becomes another
> manual process.

Pip's loose dependency processing where each entry is treated as an
entirely separate resolution with no consideration of previous or later
constraints came up recently in a discussion at work.

Looking around I discovered someone had already started working on a
solution, which involves moving the current requirements to a
'requirements.in' file and then compiling that to requirements.txt,
which would contain only fully pinned packages in the correct order for
pip to install them without needing to add any special options.

Might be worth seeing if that could be developed further:
http://nvie.com/posts/better-package-management/
https://github.com/nvie/pip-tools/tree/future



>> * We can still break if a package version is removed from pypi
>> * in pip-freeze we sometimes install versions lower then our minimum
>> version (python-libvirt!)
> That's because python-libvirt is not in any requirements.txt file, so
> we're taking the system version.
>
> 	-Sean
>

Regards,
Darragh Bailey

"Nothing is foolproof to a sufficiently talented fool" - Unknown