Open Stack

Excerpts from Jim Rollenhagen's message of 2018-04-18 13:44:08 -0400:
> Hi all,
> 
> We have a number of stable branch jobs failing[0] with an error about pep8
> not being importable[1], when it's clearly installed[2]. We first saw this
> when installing networking-generic-switch on queens in our multinode
> grenade job. We hacked a fix there[3], as we couldn't figure it out and
> thought it was a fluke. Now it's showing up elsewhere.
> 
> I suspected a new pycodestyle was the culprit (maybe it kills off the pep8
> package somehow?) but pinning pycodestyle back a version didn't seem to
> help.
> 
> Any ideas what might be going on here? I'm completely lost.
> 
> P.S. if anyone has the side question of why pep8 is being imported at
> install time, it seems that pbr iterates over any entry points under
> 'distutils.commands' for any installed package. flake8 has one of these
> which must import pep8 to be resolved. I'm not sure *why* pbr needs to do
> this, but I'll assume it's necessary.
> 
> [0] https://review.openstack.org/#/c/557441/
> [1]
> http://logs.openstack.org/41/557441/1/gate/ironic-tempest-dsvm-ironic-inspector-queens/5a4a6c9/logs/devstacklog.txt.gz#_2018-04-16_15_48_01_508
> [2]
> http://logs.openstack.org/41/557441/1/gate/ironic-tempest-dsvm-ironic-inspector-queens/5a4a6c9/logs/devstacklog.txt.gz#_2018-04-16_15_47_40_822
> [3] https://review.openstack.org/#/c/561358/
> 
> // jim

Reading through that log more carefully, I see an early attempt to pin
pycodestyle <= 2.3.1 [1], followed later by pycodestyle == 2.4.0 being
pulled in as a dependency of flake8-import-order==0.12 when neutron's
test-requirements.txt is installed [2]. Then later when ironic's
test-requirements.txt is installed pycodestyle is downgraded to 2.3.1
[3].

Reproducing those install & downgrade steps, I see that pycodestyle
2.4.0 claims to own pep8.py but pycodestyle 2.3.1 does not [4]. So that
explains why pep8 is not re-installed when pycodestyle is downgraded.

I think the real problem here is that we have linter dependencies listed
in the test-requirements.txt files for our projects, and they are
somehow being installed without the constraints. I don't think they need
to be installed for devstack at all, so one way to fix it would be to
move those dependencies to the tox.ini section for running pep8, or to
have devstack look at the blacklisted packages and skip installing them.

Doug

[1] http://logs.openstack.org/41/557441/1/gate/ironic-tempest-dsvm-ironic-inspector-queens/5a4a6c9/logs/devstacklog.txt.gz#_2018-04-16_15_39_00_392
[2] http://logs.openstack.org/41/557441/1/gate/ironic-tempest-dsvm-ironic-inspector-queens/5a4a6c9/logs/devstacklog.txt.gz#_2018-04-16_15_44_56_527
[3] http://logs.openstack.org/41/557441/1/gate/ironic-tempest-dsvm-ironic-inspector-queens/5a4a6c9/logs/devstacklog.txt.gz#_2018-04-16_15_47_40_120
[4] http://paste.openstack.org/show/719580/