Open Stack

Hi,

This is a special message for Robert Collins, as I believe he's the one
responsible for the breakage. If it's not your fault, then I'm sorry,
and whoever does the breakage should read what's below carefully, so
that it doesn't happen again.

Robert, while I do appreciate all of your work, and your technically
sound contributions, I am having a hard time with your habit to
regularly break backward AND forward API compatibility. Yes, sometimes
we unfortunately must do it. But this should be a very rare exception,
and you've been doing it over and over again, making package
maintainer's life miserable.

This first happened with PBR. Kilo can't use >= 1.x, and Liberty can't
use <= 1.x. So I can't upload PBR 1.3.0 to Sid. This has been dealt with
because I am the maintainer of PBR, but really, it shouldn't have
happen. How come for years, upgrading PBR always worked, and suddenly,
when you start contributing to it, it breaks backward compat? I'm having
a hard time to understand what's the need to break something which
worked perfectly for so long. I'd appreciate more details.

But for mock, that's another story. I'm not the maintainer, and the one
who is, decided it was a good moment to upload to Sid. The result is 9
FTBFS (failures to build from source) so far, because mock >= 1.1 is
incompatible with Kilo (but does work well with Liberty, which
*requires* it).

I am currently unsure why the maintainer of mock uploaded to Sid and not
to experimental. What needs to be considered is that mock is used not
only by OpenStack. Here's the result of an "apt-rdepends -r python-mock"
in Sid, today:

  Reverse Depends: atheist (0.20110402-2.1)
  Reverse Depends: python-cookiecutter (1.0.0-2)
  Reverse Depends: python-jsonschema (2.4.0-1)
  Reverse Depends: python-lamson (1.0pre11-1.1)
  Reverse Depends: python-matplotlib (1.4.2-3.1)
  Reverse Depends: python-mockldap (0.2.5-1)
  Reverse Depends: python-model-mommy (1.2-1)
  Reverse Depends: python-oslo.versionedobjects (0.1.1-2)
  Reverse Depends: python-oslotest (>= 1.5.1-1)
  Reverse Depends: python-responses (0.3.0-1)
  Reverse Depends: python-softlayer (4.0.4-1)
  Reverse Depends: python-vcr (1.6.1-1)

Clearly, we're not alone using mock. And we should always consider that
we aren't alone. So the usual "yeah, but we have pinned the versions, so
it's Debian's fault to have uploaded version 1.3 in Sid" would be very
naive in this case, and absolutely not valid. This is an ok-ish answer
for OpenStack only components like Oslo libraries. And even so, I'm
convince that we shouldn't break APIs there either.

So the issue here, really, is backward and forward compatibility
breakage in mock. Robert, you're a DD and you've been working for
Canonical, so you must know about these. You just need to care more for
this type of things. In the Linux kernel development space, they *never*
break userland as a rule. Why are Python developers allowing themselves
to do so?

Worse case if we really want to break things: isn't there ways to keep
the old API and write a new one, let everyone migrate, then eventually
deprecate the old one?

Anyway, the result is that mock 1.3 broke 9 packages at least in Kilo,
currently in Sid [1]. Maybe, as packages gets rebuilt, I'll get more bug
reports. This really, is a depressing situation. Now, as the package
maintainer for the failed packages, I have 4 solutions:

1/ Reassign these bugs to python-mock.
2/ Remove all of the unit tests which are currently failing because of
the new python-mock version. This isn't great, but as I already ran
these tests with mock 1.0.1, it should be ok.
3/ Completely remove unit tests for these Kilo packages (or at least
allow them to fail).
4/ See what's been done in Liberty to fix these tests with the newer
version of mock, and backport that to Kilo.

In the case of 1/, I don't think the python-mock package maintainer will
be able to do anything about it, and eventually, python-mock will get
AUTORM from Debian testing, which doesn't help me at all.

Unfortunately, 4/ isn't practical, because I'm also maintaining
backports to Jessie, which means I'd have to write fixes so that the
packages would work for both mock 1.0.1 and 1.3, plus it would take a
very large amount of my time in a non-useful way (I know the package
works as it passed unit tests with 1.0.1, so just fixing the tests is
useless).

So I'm left with either option 2/ and 3/. But really, I'd have preferred
if mock didn't break things... :/

Now, the most annoying one is with testtools (ie: #796542). I'd
appreciate having help on that one.

I hope the message is heard and that it wont happen again.

Cheers,

Thomas Goirand (zigo)

[1]

https://bugs.debian.org/795128 [src:python-barbicanclient]
python-barbicanclient: FTBFS: test_delete_checks_status_code:
AttributeError: assert_called

https://bugs.debian.org/795587 [src:python-heatclient]
python-heatclient: FTBFS: AttributeError: assert_called_once

https://bugs.debian.org/795588 [src:python-glance-store]
python-glance-store: FTBFS: AttributeError: assert_called_once

https://bugs.debian.org/795811 [src:barbican] barbican: FTBFS: 'self'
parameter lacking default value

https://bugs.debian.org/795875 [src:murano] murano: FTBFS:
AttributeError: assert_is_called

https://bugs.debian.org/795972 [src:python-oslotest] python-oslotest:
FTBFS: AttributeError: assert_any_calls

https://bugs.debian.org/796460 [src:python-glanceclient]
python-glanceclient: FTBFS: AttributeError: assert_called_once

https://bugs.debian.org/796463 [src:python-muranoclient]
python-muranoclient: FTBFS: AttributeError: assert_called_once

https://bugs.debian.org/796542 [src:python-testtools] python-testtools:
FTBFS: TypeError: 'NoneType' object is not callable