[openstack-dev] global-reqs on tooz pulls in worrisome transitive dep

Thomas Goirand zigo at debian.org
Thu Sep 11 07:17:18 UTC 2014


Hi Matt,

On 09/10/2014 04:30 AM, Matt Riedemann wrote:
> It took me a while to untangle this so prepare for links. :)
> 
> I noticed this change [1] today for global-requirements to require tooz
> [2] for a ceilometer blueprint [3].
> 
> The sad part is that tooz requires pymemcache [4] which is, from what I
> can tell, a memcached client that is not the same as python-memcached [5].
> 
> Note that python-memcached is listed in global-requirements already [6].
> 
> The problem I have with this is it doesn't appear that RHEL/Fedora
> package pymemcache (they do package python-memcached).  I see that
> openSUSE builds separate packages for each.  It looks like Ubuntu also
> has separate packages.
> 
> My question is, is this a problem?  I'm assuming RDO will just have to
> package python-pymemcache themselves but what about people not using RDO
> (SOL? Don't care? Other?).
> 
> Reverting the requirements change would probably mean reverting the
> ceilometer blueprint (or getting a version of tooz out that works with
> python-memcached which is probably too late for that right now).  Given
> the point in the schedule that seems pretty drastic.
> 
> Maybe I'm making more of this than it's worth but wanted to bring it up
> in case anyone else has concerns.
> [1] https://review.openstack.org/#/c/93443/
> [2] https://github.com/stackforge/tooz/blob/master/requirements.txt#L6
> [3]
>
http://specs.openstack.org/openstack/ceilometer-specs/specs/juno/central-agent-partitioning.html
>
> [4] https://pypi.python.org/pypi/pymemcache
> [5] https://pypi.python.org/pypi/python-memcached/
> [6]
>
https://github.com/openstack/requirements/blob/master/global-requirements.txt#L108
>
>

On my side (as the Debian package maintainer of OpenStack), I was more
than happy to see that Ceilometer made the choice to use a Python module
for memcache which supports Python 3. Currently python-memcache does
*not* support Python 3. It's in fact standing in the way to add Python 3
compatibility to *a lot* of the OpenStack packages, because this
directly impact python-keystoneclient, which is a (build-)dependency of
almost everything.

This situation has been very frustrating for me. I really would like
this to be solved. I see 2 ways to have it solved:
1- Complete the Python 3 support for python-memcache
or
2- Switch all OpenStack packages to pymemcache

There has been a few attempts with 1- already, with even a complete fork
of the project which you can see on both github and pypi. But it's a
real problem that it hasn't been upstreamed, and that the fork doesn't
support *both* Python 2 and 3.

I've tried to work out the port to Python 3 myself, but lamely failed.
It's not that easy, as one has to deal with unicode string from and to
the memcache server, and I have to admin I lack knowledge of how this
works. Also, the unit tests of python-memcache would have to be
re-written in a much better way than it is right now.

It would be really cool if someone could work on porting python-memcache
to Python 3, regardless if we decide to switch to pymemcache: that way,
we'd addressed the Python 3 compatibility quickly. If one wants to work
out a Python 3 compat in python-memcache, it'd be really great to have
it before Debian Jessie is frozen on the 5th of Nov, so that I get a
chance to package it. I asked several people, it seems that none really
wants to do it (is the code too ugly? probably...).

As for 2-, well, that would be a lot of work, I guess. I haven't
compared both APIs yet, but they should be different. And we have calls
to python-memcache in a lot of our projects. But this seems to be the
best way forward, as pymemcache seems to be written in a much better
way: smaller method, and the code is easier to understand., with more
extensive unit tests.

As for the issue with Redhat packaging, well, I'm sorry if that's a
problem for you, though really, pymemcache is a good choice, and I
support Julien on that one.

Cheers,

Thomas Goirand (zigo)

P.S: It's to be noted that "python-memcached" is called
"python-memcache" in Debian, because that's the name of the module when
doing imports...




More information about the OpenStack-dev mailing list