Open Stack

On 07/09/2014 05:31 PM, Julien Danjou wrote:
> On Wed, Jul 09 2014, Thomas Goirand wrote:
> 
>> The thing is, having a lack of Python 3 support in python-memcached
>> prevents from having python-keystoneclient to support Python 3, and as a
>> consequence, about everything else in OpenStack.
> 
> The main problem I see here is that python-memcached has no unit tests,
> so trying to port it to Python 3 is just going to be painful…
> 
> For future reference, I've switched to
> https://github.com/pinterest/pymemcache for memcache under Python.
> Better code, API, have unit tests and supports Python 3. I'd encourage
> to migrate to this implementation.
> 
> Now that doesn't help you Thomas.

Indeed.

> But if I had to spend time and energy
> helping you, that's more likely what I'd like to propose as a solution
> rather than porting python-memcached to Python 3.

There's not only python-keystoneclient, but also nova, keystone,
python-dogpile.cache and swift are using python-memcached. So we'd have
to fix them all. If someone wants to work on that, rather than fixing
python-memcached, I'm fine with it. But I wonder if just fixing the 1500
line memcache.py of python-memcached wouldn't be more efficient.

> Do you know what tests fail exactly with keystoneclient? I'd imagine
> it's something from oslo-incubator, I'm not sure keystoneclient uses
> memcache directly.

It's really in the unit tests of keystoneclient. See
keystoneclient/tests/test_auth_token_middleware.py around line 120. It
checks if python-memecached is installed. But truth, there's also some
stuff in keystoneclient/openstack/common/memorycache.py, so it's also in
oslo-incubator.

> Maybe I can try to port that?

Sorry, I forgot to past it. What fails is importing
keystoneclient.tests.test_auth_token_middleware in testr. You can
reproduce it by starting the Python 3.4 interpreter, and doing:

import keystoneclient.tests.test_auth_token_middleware

Here's the stack dump:

======================================================================
FAIL:
unittest.loader.ModuleImportFailure.keystoneclient.tests.test_auth_token_middleware
unittest.loader.ModuleImportFailure.keystoneclient.tests.test_auth_token_middleware
----------------------------------------------------------------------
_StringException: Traceback (most recent call last):
ImportError: Failed to import test module:
keystoneclient.tests.test_auth_token_middleware
Traceback (most recent call last):
  File "/usr/lib/python3.4/unittest/loader.py", line 312, in _find_tests
    module = self._get_module_from_name(name)
  File "/usr/lib/python3.4/unittest/loader.py", line 290, in
_get_module_from_name
    __import__(name)
  File
"/home/zigo/sources/openstack/icehouse/python-keystoneclient/build-area/python-keystoneclient-0.9.0/keystoneclient/tests/test_auth_token_middleware.py",
line 425, in
    testresources.ResourcedTestCase):
  File
"/home/zigo/sources/openstack/icehouse/python-keystoneclient/build-area/python-keystoneclient-0.9.0/keystoneclient/tests/test_auth_token_middleware.py",
line 460, in
    @testtools.skipUnless(memcached_available(), 'memcached not available')
  File
"/home/zigo/sources/openstack/icehouse/python-keystoneclient/build-area/python-keystoneclient-0.9.0/keystoneclient/tests/test_auth_token_middleware.py",
line 120, in
    c.set('ping', 'pong', time=1)
  File "/usr/lib/python3/dist-packages/memcache.py", line 651, in set
    return self._set("set", key, val, time, min_compress_len)
  File "/usr/lib/python3/dist-packages/memcache.py", line 929, in _set
    return _unsafe_set()
  File "/usr/lib/python3/dist-packages/memcache.py", line 919, in
_unsafe_set
    server.send_cmd(fullcmd)
  File "/usr/lib/python3/dist-packages/memcache.py", line 1289, in send_cmd
    self.socket.sendall(cmd + '\r\n')
TypeError: 'str' does not support the buffer interface