[gate][keystone][nova][neutron] *-grenade-multinode jobs failing with UnicodeDecodeError in keystone
melwittt at gmail.com
Wed Aug 12 07:49:10 UTC 2020
Adding [nova][neutron] since their gates will continue to be blocked
until one of the following proposed fixes merges. They are linked inline.
On 8/11/20 19:57, melanie witt wrote:
> On 8/11/20 14:53, melanie witt wrote:
>> Howdy all,
>> FYI the *-grenade-multinode gate jobs are currently failing with the
>> following error in keystone:
>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in
>> position 3: invalid start byte
>> This appears to be an issue with a new default data format in msgpack
>> v1.0  which was brought in by a recent bump of upper constraints .
>> *-grenade-multinode jobs are affected because they test a rolling
>> upgrade where the controller is upgraded to the N release version but
>> one compute node is on the N-1 release version. It looks like cached
>> keystone tokens being used by the N-1 node are erroring out during
>> msgpack unpacking because they are in the old data format and msgpack
>> v1.0 has a new default data format.
>> I've opened a bug  about and I'm trying out the following keystone
>> patch to fix it:
>> Reviews appreciated.
I tested ^ with a DNM patch to nova and nova-grenade-multinode passes
>> If this is not the best approach or if this affects other projects as
>> well, alternatively we could revert the upper constraint bump to
>> msgpack v1.0 while we figure out the best fix.
> Here's a patch for reverting the upper constraint for msgpack in case
> that approach is preferred:
And this reqs pin ^ is also available if the reviewers find the keystone
>>  https://review.opendev.org/#/c/745437/2/upper-constraints.txt@373
>>  https://launchpad.net/bugs/1891244
More information about the openstack-discuss