Open Stack

On 04/01/2014 09:44 AM, John Dennis wrote:
> FWIW projects that deal with web services, wire protocols, external
> datastores, etc. who have already started porting to Py3 have
> encountered significant pain points with Py3, some of which is just
> being resolved and which have caused on-going changes in Py3. We deal
> with a lot of these same issues in OpenStack.

Oh, almost forgot. One of the significant issues in Py3 string handling
occurs when dealing with the underlying OS, specifically Posix, the
interaction with Posix "objects" such as pathnames, hostnames,
environment values, etc. Virtually any place where in C you would pass a
pointer to char in the Posix API where the intention is you're passing a
character string. Unfortunately Posix does not enforce the concept of a
character or a character string, the pointer to char ends up being a
pointer to octets (e.g. binary data) which means you can end up with
strings that can't be encoded.

Py3 has attempted to deal with this by introducing something called
"surrogate escapes" which attempts to preserve non-encodable binary data
in what is supposed to be a character string so as not to corrupt data
as it transitions between Py3 and a host OS.

OpenStack deals a lot with Posix API's, thus this is another area where
we need to be careful and have clear guidelines. We're going to have to
deal with the whole problem of encoding/decoding in the presence of
surrogates.


-- 
John