Open Stack

On Wed, 2019-08-28 at 09:47 +0100, Chris Dent wrote:
> Requirements people:
> 
> For the past few months we've been doing profiling of placement,
> modeling large clouds where many results (7000 resource providers)
> will be returned in response to common requests.
> 
> One of the last remaining chunks of slowness is serializing the very
> large dicts representing those providers (and associated allocation
> requests) to JSON (greater than 100k lines when pretty printed).
> Using the oslo jsonutils (which is a light wrapper over the stdlib
> 'json' module) approximately 25% of time is consumed by dumps().
> 
> Using orjson [1] instead, time consumption drops to 1%, so this is
> something it would be good to make real in placement (and perhaps
> other projects that have large result sets).
if the 25% to 1% numbers are real. e.g. the cpu time does not move to
somewhere else instead then i would be very interested to see this
used in oslo.versioned object and/or the sdk.
>  There's a WIP that
> demonstrates its use at [2].
> 
> I'm posting about it because JSON serialization seems like it might
> be an area that the requirements team has greater interest than some
> other areas.
> 
> Also, orjson is only Python 3, so use needs to be adapted for Python
> 2, as shown in the WIP [2].
or wait untill octber/novemebr when we officly drop python 2 support
that said from a redhat point of view all our future openstack release form
stien will be on rhel8 which is python36 only by default(python2 is not install on rhel8).
so we will be shipping openstack on python 3 stating with stien.
> 
> Otherwise it's fine: active, Apache2 or MIT license.
> 
> Thoughts?
> 
> [1] https://pypi.org/project/orjson/
> [2] https://review.opendev.org/#/c/674661/
> 
> 
>