Open Stack

On Wed, Feb 05, 2020 at 10:46:30AM -0600, Matthew Thode wrote:
> On 20-02-05 10:17:18, Witek Bedyk wrote:
> > Hi,
> > 
> > we're using ujson in Monasca for serialization all over the place. While
> > changing it to any other alternative is probably a drop-in replacement, we
> > have in the past chosen to use ujson because of better performance. It is of
> > great importance, in particular in persister. Current alternatives include
> > orjson [1] and rapidjson [2]. We're going to measure which of them works
> > best for our use case and how much faster they are compared to standard
> > library module.
> > 
> > Assuming there is a significant performance benefit, is there any preference
> > from requirements team which one to include in global requirements? I
> > haven't seen any distro packages for any of them.
> > 
> > [1] https://pypi.org/project/orjson/
> > [2] https://pypi.org/project/python-rapidjson/
> > 
> > Best greetings
> > Witek
> > 
> > 
> > On 1/31/20 9:34 AM, Radosław Piliszek wrote:
> > > This is a spinoff discussion of [1] to attract more people.
> > > 
> > > As the subject goes, the situation of ujson is bad. Still, monasca and
> > > gnocchi (both server and client) seem to be using it which may break
> > > depending on compiler.
> > > The original issue is that the released version of ujson is in
> > > non-spec-conforming C which may break randomly based on used compiler
> > > and linker.
> > > There has been no release of ujson for more than 4 years.
> > > 
> > > Based on general project activity, Monasca is probably able to fix it
> > > but Gnocchi not so surely...
> > > 
> > > [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-January/thread.html
> > > 
> > > -yoctozepto
> > > 
> > 
> 
> orjson supports CPython 3.6, 3.7, 3.8, and 3.9. It distributes wheels
> for Linux, macOS, and Windows. The manylinux1 wheel differs from PEP 513
> in requiring glibc 2.18, released 2013, or later. orjson does not
> support PyPy.
> 
> Given the above (I think we still need to support py35 at least) I'm not
> sure we can use it.  Though it is my preferred other than that...
> (faster than ujson, more updates (last release yesterday), etc)

It's also probably worth looking at the thread on this from August [1]
discussing potentially using orjson. While they've added sdists since that
original discussion (because of the pyo3-pack support being added) building
it locally requires having rust nightly installed. This means for anyone on
a non-x86_64 platform (including i686) will need to have rust nightly
installed to pip install a package. Not that it's a super big burden, rustup
makes it pretty easy to do, but it's a pretty uncommon thing for most people.
But, I think that combined with no py35 support probably makes it a difficult
thing to add to g-r. 

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008849.html

-Matthew Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200205/bec70679/attachment.sig>