[openstack-dev] [MagnetoDB] Best practices for uploading large amounts of data

Serge Kovaleff skovaleff at mirantis.com
Fri Mar 28 09:29:50 UTC 2014


Hi Iliia,

I would take a look into BSON http://bsonspec.org/

Cheers,
Serge Kovaleff

On Thu, Mar 27, 2014 at 8:23 PM, Illia Khudoshyn <ikhudoshyn at mirantis.com>wrote:

> Hi, Openstackers,
>
> I'm currently working on adding bulk data load functionality to MagnetoDB.
> This functionality implies inserting huge amounts of data (billions of
> rows, gigabytes of data). The data being uploaded is a set of JSON's (for
> now). The question I'm interested in is a way of data transportation. For
> now I do streaming HTTP POST request from the client side with
> gevent.pywsgi on the server side.
>
> Could anybody suggest any (better?) approach for the transportation,
> please?
> What are best practices for that.
>
> Thanks in advance.
>
> --
>
> Best regards,
>
> Illia Khudoshyn,
> Software Engineer, Mirantis, Inc.
>
>
>
> 38, Lenina ave. Kharkov, Ukraine
>
> www.mirantis.com <http://www.mirantis.ru/>
>
> www.mirantis.ru
>
>
>
> Skype: gluke_work
>
> ikhudoshyn at mirantis.com
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140328/31a42b66/attachment.html>


More information about the OpenStack-dev mailing list