[openstack-dev] [MagnetoDB] Best practices for uploading large amounts of data

Maksym Iarmak miarmak at mirantis.com
Fri Mar 28 10:33:50 UTC 2014


2014-03-28 11:29 GMT+02:00 Serge Kovaleff <skovaleff at mirantis.com>:

> Hi Iliia,
>
> I would take a look into BSON http://bsonspec.org/
>
> Cheers,
> Serge Kovaleff
>
> On Thu, Mar 27, 2014 at 8:23 PM, Illia Khudoshyn <ikhudoshyn at mirantis.com>wrote:
>
>> Hi, Openstackers,
>>
>> I'm currently working on adding bulk data load functionality to
>> MagnetoDB. This functionality implies inserting huge amounts of data
>> (billions of rows, gigabytes of data). The data being uploaded is a set of
>> JSON's (for now). The question I'm interested in is a way of data
>> transportation. For now I do streaming HTTP POST request from the client
>> side with gevent.pywsgi on the server side.
>>
>> Could anybody suggest any (better?) approach for the transportation,
>> please?
>> What are best practices for that.
>>
>> Thanks in advance.
>>
>> --
>>
>> Best regards,
>>
>> Illia Khudoshyn,
>> Software Engineer, Mirantis, Inc.
>>
>>
>>
>> 38, Lenina ave. Kharkov, Ukraine
>>
>> www.mirantis.com <http://www.mirantis.ru/>
>>
>> www.mirantis.ru
>>
>>
>>
>> Skype: gluke_work
>>
>> ikhudoshyn at mirantis.com
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140328/a50d23de/attachment.html>


More information about the OpenStack-dev mailing list