[openstack-dev] [MagnetoDB] Bulk load API draft

Ilya Sviridov isviridov at mirantis.com
Wed May 28 10:03:22 UTC 2014


Hello Illia,

Great job!

Several moments about spec itself.

"read": "string",
"processed": "string",
"failed": "string",

I believe all of them are integers, but not strings.

About Dmitri's suggestion to rename "errors", I would also prefer to do it
as far as error itself goes below with item.
What do you think about  "unprocessed_items"?

And I think we can go with it.

Ilya



On Wed, May 28, 2014 at 10:28 AM, Illia Khudoshyn
<ikhudoshyn at mirantis.com>wrote:

> Hi Dima,
>
> Sounds good, thank you for the point.
>
>
> On Tue, May 27, 2014 at 7:34 PM, Dmitriy Ukhlov <dukhlov at mirantis.com>wrote:
>
>> Hi Illia,
>>
>> Looks good. But I suggest to return all of these fields for positive
>> request as well as for error request:
>>
>>     "read": "string",
>>     "processed": "string",
>>     "failed": "string",
>>
>> but leave next fields optional and fill them in case of error response
>> ("failed" > 0) to specify what exactly was happened:
>>
>> "last_read":
>> "errors" (maybe not processed will be better)
>>
>>
>>
>> On Tue, May 27, 2014 at 3:39 PM, Illia Khudoshyn <ikhudoshyn at mirantis.com
>> > wrote:
>>
>>> Hi openstackers,
>>>
>>> While working on bulk load, I found previously proposed batch-oriented
>>> asynchronous approach both resource consuming on server side and somewhat
>>> complicated to use.
>>> So I tried to outline some more straightforward streaming way of
>>> uploading data.
>>>
>>> By the link below you can found a draft for a new streaming API
>>> https://wiki.openstack.org/wiki/MagnetoDB/streamingbulkload.
>>>
>>> Any feedback is welcome as usual.
>>>
>>>
>>>
>>> On Wed, May 14, 2014 at 5:04 PM, Illia Khudoshyn <
>>> ikhudoshyn at mirantis.com> wrote:
>>>
>>>> Hi openstackers,
>>>>
>>>> I'm working on bulk load for MagnetoDB, the facility for inserting
>>>> large amounts of data, like,  millions of rows, gigabytes of data. Below is
>>>> the link to draft API description.
>>>>
>>>>
>>>> https://wiki.openstack.org/wiki/MagnetoDB/bulkload#.5BDraft.5D_MagnetoDB_Bulk_Load_workflow_and_API
>>>>
>>>> Any feedback is welcome.
>>>>
>>>> --
>>>>
>>>> Best regards,
>>>>
>>>> Illia Khudoshyn,
>>>> Software Engineer, Mirantis, Inc.
>>>>
>>>>
>>>>
>>>> 38, Lenina ave. Kharkov, Ukraine
>>>>
>>>> www.mirantis.com <http://www.mirantis.ru/>
>>>>
>>>> www.mirantis.ru
>>>>
>>>>
>>>>
>>>> Skype: gluke_work
>>>>
>>>> ikhudoshyn at mirantis.com
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Best regards,
>>>
>>> Illia Khudoshyn,
>>> Software Engineer, Mirantis, Inc.
>>>
>>>
>>>
>>> 38, Lenina ave. Kharkov, Ukraine
>>>
>>> www.mirantis.com <http://www.mirantis.ru/>
>>>
>>> www.mirantis.ru
>>>
>>>
>>>
>>> Skype: gluke_work
>>>
>>> ikhudoshyn at mirantis.com
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Dmitriy Ukhlov
>> Mirantis Inc.
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
>
> Best regards,
>
> Illia Khudoshyn,
> Software Engineer, Mirantis, Inc.
>
>
>
> 38, Lenina ave. Kharkov, Ukraine
>
> www.mirantis.com <http://www.mirantis.ru/>
>
> www.mirantis.ru
>
>
>
> Skype: gluke_work
>
> ikhudoshyn at mirantis.com
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140528/5550a86b/attachment.html>


More information about the OpenStack-dev mailing list