Open Stack

On Mon, Jan 12, 2015 at 3:33 PM, Angus Salkeld <asalkeld at mirantis.com>
wrote:

>
> On Mon, Jan 12, 2015 at 10:17 PM, Konstantin Danilov <
> kdanilov at mirantis.com> wrote:
>
>> Boris,
>>
>> Move from sync http to something like websocket requires a lot of work
>> and not directly connected
>> with API issue. When openstack api servers begin to support
>> websockets - it would be easy to change implementation of monitoring
>> thread without breaking compatibility.
>> At the moment periodical pooling from additional thread looks reasonable
>> for me
>> and it creates same amount of http requests as all current
>> implementation.
>>
>> BP is not about improving performance,
>> but about providing convenient and common API to handle background tasks.
>>
>> > So we won't need to retrieve 100500 times information about object.
>> As I sad before - this API creates same amount of load as any code which
>> we use to check background task currently.
>> It even can decrease load due to requests aggregation in some cases (but
>> there a points to discuss).
>>
>> > As well this pattern doesn't look great.
>> > I would prefer to see something like:
>> > vm = novaclient.servers.create(...., sync=True)
>>
>> This is completely different pattern. It is blocking call, which don't
>> allows you to start two(or more) background tasks
>> and from same thread and make some calculations while they running in
>> background.
>>
>
> Except if you use threads (eventlet or other) - I am still struggling to
> enjoy Futures/yield based flowcontrol, lost battle i guess:(.
>

>>
>>
>>
>>
>> On Mon, Jan 12, 2015 at 1:42 PM, Boris Pavlovic <bpavlovic at mirantis.com>
>> wrote:
>>
>>> Konstantin,
>>>
>>>
>>> I believe it's better to work on server side, and use some modern
>>> approach like web sockets for async operations. So we won't need to
>>> retrieve 100500 times information about object. And then use this feature
>>> in clients.
>>>
>>>  create_future = novaclient.servers.create_async(....)
>>>>     .....
>>>>     vm = create_future.result()
>>>
>>>
>>> As well this pattern doesn't look great.
>>>
>>> I would prefer to see something like:
>>>
>>>   vm = novaclient.servers.create(...., sync=True)
>>>
>>>
>>> Best regards,
>>> Boris Pavlovic
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Jan 12, 2015 at 2:30 PM, Konstantin Danilov <
>>> kdanilov at mirantis.com> wrote:
>>>
>>>> Hi all.
>>>>
>>>> There a set of openstack api functions which starts background actions
>>>> and return preliminary results - like 'novaclient.create'. Those
>>>> functions
>>>> requires periodically check results and handle timeouts/errors
>>>> (and often cleanup + restart help to fix an error).
>>>>
>>>> Check/retry/cleanup code duplicated over a lot of core projects.
>>>> As examples - heat, tempest, rally, etc and definitely in many
>>>> third-party scripts.
>>>>
>>>
> We have some very similar code at the moment, but we are keen to move away
> from it to
> something like making use of rpc ".{start,end}" notifications to reduce
> the load we put on keystone and
> friends.
>

This is nice approach for core projects, yet novaclient users are typically
can't use such approach.

But the nice things about futures is that we can have different engines
(websockets, sync http, rpc with callbacks, etc)
behind same API, and hide all implementation details behind it.  It's even
possible to use them in simultaneously -
different engine would used to handle different calls.


>
>
>>
>>>> I propose to provide common higth-level API for such functions, which
>>>> uses
>>>> 'futures' (http://en.wikipedia.org/wiki/Futures_and_promises) as a way
>>>> to
>>>> present background task.
>>>>
>>>> Idea is to add to each background-task-starter function a complimentary
>>>> call,
>>>> that returns 'future' object. E.g.
>>>>
>>>>     create_future = novaclient.servers.create_async(....)
>>>>     .....
>>>>     vm = create_future.result()
>>>>
>>>
> Is that going to return on any state change or do you pass in a list of
> acceptable states?
>

In general it should returns result if background task is completed
successfully or raise exception
if it fails. For server I currently use 'active' as success marker and
'error' or timeout for exception
( https://github.com/koder-ua/os_api/blob/master/os_api/nova.py#L74 ), and
hope that expected states
can be calculated from api call/parameters automatically, but not 100%
sure. So - yes, additional
parameter might be required.


>
> -Angus
>
>
>>
>>>> This allows to unify(and optimize) monitoring cycles, retries, etc.
>>>> Please found complete BP at
>>>> https://github.com/koder-ua/os_api/blob/master/README.md
>>>>
>>>> Thanks
>>>> --
>>>> Kostiantyn Danilov aka koder <http://koder.ua>
>>>> Principal software engineer, Mirantis
>>>>
>>>> skype:koder.ua
>>>> http://koder-ua.blogspot.com/
>>>> http://mirantis.com
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Kostiantyn Danilov aka koder.ua
>> Principal software engineer, Mirantis
>>
>> skype:koder.ua
>> http://koder-ua.blogspot.com/
>> http://mirantis.com
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kostiantyn Danilov aka koder.ua
Principal software engineer, Mirantis

skype:koder.ua
http://koder-ua.blogspot.com/
http://mirantis.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150112/660124d4/attachment.html>