Open Stack

Clint, Morgan,

I totally agree that the pub/sub model is better approach.

However, there are 2 great things about polling:
1) it's simpler to use than pub/sub (especially in shell)
2) it has really simple implementation & we can get this in OpenStack in
few days/weeks

What about just supporting both approaches?


Best regards,
Boris Pavlovic

On Wed, Nov 4, 2015 at 9:33 AM, Morgan Fainberg <morgan.fainberg at gmail.com>
wrote:

>
> On Nov 3, 2015 4:29 PM, "Clint Byrum" <clint at fewbar.com> wrote:
> >
> > Excerpts from Boris Pavlovic's message of 2015-11-03 14:20:10 -0800:
> > > Hi stackers,
> > >
> > > Usually such projects like Heat, Tempest, Rally, Scalar, and other tool
> > > that works with OpenStack are working with resources (e.g. VM, Volumes,
> > > Images, ..) in the next way:
> > >
> > > >>> resource = api.resouce_do_some_stuff()
> > > >>> while api.resource_get(resource["uuid"]) != expected_status
> > > >>>    sleep(a_bit)
> > >
> > > For each async operation they are polling and call many times
> > > resource_get() which creates significant load on API and DB layers due
> the
> > > nature of this request. (Usually getting full information about
> resources
> > > produces SQL requests that contains multiple JOINs, e,g for nova vm
> it's 6
> > > joins).
> > >
> > > What if we add new API method that will just resturn resource status by
> > > UUID? Or even just extend get request with the new argument that
> returns
> > > only status?
> >
> > I like the idea of being able pass in the set of fields you want to
> > see with each get. In SQL, often times only passing in indexed fields
> > will allow a query to be entirely serviced by a brief range scan in
> > the B-tree. For instance, if you have an index on '(UUID, status)',
> > then this lookup will be a single read from an index in MySQL/MariaDB:
> >
> > SELECT status FROM instances WHERE UUID='foo';
> >
> > The explain on this will say 'Using index' and basically you'll just do
> > a range scan on the UUID portion, and only find one entry, which will
> > be lightning fast, and return only status since it already has it there
> > in the index. Maintaining the index is not free, but probably worth it
> > if your users really do poll this way a lot.
> >
> > That said, this is optimizing for polling, and I'm not a huge fan. I'd
> > much rather see a pub/sub model added to the API, so that users can
> > simply subscribe to changes in resources, and poll only when a very long
> > timeout has passed. This will reduce load on API services, databases,
>
> ++ this is a much better long term solution if we are investing
> engineering resources along these lines.
>
> > caches, etc. There was a thread some time ago about using Nova's built
> > in notifications to produce an Atom feed per-project. That seems like
> > a much more scalable model, as even polling just that super fast query
> > will still incur quite a bit more cost than a GET with If-Modified-Since
> > on a single xml file.
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151104/21334f0a/attachment.html>