Open Stack

Excerpts from Joe Gordon's message of 2015-04-27 10:36:57 -0700:
> On Fri, Apr 24, 2015 at 5:05 PM, Zane Bitter <zbitter at redhat.com> wrote:
> 
> > On 24/04/15 19:02, Joe Gordon wrote:
> >
> >>
> >>
> >> On Mon, Apr 20, 2015 at 5:54 AM, Flavio Percoco <flavio at redhat.com
> >> <mailto:flavio at redhat.com>> wrote:
> >>
> >>     Greetings,
> >>
> >>     I'd like my first action as Zaqar's PTL to be based on reflections and
> >>     transparency with regards to what our past has been, to what our
> >>     present is and to what our future could be as a project and community.
> >>     Therefore, I'm sending this call for adoption and support before
> >>     taking other actions (also mentioned below).
> >>
> >>     The summit is very close and the Zaqar team is looking forward to it.
> >>
> >>     The upcoming summit represents an important opportunity for Zaqar to
> >>     integrate with other projects. In the previous summits - since
> >>
> >>
> >> I get integration with Horizon etc. But to use the SQS/SNS analogy how
> >> would say Nova integrate with Zaqar?
> >>
> >
> > Speaking very generally, anything where it makes sense for Nova to tell
> > the user - or, more importantly, the application - when something is
> > happening. The cloud can't afford to be making synchronous calls to the
> > client-side, and applications may not be able to afford missing the
> > notifications, so a reliable, asynchronous transport like Zaqar is a good
> > candidate.
> >
> > So examples might be:
> >  - Hey, your resize is done
> >  - Hey, your [re]build is done
> >  - Hey, your VM rebooted
> >  - Hey, your VM disappeared
> >
> > Now, this is not to presuppose that having Nova put messages directly into
> > Zaqar is the correct design. It may be that it's better to have some other
> > service (Ceilometer?) collect some or all of those notifications and handle
> > putting them into Zaqar (though the reliability would be a concern).
> > Certainly EC2 seems to funnel all this stuff to CloudWatch, although other
> > services like S3, CloudFormation & Auto Scaling deliver notifications to
> > SNS directly. There is some integration work either way though, to produce
> > the notification.
> >
> > Obviously there's less integration to do for a project like Nova that only
> > produces notifications than there would be for those that could actually
> > consume notifications. Heat would certainly like to use these notifications
> > to reduce the amount of polling we do of the APIs (and ditch it altogether
> > if reliability is guaranteed). But if we can get both ends integrated then
> > the *user* can start doing really interesting things like:
> >
> 
> This is one of the bigger questions for me, as a nova developer. What would
> integration look like from Nova's POV.  I am a little weary of adding yet
> another API when we have so much trouble with our existing ones.
> Especially the notification based API.
> 

It seems to me like there's no need for Nova to do anything except keep
sending notifications. The service that is needed is the one that
exposes appropriate notifications to end-users, and it would have an API
which would work something like


POST /subscriptions
Auth-Things: go-here

{ "topic": "compute",
  "subject": [{"id": "12346566789"}],
  "destination": {"driver": "zaqar", "args": {"queue-id": "id-of-zaqar-inbox"}} }

Domain/Tenant is implied from auth details, and off to the races,
a filter is added and some notification-bus-aware workers receive the
fanout of notifications, passing applicable messages into the configured
driver. Drivers could be zaqar, or logstash, or SMTP inboxes, or whatever.
Point being, Nova wouldn't care, as long as its notifications can be
transformed into the output of a driver that the cloud operator has made
available. This would solve Heat's use case, and would probably be useful
for infra's nodepool if it were on the clouds we use since we could wait
for messages instead of polling the server list.

I've heard tell this might be something that could be built on top of,
or split out from, ceilometer, since it already sort of has to do this
to facilitate billing. Anyway, bottom line, if there aren't any users
willing to put resources into building it, then this has probably just
been a fun exercise in crystal ball gazing for now.