Open Stack

+1 for Zane's suggestion. Getting Mistral+Zaqar working together has many other side benefits that are great enough that it will help push both Mistral and Zaqar to be more commonly deployed.

It also may be able to solve the single keystone, multiple region case more easily....

If you have keystone sending only to one zaqar in only one region, you could still have Mistral in other regions accessing that zaqar to get the notifications, so long as the user had the choice of which region to talk to in their workflow. This would be much better then the rabbit solution which might require rabbit shovels or other cleverness to try and deal with the fact there might be multiple regions.

Thanks,
Kevin

________________________________________
From: Zane Bitter [zbitter at redhat.com]
Sent: Thursday, November 12, 2015 10:22 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

On 11/11/15 13:11, Clint Byrum wrote:
> Excerpts from Zane Bitter's message of 2015-11-11 09:43:43 -0800:
>> 1. Keystone (or some Rabbit->Zaqar proxy service reading notifications
>> from Keystone) sends "new federated user" notification out via Zaqar.
>> 2. Mistral picks up the message and checks policy to see what should be
>> done.
>> 3. Mistral calls either Heat or Keystone to autoprovision user.
>>
>
> Zane I like most of what you said here, and agree with nearly all of it.
> I actually started typing a question asking why Zaqar, but I think I
> understand, and you can correct me if I'm wrong.
>
> There's a notification bus. It is generally accessible to all of the
> things run by the operator if the operator wants it to be. Zaqar is for
> communication toward the user, whether from user hosted apps or operator
> hosted services. The thing we're discussiong seems entirely operator
> hosted, to operator hosted. Which to me, at first, meant we should just
> teach Mistral to listen to Keystone notifications and to run workflows
> using trusts acquired similarly to the way Heat acquires them.

Candidly, I'm using this as an opportunity to push my pet feature ;)

I genuinely think it could go either way in this case. What makes the
difference for me is that if we teach Mistral to read notifications from
Rabbit then only operators can use that feature, but if we teach Mistral
to read messages from a Zaqar queue then users can make use of this
feature too. We get more bang for our buck.

I think if this feature already existed, it would be an obvious
candidate for this use case. And since it should exist anyway, we should
just implement it instead of implementing another thing that isn't
really useful for any other purposes.

As a (theoretical) user I really, really want this. A couple of example
use cases:
- Heat sends a message and pauses for a response before updating or
deleting a server resource. The user can configure Mistral to run a
workflow to quiesce the server and send the response that allows Heat to
continue.
- Ceilometer sends a message in the event of an alarm. Instead of piping
it directly to Heat autoscaling, the user configures Mistral to listen
on the queue and do signal conditioning on the alarm.

Some of the requirements that fall out of this make Zaqar an obvious choice:
- Needs to be user-facing (Keystone auth)
- We MUST NOT lose the message...
- even if the listener is temporarily down

> However, it just ocurred to me that if we teach Mistral to read messages
> in a Zaqar queue belonging to a user, then there's no weirdness around
> user authentication and admin powers. Messages in a user's queue are
> entirely acted on using trusts for that user.

Yes, that is also a very nice property.

> That said, I think this is overly abstracted. I'd rather just see
> operator hosted services listen to the notification bus and react to the
> notifications they care about. You have to teach Mistral about trusts
> either way so it can do things as a user, and having the notification
> go an extra step:
>
> Keystone->[notifications]->Zaqar->Mistral
>
> vs.
>
> Keystone->[notifications]->Mistral
>
> Doesn't make a ton of sense to me.

I agree to some extent, I think I would prefer:

  Keystone->Zaqar->Mistral

Although, honestly, it's only Keystone's desire not to be tightly
coupled with this that would prevent them from calling Mistral directly.

However, I think:

   Keystone->[notifications]->Zaqar->Mistral

would make sense if:

- Keystone were ordinarily connected to the notification bus; and
- everyone were doing it (i.e. some service existed to routinely pick
notifications off the notification bus firehose, filter and sanitise
them for user consumption and redirect them to Zaqar)

Kevin indicates in another part of the thread that the former may not be
the case in a multi-region deployment, so that is one point against it.

The latter certainly isn't the case *yet*. But it has been discussed
quite a bit. (Most recently
https://etherpad.openstack.org/p/mitaka-horizon-async) A lot of things
need it (it would be really useful for Heat, for example). I wouldn't
say it's looking inevitable, but there's a good chance it may happen at
some point.

cheers,
Zane.

>> But as Renat mentioned, the part about triggering Mistral workflows from
>> a message does not yet exist. As Tim pointed out, Congress could be a
>> solution to that (listening for a message and then starting the Mistral
>> workflow). That may be OK in the short term, but in the long term I'd
>> prefer that we implement the triggering thing in Mistral (since there
>> are *lots* of end-user use cases for this too), and have the workflow
>> optionally query Congress for the policy rather than having Congress in
>> the loop.
>>
>
> I agree 100% on the positioning of Congress vs. Mistral here.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev