Open Stack

On 25/05/16 07:52, Zane Bitter wrote:
> On 22/05/16 22:38, Lingxian Kong wrote:
>> Hi, Zane, I think you must be interested in this:
>> https://review.openstack.org/#/c/308664/
>
> Oh! Yes that is very interesting. I'm glad that other people are
> thinking about these kinds of problems also.
>
> My blueprint has a different focus, in that it's more about allowing
> the _user_ to configure these actions. Listening to oslo_messaging
> notifications instead of to Zaqar messages limits you to operator use
> cases only. (It's also likely much more efficient - listening to a
> large number of queues simultaneously is not the use case Zaqar is
> optimised for, which is essentially why I changed my mind about
> whether this should be implemented within Mistral.)
>
> At least two projects that I know of (Ceilometer and Searchlight) are
> looking at implementing proxying of oslo_messaging notifications into
> Zaqar queues, and when that is combined with the Zaqar notification
> trigger we'll open up this functionality to everyone - other OpenStack
> services, operators and end-users.
>
> e.g. a user will be able to subscribe (via Ceilometer or Searchlight)
> to a notification about a particular server from Nova in a Zaqar queue
> and have that trigger a Mistral workflow that marks the server
> unhealthy in Heat and triggers a reconvergence of the stack.
>
> Unfortunately, there's always a tension between getting something done
> and in users' hands right now - which is easier to do if you stay
> within the confines of one project - and building flexible,
> orthogonal, user-pluggable abstractions, which are IMHO in the
> long-term best interests of users but generally require the
> co-operation, co-ordination and deployment of multiple projects.
>
Thanks for raising the topic and adding Ceilometer and Searchlight to
this thread. I think a similar topic about user notification was
discussed before. There are a lot of events/notifications in the infra's
message queues and tenant users are really interested in
them so that when there are some resources changed, users can be
notified instead of polling again and again. Based on my investigation
last time, I would like to see a driver at here
https://github.com/openstack/ceilometer/tree/master/ceilometer/publisher
Julien, can you comment this?
> It seems like we, the OpenStack community, are failing not only at
> co-ordinating these kinds of features across projects, but in even
> know what is being planned in other projects. Perhaps we need some
> sort of "Autonomous Applications working group"?
+1 That's a good idea.
>
> cheers,
> Zane.
>
>>
>> Regards!
>> -----------------------------------
>> Lingxian Kong
>>
>>
>> On Fri, May 20, 2016 at 9:49 AM, Zane Bitter <zbitter at redhat.com> wrote:
>>> On 19/05/16 04:20, Thomas Herve wrote:
>>>>
>>>> On Wed, May 18, 2016 at 8:49 PM, Zane Bitter <zbitter at redhat.com>
>>>> wrote:
>>>>>
>>>>> I've been lobbying the Mistral developers for $SUBJECT since,
>>>>> basically,
>>>>> forever.[1][2][3] I like to think after a couple of years I
>>>>> succeeded in
>>>>> changing their view on it from "crazy" to merely "unrealistic".[4]
>>>>> In the
>>>>> last few months I've had a couple of realisations though:
>>>>>
>>>>> 1) The 'pull' model I've been suggesting is the wrong one,
>>>>> architecturally
>>>>> speaking. It's asking Mistral to do too much to poll Zaqar queues.
>>>>> 2) A 'push' model is the correct architecture and it already
>>>>> exists in
>>>>> the
>>>>> form of Zaqar's Notifications, which suddenly makes this goal look
>>>>> very
>>>>> realistic indeed.
>>>>>
>>>>> I've posted a Zaqar spec for this here:
>>>>>
>>>>> https://review.openstack.org/#/c/318202/
>>>>
>>>>
>>>> Commented. I certainly need some more time to think about it.
>>>
>>>
>>> Thanks, I updated the spec based on your comments.
>>>
>>>>> Not being super familiar with either project myself, I think this
>>>>> needs
>>>>> close scrutiny from Mistral developers as well as Zaqar developers to
>>>>> make
>>>>> sure I haven't got any of the details wrong. I'd also welcome any
>>>>> volunteers
>>>>> interested in implementing this :)
>>>>>
>>>>>
>>>>> One more long-term thing that I did *not* mention in the spec:
>>>>> there are
>>>>> both Zaqar notifications and Mistral actions for sending email and
>>>>> hitting
>>>>> webhooks. These are two of the hardest things for a cloud operator to
>>>>> secure. It would be highly advantageous if there were only _one_
>>>>> place in
>>>>> OpenStack where these were implemented. Either project would
>>>>> potentially
>>>>> work - Zaqar notifications could call a simple, operator defined
>>>>> workflow
>>>>> behind the scenes for email/webhook notifications; alternatively the
>>>>> Mistral
>>>>> email/webhook actions could drop a message on a Zaqar queue
>>>>> connected to
>>>>> a
>>>>> notification - although the former sounds easier to me. (And of
>>>>> course
>>>>> clouds with only one of the services available could fall back to the
>>>>> current plugins.) Something to think about for the future...
>>>>
>>>>
>>>> And you're forgetting about Aodh alarm notifications, which are
>>>> webhooks as well.
>>>
>>>
>>> Yeah, those just need to go away :) Aodh can already post the
>>> notifications
>>> to Zaqar instead, and hopefully we'll be able to migrate everything
>>> to that
>>> method over time.
>>>
>>>> Unfortunately, I think none of them do a
>>>> particularly great job in terms of robustness and security.
>>>
>>>
>>> Agree. The best you can do is still middling, and we're not there yet.
>>>
>>>> I
>>>> suggested moving away from doing webhook in Zaqar to Flavio in the
>>>> past, and I think he responded that he thought it was part of the core
>>>> functionality. It's hard to delegate such operations and create a
>>>> dependency on another project. Maybe we can start by creating some
>>>> library code.
>>>
>>>
>>> I guess that's a start, but if I were running a large public cloud I'd
>>> probably want to isolate this on its own network surrounded by some
>>> pretty
>>> custom firewalls. A library doesn't really help make that easier.
>>>
>>> cheers,
>>> Zane.
>>>
>>>
>>> __________________________________________________________________________
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __________________________________________________________________________
>
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Cheers & Best regards,
Fei Long Wang (王飞龙)
--------------------------------------------------------------------------
Senior Cloud Software Engineer
Tel: +64-48032246
Email: flwang at catalyst.net.nz
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
--------------------------------------------------------------------------