Open Stack

> DIms,
>>
>> Use case could be anything, which would needed by either
>> operator/community, who wants to perform an required task before and
after
>> service is started. This requirement is very generic by nature, and I
>> believe it will be very useful.
>>
>> Would like to give the sample use cases from from Operator & OpenStack
>> community side as below.
>> Operator side, any pre/post actions could be hooked which is the
>> requirement for them. Simple example would be, one who wants to create an
>> journal of start/stop details like time, number of worker,
configurations,
>> etc in a common portal, this life-cycle hook would help.

> Is that information not available in the logs?

[Kanagaraj M] its available in log, but one who wants to collect these info
in centralized portal,
it would help.

>
> OpenStack community side, sample use cases would be:
> 1. Most of the OpenStack components starts TextGuruMeditation, logging
> while those components are get started. These tasks could be provided as
> life cycle hooks and all OpenStack components could start to leverage it.

>All of those are things that need to be built into the app in a way that
>they are started at the right time, rather than at an arbitrary point
>defined by the plugin order a deployer has specified.

[Kanagaraj M] while i investigated the OpenStack service cmd, mostly
it follows the similar pattern on use these utils, so i thought, it would be
good to provide an plugin, which take care of it instead every service
code does take care of it. helps to reduces maintenance effort.

>> 2. For automatically discovering the OpenStack deployment, this hooks
will
>> be very useful. Auto-discover-hook would report to pre-defined
destinations
>> while starting/stopping the service.

>Doing that usefully is going to require passing information to the hook
>so it knows where it is running (what service, what port, etc.). None of
>the APIs for doing this have been described yet. Do you have any plans
>put together?

[Kanagaraj M] I am trying to get all of these information from oslo.confg
global config variable. As we discussed about namos during austin summit,
namos does collect these details
https://github.com/openstack/os-namos/blob/master/os_namos/sync.py#L124

>It feels very much like you're advocating that we create a thing like
>paste-deploy for non-WSGI apps: allow anyone to insert anything into
>the executing process for any purpose and with little control on the
>application authors' part. That introduces potential stability issues,
>and a lot of questions that haven't been answered. For example:

>Does service startup block until all of the plugins are done? If not,
>do we need to have a timeout management system built in or do we run
>the plugins in their own thread/subprocess so they can run in the
>background?

>Can a plugin change the execution of the service in some way (you
>mentioned having a plugin download config files when we spoke at the
>summit, is that still something you want to slip in this way instead of
>putting it into oslo.config)?

>Can a plugin cause the service to not start at all by exiting?

>What happens if one plugin fails but others succeed? Do we keep running?

>What information about the service can a plugin see? It's running in the
>same process, so it could see the configuration object, for example.
>It would only be able to see configuration options it adds (yes, that
>would work) or that were registered by the application before the plugin
>is run. So, not necessarily all options but potentially many, with more
>and more as apps shift to pre-registering all of their options in one
>place. Assuming these are things the deployer has selectively installed,
>maybe that's OK. OTOH, it does open another security surface.

>What happens when a service is told to restart its workers? Do the
>plugins run again?

>Can a plugin start listening for network connections on its own? Connect
>to the message bus?  Provide an RPC endpoint? Start processes? Threads?

[KanagarajM] It gives me a lots insight on what are different problems
would come and i really thank you.
Hooks will be provided by community and/or deployers. while community
provides, those hooks will be well documented, tested and configuration
would be
provided. so all of the above mentioned aspects would be taken care well by
community based on the hooks functionality. And deployer also would take
 care of similar safety measurement before using their hooks similar to how
would they take care when using startup scripts.



>> Regards
>> Kanagaraj M
>>
>> On Wed, May 11, 2016 at 7:05 PM, Davanum Srinivas <davanum at gmail.com>
wrote:
>>
>> > Kanagaraj,
> >
>> > Who is the first consumer? for what specific purpose?
>> >
>> > Thanks,
>> > Dims
>> >
>> > On Wed, May 11, 2016 at 9:27 AM, Kanagaraj Manickam <mkr1481 at gmail.com>
>> > wrote:
>> > > Hi,
>> > >
>> > > When OpenStack service components are started/stooped,
>> > > operators or OpenStack Services want to execute some actives
>> > > before and/or after component is started/stopped.
>> > > Most of the time, operator needs to depends
>> > > on the start-up scripts to do it, which is an installer
>> > > dependent, while OpenStack service can't use this approach.

>Can you elaborate on this? I think you're saying that having a start-up
>script take action before a service is started won't work because it
>would require the operator to customize that script. Is that right?
>Aren't there already tools for doing this, though?

[KanagarajM] when we use start-up script, we can't do any pre/post
actions in every worker process, instead we can only do pre/post once
before/after launcher and all worker process started.


Regards
Kanagaraj M

On Fri, May 13, 2016 at 11:38 PM, Doug Hellmann <doug at doughellmann.com>
wrote:

> Excerpts from Kanagaraj Manickam's message of 2016-05-13 14:46:06 +0530:
> > DIms,
> >
> > Use case could be anything, which would needed by either
> > operator/community, who wants to perform an required task before and
> after
> > service is started. This requirement is very generic by nature, and I
> > believe it will be very useful.
> >
> > Would like to give the sample use cases from from Operator & OpenStack
> > community side as below.
> > Operator side, any pre/post actions could be hooked which is the
> > requirement for them. Simple example would be, one who wants to create an
> > journal of start/stop details like time, number of worker,
> configurations,
> > etc in a common portal, this life-cycle hook would help.
>
> Is that information not available in the logs?
>
> >
> > OpenStack community side, sample use cases would be:
> > 1. Most of the OpenStack components starts TextGuruMeditation, logging
> > while those components are get started. These tasks could be provided as
> > life cycle hooks and all OpenStack components could start to leverage it.
>
> All of those are things that need to be built into the app in a way that
> they are started at the right time, rather than at an arbitrary point
> defined by the plugin order a deployer has specified.
>
> > 2. For automatically discovering the OpenStack deployment, this hooks
> will
> > be very useful. Auto-discover-hook would report to pre-defined
> destinations
> > while starting/stopping the service.
>
> Doing that usefully is going to require passing information to the hook
> so it knows where it is running (what service, what port, etc.). None of
> the APIs for doing this have been described yet. Do you have any plans
> put together?
>
> It feels very much like you're advocating that we create a thing like
> paste-deploy for non-WSGI apps: allow anyone to insert anything into
> the executing process for any purpose and with little control on the
> application authors' part. That introduces potential stability issues,
> and a lot of questions that haven't been answered. For example:
>
> Does service startup block until all of the plugins are done? If not,
> do we need to have a timeout management system built in or do we run
> the plugins in their own thread/subprocess so they can run in the
> background?
>
> Can a plugin change the execution of the service in some way (you
> mentioned having a plugin download config files when we spoke at the
> summit, is that still something you want to slip in this way instead of
> putting it into oslo.config)?
>
> Can a plugin cause the service to not start at all by exiting?
>
> What happens if one plugin fails but others succeed? Do we keep running?
>
> What information about the service can a plugin see? It's running in the
> same process, so it could see the configuration object, for example.
> It would only be able to see configuration options it adds (yes, that
> would work) or that were registered by the application before the plugin
> is run. So, not necessarily all options but potentially many, with more
> and more as apps shift to pre-registering all of their options in one
> place. Assuming these are things the deployer has selectively installed,
> maybe that's OK. OTOH, it does open another security surface.
>
> What happens when a service is told to restart its workers? Do the
> plugins run again?
>
> Can a plugin start listening for network connections on its own? Connect
> to the message bus?  Provide an RPC endpoint? Start processes? Threads?
>
>
> > Regards
> > Kanagaraj M
> >
> > On Wed, May 11, 2016 at 7:05 PM, Davanum Srinivas <davanum at gmail.com>
> wrote:
> >
> > > Kanagaraj,
> > >
> > > Who is the first consumer? for what specific purpose?
> > >
> > > Thanks,
> > > Dims
> > >
> > > On Wed, May 11, 2016 at 9:27 AM, Kanagaraj Manickam <mkr1481 at gmail.com
> >
> > > wrote:
> > > > Hi,
> > > >
> > > > When OpenStack service components are started/stooped,
> > > > operators or OpenStack Services want to execute some actives
> > > > before and/or after component is started/stopped.
> > > > Most of the time, operator needs to depends
> > > > on the start-up scripts to do it, which is an installer
> > > > dependent, while OpenStack service can't use this approach.
>
> Can you elaborate on this? I think you're saying that having a start-up
> script take action before a service is started won't work because it
> would require the operator to customize that script. Is that right?
> Aren't there already tools for doing this, though?
>
> > > >
> > > > Also using start-up script does not suite for below situations:
> > > > oslo.service spawns component in more than one processes
> > > > when workers count is more than 1. In this case, if we want
> > > > to execute some activities before/after on each process, start-up
> > > > script does not help.
> > > >
> > > > So to support these scenarios, thinking of below enhancement
> > > > in oslo.service as mentioned in blueprint [1]
> > > >
> > > > Most of the projects in OpenStack does make use of oslo.service
> > > > library to create/start/stop the service api and back-end components.
> > > > And by providing an configurable python hooks as below, and
> > > > enhance oslo.service to execute them appropriately.
> > > >
> > > > [oslo_service]
> > > > List of of pre-hook executed in sequence
> > > > pre-hook=<comma separated python module used as hooks>
> > > > List of of pre-hook executed in sequence
> > > > post-hook=<comma separated python module used as hooks>
> > > >
> > > > And to make sure the hooks does not break the running process,
> > > > try to execute them in try block.
> > > >
> > > > Kindly provide your comments/inputs. Thanks
> > > >
> > > > [1]:
> https://blueprints.launchpad.net/oslo.service/+spec/service-hook
> > > >
> > > > Regards,
> > > > Kanagaraj M
> > > >
> > > >
> > >
> __________________________________________________________________________
> > > > OpenStack Development Mailing List (not for usage questions)
> > > > Unsubscribe:
> > > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > > >
> > >
> > >
> > >
> > > --
> > > Davanum Srinivas :: https://twitter.com/dims
> > >
> > >
> __________________________________________________________________________
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160518/a31819a8/attachment.html>