Open Stack

>Excerpts from Kanagaraj Manickam's message of 2016-05-18 19:48:13 +0530:
>> > DIms,
>> >>
>> >> Use case could be anything, which would needed by either
>> >> operator/community, who wants to perform an required task before and
>> after
>> >> service is started. This requirement is very generic by nature, and I
>> >> believe it will be very useful.
>> >>
>> >> Would like to give the sample use cases from from Operator & OpenStack
>> >> community side as below.
>> >> Operator side, any pre/post actions could be hooked which is the
>> >> requirement for them. Simple example would be, one who wants to
create an
>> >> journal of start/stop details like time, number of worker,
>> configurations,
>> >> etc in a common portal, this life-cycle hook would help.
>>
>> > Is that information not available in the logs?
>>
>> [Kanagaraj M] its available in log, but one who wants to collect these
info
>> in centralized portal,
>> it would help.
>>
>> >
>> > OpenStack community side, sample use cases would be:
>> > 1. Most of the OpenStack components starts TextGuruMeditation, logging
>> > while those components are get started. These tasks could be provided
as
>> > life cycle hooks and all OpenStack components could start to leverage
it.
>>
>> >All of those are things that need to be built into the app in a way that
>> >they are started at the right time, rather than at an arbitrary point
>> >defined by the plugin order a deployer has specified.
>>
>> [Kanagaraj M] while i investigated the OpenStack service cmd, mostly
>> it follows the similar pattern on use these utils, so i thought, it
would be
>> good to provide an plugin, which take care of it instead every service
>> code does take care of it. helps to reduces maintenance effort.
>
>Except that we don't want deployers to turn them off, and we need to
>control the initialization order, and so we don't want them to be
>specified through a configuration option.
>

[KanagarajM] sure. I hope we could start with this option,
as a better choice now.

>>
>> >> 2. For automatically discovering the OpenStack deployment, this hooks
>> will
>> >> be very useful. Auto-discover-hook would report to pre-defined
>> destinations
>> >> while starting/stopping the service.
>>
>> >Doing that usefully is going to require passing information to the hook
>> >so it knows where it is running (what service, what port, etc.). None of
>> >the APIs for doing this have been described yet. Do you have any plans
>> >put together?
>>
>> [Kanagaraj M] I am trying to get all of these information from oslo.confg
>> global config variable. As we discussed about namos during austin summit,
>> namos does collect these details
>> https://github.com/openstack/os-namos/blob/master/os_namos/sync.py#L124
>
>There are 2 issues with having a plugin access config values directly.
>
>Configuration options are owned by the code that defines them, and
>are not considered a public "API" for other parts of the code. That
>means an application or library developer is free to change the
>name or location of a configuration option, without regard to code
>that might be trying to use it from outside of the owning module.
>oslo.config has support for renames so that *deployers* aren't
>broken, but we don't do anything to prevent code that accesses
>private values from breaking.  So, you don't want to build any
>assumptions into the plugins that they will be able to see configuration
>values.
>
>Second, options do not "exist" as far as oslo.config is concerned
>until they are registered.  The registration step may happen at
>runtime in code that has not executed before the plugin is loaded. So
>even if we ignore the "private" nature of the option, there is a timing
>issue.
[KanagarajM] As part of the investigation, found that most of OpenStack
service components logs the complete configuration options in the log,
once service is started and I hope the same point, the discovery-hook
could be registered. (This could be avoided in future by using the
oslo-config-generator to get all options, i hope, if my understanding is
correct :)

>Well, I still think this is a poor design for the feature you want.
>It really seems like service management should happen from outside
>of the service being managed. I'm going to need to see a lot more
>detailed thought put into answers to those questions before I could
>support adding this feature to oslo.service.
>
[KanagarajM] we already doing the in-band service deployment discovery
for nova, cinder, neutron and heat, which are reported in horizon
panel as well under 'system information' panel. When we look at the impl
of service discovery in each of these projects, all rpeats the same
implementation in each of their repository. Once this is in place, we
could enable the auto-discovery of each service deployment without
repeating the impl, and those projects also could start to levarage it
which releaves a considerable maintenance effort from each of the projects
for its deployment discovery.
>> >> On Wed, May 11, 2016 at 7:05 PM, Davanum Srinivas <davanum at gmail.com>
>> wrote:
>> >>
>> >> > Kanagaraj,
>> > >
>> >> > Who is the first consumer? for what specific purpose?
>> >> >
>> >> > Thanks,
>> >> > Dims
>> >> >
>> >> > On Wed, May 11, 2016 at 9:27 AM, Kanagaraj Manickam <
mkr1481 at gmail.com>
>> >> > wrote:
>> >> > > Hi,
>> >> > >
>> >> > > When OpenStack service components are started/stooped,
>> >> > > operators or OpenStack Services want to execute some actives
>> >> > > before and/or after component is started/stopped.
>> >> > > Most of the time, operator needs to depends
>> >> > > on the start-up scripts to do it, which is an installer
>> >> > > dependent, while OpenStack service can't use this approach.
>>
>> >Can you elaborate on this? I think you're saying that having a start-up
>> >script take action before a service is started won't work because it
>> >would require the operator to customize that script. Is that right?
>> >Aren't there already tools for doing this, though?
>>
>> [KanagarajM] when we use start-up script, we can't do any pre/post
>> actions in every worker process, instead we can only do pre/post once
>> before/after launcher and all worker process started.
>
>Can you describe some of the sorts of things you expect to need to do
>for each process?

[KanagarajM] one of the use case i was thinking of is, while upgrading, we
need to put
the service components to maintenance state, so that it would stop taking
further request and once it finishes it current on going request processing,
it can exist. to do that, each worker process to be notified to go to
maintenance
mode, by providing an maintenance-hook, any service it wants to support
the maintenance cycle, could leverage it.

On Wed, May 18, 2016 at 8:10 PM, Doug Hellmann <doug at doughellmann.com>
wrote:

> Excerpts from Kanagaraj Manickam's message of 2016-05-18 19:48:13 +0530:
> > > DIms,
> > >>
> > >> Use case could be anything, which would needed by either
> > >> operator/community, who wants to perform an required task before and
> > after
> > >> service is started. This requirement is very generic by nature, and I
> > >> believe it will be very useful.
> > >>
> > >> Would like to give the sample use cases from from Operator & OpenStack
> > >> community side as below.
> > >> Operator side, any pre/post actions could be hooked which is the
> > >> requirement for them. Simple example would be, one who wants to
> create an
> > >> journal of start/stop details like time, number of worker,
> > configurations,
> > >> etc in a common portal, this life-cycle hook would help.
> >
> > > Is that information not available in the logs?
> >
> > [Kanagaraj M] its available in log, but one who wants to collect these
> info
> > in centralized portal,
> > it would help.
> >
> > >
> > > OpenStack community side, sample use cases would be:
> > > 1. Most of the OpenStack components starts TextGuruMeditation, logging
> > > while those components are get started. These tasks could be provided
> as
> > > life cycle hooks and all OpenStack components could start to leverage
> it.
> >
> > >All of those are things that need to be built into the app in a way that
> > >they are started at the right time, rather than at an arbitrary point
> > >defined by the plugin order a deployer has specified.
> >
> > [Kanagaraj M] while i investigated the OpenStack service cmd, mostly
> > it follows the similar pattern on use these utils, so i thought, it
> would be
> > good to provide an plugin, which take care of it instead every service
> > code does take care of it. helps to reduces maintenance effort.
>
> Except that we don't want deployers to turn them off, and we need to
> control the initialization order, and so we don't want them to be
> specified through a configuration option.
>
> >
> > >> 2. For automatically discovering the OpenStack deployment, this hooks
> > will
> > >> be very useful. Auto-discover-hook would report to pre-defined
> > destinations
> > >> while starting/stopping the service.
> >
> > >Doing that usefully is going to require passing information to the hook
> > >so it knows where it is running (what service, what port, etc.). None of
> > >the APIs for doing this have been described yet. Do you have any plans
> > >put together?
> >
> > [Kanagaraj M] I am trying to get all of these information from oslo.confg
> > global config variable. As we discussed about namos during austin summit,
> > namos does collect these details
> > https://github.com/openstack/os-namos/blob/master/os_namos/sync.py#L124
>
> There are 2 issues with having a plugin access config values directly.
>
> Configuration options are owned by the code that defines them, and
> are not considered a public "API" for other parts of the code. That
> means an application or library developer is free to change the
> name or location of a configuration option, without regard to code
> that might be trying to use it from outside of the owning module.
> oslo.config has support for renames so that *deployers* aren't
> broken, but we don't do anything to prevent code that accesses
> private values from breaking.  So, you don't want to build any
> assumptions into the plugins that they will be able to see configuration
> values.
>
> Second, options do not "exist" as far as oslo.config is concerned
> until they are registered.  The registration step may happen at
> runtime in code that has not executed before the plugin is loaded. So
> even if we ignore the "private" nature of the option, there is a timing
> issue.
>
> >
> > >It feels very much like you're advocating that we create a thing like
> > >paste-deploy for non-WSGI apps: allow anyone to insert anything into
> > >the executing process for any purpose and with little control on the
> > >application authors' part. That introduces potential stability issues,
> > >and a lot of questions that haven't been answered. For example:
> >
> > >Does service startup block until all of the plugins are done? If not,
> > >do we need to have a timeout management system built in or do we run
> > >the plugins in their own thread/subprocess so they can run in the
> > >background?
> >
> > >Can a plugin change the execution of the service in some way (you
> > >mentioned having a plugin download config files when we spoke at the
> > >summit, is that still something you want to slip in this way instead of
> > >putting it into oslo.config)?
> >
> > >Can a plugin cause the service to not start at all by exiting?
> >
> > >What happens if one plugin fails but others succeed? Do we keep running?
> >
> > >What information about the service can a plugin see? It's running in the
> > >same process, so it could see the configuration object, for example.
> > >It would only be able to see configuration options it adds (yes, that
> > >would work) or that were registered by the application before the plugin
> > >is run. So, not necessarily all options but potentially many, with more
> > >and more as apps shift to pre-registering all of their options in one
> > >place. Assuming these are things the deployer has selectively installed,
> > >maybe that's OK. OTOH, it does open another security surface.
> >
> > >What happens when a service is told to restart its workers? Do the
> > >plugins run again?
> >
> > >Can a plugin start listening for network connections on its own? Connect
> > >to the message bus?  Provide an RPC endpoint? Start processes? Threads?
> >
> > [KanagarajM] It gives me a lots insight on what are different problems
> > would come and i really thank you.
> > Hooks will be provided by community and/or deployers. while community
> > provides, those hooks will be well documented, tested and configuration
> > would be
> > provided. so all of the above mentioned aspects would be taken care well
> by
> > community based on the hooks functionality. And deployer also would take
> >  care of similar safety measurement before using their hooks similar to
> how
> > would they take care when using startup scripts.
>
> Well, I still think this is a poor design for the feature you want.
> It really seems like service management should happen from outside
> of the service being managed. I'm going to need to see a lot more
> detailed thought put into answers to those questions before I could
> support adding this feature to oslo.service.
>
> > >> On Wed, May 11, 2016 at 7:05 PM, Davanum Srinivas <davanum at gmail.com>
> > wrote:
> > >>
> > >> > Kanagaraj,
> > > >
> > >> > Who is the first consumer? for what specific purpose?
> > >> >
> > >> > Thanks,
> > >> > Dims
> > >> >
> > >> > On Wed, May 11, 2016 at 9:27 AM, Kanagaraj Manickam <
> mkr1481 at gmail.com>
> > >> > wrote:
> > >> > > Hi,
> > >> > >
> > >> > > When OpenStack service components are started/stooped,
> > >> > > operators or OpenStack Services want to execute some actives
> > >> > > before and/or after component is started/stopped.
> > >> > > Most of the time, operator needs to depends
> > >> > > on the start-up scripts to do it, which is an installer
> > >> > > dependent, while OpenStack service can't use this approach.
> >
> > >Can you elaborate on this? I think you're saying that having a start-up
> > >script take action before a service is started won't work because it
> > >would require the operator to customize that script. Is that right?
> > >Aren't there already tools for doing this, though?
> >
> > [KanagarajM] when we use start-up script, we can't do any pre/post
> > actions in every worker process, instead we can only do pre/post once
> > before/after launcher and all worker process started.
>
> Can you describe some of the sorts of things you expect to need to do
> for each process?
>
> Doug
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160523/2152c038/attachment-0001.html>