Open Stack

Doug Hellmann wrote:
> Excerpts from Kanagaraj Manickam's message of 2016-05-18 19:48:13 +0530:
>>> DIms,
>>>> Use case could be anything, which would needed by either
>>>> operator/community, who wants to perform an required task before and
>> after
>>>> service is started. This requirement is very generic by nature, and I
>>>> believe it will be very useful.
>>>>
>>>> Would like to give the sample use cases from from Operator&  OpenStack
>>>> community side as below.
>>>> Operator side, any pre/post actions could be hooked which is the
>>>> requirement for them. Simple example would be, one who wants to create an
>>>> journal of start/stop details like time, number of worker,
>> configurations,
>>>> etc in a common portal, this life-cycle hook would help.
>>> Is that information not available in the logs?
>> [Kanagaraj M] its available in log, but one who wants to collect these info
>> in centralized portal,
>> it would help.
>>
>>> OpenStack community side, sample use cases would be:
>>> 1. Most of the OpenStack components starts TextGuruMeditation, logging
>>> while those components are get started. These tasks could be provided as
>>> life cycle hooks and all OpenStack components could start to leverage it.
>>> All of those are things that need to be built into the app in a way that
>>> they are started at the right time, rather than at an arbitrary point
>>> defined by the plugin order a deployer has specified.
>> [Kanagaraj M] while i investigated the OpenStack service cmd, mostly
>> it follows the similar pattern on use these utils, so i thought, it would be
>> good to provide an plugin, which take care of it instead every service
>> code does take care of it. helps to reduces maintenance effort.
>
> Except that we don't want deployers to turn them off, and we need to
> control the initialization order, and so we don't want them to be
> specified through a configuration option.
>
>>>> 2. For automatically discovering the OpenStack deployment, this hooks
>> will
>>>> be very useful. Auto-discover-hook would report to pre-defined
>> destinations
>>>> while starting/stopping the service.
>>> Doing that usefully is going to require passing information to the hook
>>> so it knows where it is running (what service, what port, etc.). None of
>>> the APIs for doing this have been described yet. Do you have any plans
>>> put together?
>> [Kanagaraj M] I am trying to get all of these information from oslo.confg
>> global config variable. As we discussed about namos during austin summit,
>> namos does collect these details
>> https://github.com/openstack/os-namos/blob/master/os_namos/sync.py#L124
>
> There are 2 issues with having a plugin access config values directly.
>
> Configuration options are owned by the code that defines them, and
> are not considered a public "API" for other parts of the code. That
> means an application or library developer is free to change the
> name or location of a configuration option, without regard to code
> that might be trying to use it from outside of the owning module.
> oslo.config has support for renames so that *deployers* aren't
> broken, but we don't do anything to prevent code that accesses
> private values from breaking.  So, you don't want to build any
> assumptions into the plugins that they will be able to see configuration
> values.
>
> Second, options do not "exist" as far as oslo.config is concerned
> until they are registered.  The registration step may happen at
> runtime in code that has not executed before the plugin is loaded. So
> even if we ignore the "private" nature of the option, there is a timing
> issue.
>
>>> It feels very much like you're advocating that we create a thing like
>>> paste-deploy for non-WSGI apps: allow anyone to insert anything into
>>> the executing process for any purpose and with little control on the
>>> application authors' part. That introduces potential stability issues,
>>> and a lot of questions that haven't been answered. For example:
>>> Does service startup block until all of the plugins are done? If not,
>>> do we need to have a timeout management system built in or do we run
>>> the plugins in their own thread/subprocess so they can run in the
>>> background?
>>> Can a plugin change the execution of the service in some way (you
>>> mentioned having a plugin download config files when we spoke at the
>>> summit, is that still something you want to slip in this way instead of
>>> putting it into oslo.config)?
>>> Can a plugin cause the service to not start at all by exiting?
>>> What happens if one plugin fails but others succeed? Do we keep running?
>>> What information about the service can a plugin see? It's running in the
>>> same process, so it could see the configuration object, for example.
>>> It would only be able to see configuration options it adds (yes, that
>>> would work) or that were registered by the application before the plugin
>>> is run. So, not necessarily all options but potentially many, with more
>>> and more as apps shift to pre-registering all of their options in one
>>> place. Assuming these are things the deployer has selectively installed,
>>> maybe that's OK. OTOH, it does open another security surface.
>>> What happens when a service is told to restart its workers? Do the
>>> plugins run again?
>>> Can a plugin start listening for network connections on its own? Connect
>>> to the message bus?  Provide an RPC endpoint? Start processes? Threads?
>> [KanagarajM] It gives me a lots insight on what are different problems
>> would come and i really thank you.
>> Hooks will be provided by community and/or deployers. while community
>> provides, those hooks will be well documented, tested and configuration
>> would be
>> provided. so all of the above mentioned aspects would be taken care well by
>> community based on the hooks functionality. And deployer also would take
>>   care of similar safety measurement before using their hooks similar to how
>> would they take care when using startup scripts.
>
> Well, I still think this is a poor design for the feature you want.
> It really seems like service management should happen from outside
> of the service being managed. I'm going to need to see a lot more
> detailed thought put into answers to those questions before I could
> support adding this feature to oslo.service.

Wouldn't systemd have something for this already (assuming u are using 
systemd)? Seeing as one of the usages of systemd is dependency of 
services it almost feels like the service management from that would be 
appropriate here? Is there any major advantage that doing this in 
oslo.service would have over doing it externally?

https://fedoramagazine.org/systemd-unit-dependencies-and-order/ ?

Am I missing something that would seem to make an inline solution that 
much better vs an external one (and using an external one that is 
detected to doing these kinds of things).

>
>>>> On Wed, May 11, 2016 at 7:05 PM, Davanum Srinivas<davanum at gmail.com>
>> wrote:
>>>>> Kanagaraj,
>>>>> Who is the first consumer? for what specific purpose?
>>>>>
>>>>> Thanks,
>>>>> Dims
>>>>>
>>>>> On Wed, May 11, 2016 at 9:27 AM, Kanagaraj Manickam<mkr1481 at gmail.com>
>>>>> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> When OpenStack service components are started/stooped,
>>>>>> operators or OpenStack Services want to execute some actives
>>>>>> before and/or after component is started/stopped.
>>>>>> Most of the time, operator needs to depends
>>>>>> on the start-up scripts to do it, which is an installer
>>>>>> dependent, while OpenStack service can't use this approach.
>>> Can you elaborate on this? I think you're saying that having a start-up
>>> script take action before a service is started won't work because it
>>> would require the operator to customize that script. Is that right?
>>> Aren't there already tools for doing this, though?
>> [KanagarajM] when we use start-up script, we can't do any pre/post
>> actions in every worker process, instead we can only do pre/post once
>> before/after launcher and all worker process started.
>
> Can you describe some of the sorts of things you expect to need to do
> for each process?
>
> Doug
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev