Open Stack

On Tue, Jan 03, 2017 at 03:20:37PM -0500, Lars Kellogg-Stedman wrote:
> On Fri, Dec 23, 2016 at 02:21:00PM +0000, Steven Hardy wrote:
> > I commented on the bug, I'm not sure about this as it seems to overlap with
> > our service_config_settings interface, which IIRC landed slightly after
> > your previous patches for opstools things, and potentially provides a
> > cleaner way to approach this.
> 
> I'm not sure I see how to apply that, but let me further describe the
> use case and you can perhaps point me in the right direction.
> 
> > Perhaps you can point to some examples of this usage, then we can compare
> > with the service_config_settings approach?
> > 
> > I suspect the main difference is you need to append data for each service
> > to e.g the collectd configuration?
> 
> Let's take the existing Fluentd support as an example.  We want the
> ability for every service to provide a logging source configuation for
> Fluentd, which will get aggregated into a logging_sources
> list and then ultimately used in puppet-tripleo to populate a series
> of ::fluentd::config resources.
> 
> Currently, the aggregation happens in a combination of
> puppet/services/services.yaml (which aggregates the logging_source
> attribute from all the services in the service chain) and in
> overcloud.j2.yaml (which actually instantiates the hiera data).
> 
> With the LateServiceChain I've proposed, this could all be isolated
> inside the fluentd composable service: it would not be necessary to
> expose any of this logic in either services.yaml or overcloud.j2.yaml,
> leading.  Additionally, it would not require the fluentd service to
> have any a priori knowledge of what services were in use; it would
> simply aggregate any configuration information that is provided in the
> primary service chain.  It would also allow us to get rid of the
> "LoggingConfiguration" resource, which only exists as a way to expose
> certain parameter_defaults inside services.yaml.

Ok, so it's very similar to service_config_settings, except you need to
append to a list of settings for all services?

The problem I have with your current solution (as just mentioned on the
review) is it makes operators have to care about service ordering, because
it duplicates the per-role *Services parameters.

A key part of the custom roles work was to provide a simple interface that
enables operators to select specific services for each role, without having
to care at all about ordering (this is handled by the deployment steps in
the puppet profiles).

Thinking of ways around this, a couple of options come to mind:

1. Add a new "append" version of service_config_settings

(example based on https://review.openstack.org/#/c/411048)

  # In services/nova-compute.yaml
  service_config_settings_list:
      collectd:
        tripleo::profile::base::metrics::collectd::collectd_plugins:
          - virt

  # In services/foo.yaml
  service_config_settings_list:
      collectd:
        tripleo::profile::base::metrics::collectd::collectd_plugins:
          - foo

  Then in on the role running collectd, you'd get the hieradata key set
with a list containing "virt" and "foo" ?

This would require some data mangling in services.yaml, and a new interface
to the services templates, but it would leave the operator interfaces
unchanged?

2. Do the list manipulation in puppet, like we do for firewall rules

E.g see:

https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/ceilometer-api.yaml#L62

https://github.com/openstack/puppet-tripleo/blob/master/manifests/firewall/service_rules.pp#L32

This achieves the same logical result as the above, but it does the list
manipulation in the puppet profile instead of t-h-t.

I think either approach would be fine, but I've got a slight preference for
(1) as I think it may be more reusable in a future non-puppet world, e.g
for container deployments etc where we may not always want to use puppet.

Open to other suggestions, but would either of the above solve your
problem?

Thanks,

Steve