[openstack-dev] [rpm-packaging][chef][puppet][salt][openstack-ansible][HA] Schema proposal for config file handling for services

Alex Schultz aschultz at redhat.com
Mon Oct 10 16:07:05 UTC 2016


On Mon, Oct 10, 2016 at 5:03 AM, Thomas Bechtold <tbechtold at suse.com> wrote:
> Hi,
>
> in the rpm-packaging project we started to package the services and are
> currently discussing a possible schema for configuration files and
> snippets used by the systemd .service files (but this would also affect
> OCF resource agents).
> This affects packagers, endusers and config management systems (Chef,
> Puppet, Ansible, Salt, ...) so I want to discuss the proposal here on
> the list.

This also affects deployment tools so you may want to include tripleo,
kolla, fuel as they are downstream consumers and may have their own
assumptions about how services are launched.

>
> Most services (at least for SUSE and RDO) are using a single config
> (e.g. /etc/nova/nova.conf) when starting the service. Some services
> (e.g. neutron services like neutron-l3-agent) use multiple config files.
>
> There are multiple problems with that approach:
> - when using a config-mgmt-system, users may want to override a config
> option (for a feature that is not yet supported) but the
> config-mgmt-system will override the config later again.

Just to chime in here from a puppet standpoint, this is not a problem
because we provide a way for a user to add any extra options they wish
using the provider so it always ends up in the correct configuration
file.

> - when users adjust /etc/$service/$service.conf and a release update is
> done (e.g. mitaka->newton) /etc/$service/$service.conf wil not be
> overridden. That's good because the user changed something but otoh the
> file (with all it's config options and comments) is no longer correct.

Depending on the configuration management tool, the 'default' options
and comments may not even be there so I'm not sure this is actually
that much of a concern.  Also when you upgrade there is usually some
sort of upgrade process that has to go along with your configuration
management tool which should take care of this for you. So i'm not
sure this needs to be a packaging concern.

> - when config-mgmt-systems use templates for the $service.conf file,
> updating theses templates is a lot of work.

Yes which is why tools that don't use templates in the configuration
management tool makes this a non-issue.  I'm not sure this needs to be
a concern of packagers as it's an issue with the configuration
management tool of choice and many of these tools have switched away
from templates or are currently handling this.  If the configuration
management tool doesn't support this or is suffering from this, simply
adding conf.d support might help but then you also run into issues
about ensuring things are removed and cleaned up.

> - setting configuration options per service (let's say cinder-volume
> needs other config options than cinder-api) is not possible.

So I agree this is more likely a real problem, but i'm not sure this
should be solved by packaging as this probably needs to be addressed
in upstream.  Unless this is already a thing and it's just never been
properly implemented when deployed via packages. The issue I have with
only solving this via rpm packaging is that for tools that support
both rpms and debs this would lead to 2 different configuration
mechanisms.  So I would vote not to do anything different then what
debs also do unless both packaging methods are updated at the same
time.  Do we have any examples or instances where an end user would
specifically want to configure two of the services in a conflicting
fashion?  Or are there configuration options that fall into this
pattern? I thought the service specific items where in their own
configuration namespace to allow for such things. I would assume that
the bigger issue would be wanting to run two of the same service on
the same host with different configurations I would think that's where
something like containers would handle this case better than trying to
have multiple configuration files.

>
> So here's the proposal to fix theses problems. The proposal is based on
> what RDO is already doing with neutron and extends it a bit. Let's do it
> for Nova as an example:
>
> - /usr/share/nova/nova-dist.conf
> This is the configuration file a distribution (openSUSE, RDO, ...) can
> modify. It must not be modified by endusers and will be overridden with
> package updates
>
> - /etc/nova/nova.conf
> This is an empty file . Users/config-mgmt-systems can modify it and it
> will not be overridden (if changed) with a package update.
>
> - /etc/nova/conf.d/common/
> In this directory, config snippets can be added. By convention,
> config-mgmt-systems would install files starting with 100-XXX.conf,
> endusers would install files starting with 500-XXX.conf . Also this
> directory is used by all services (nova-api, nova-compute, ...).
>
> - /etc/nova/conf.d/$service/ (e.g. /etc/nova/conf.d/nova-compute/)
> Also a dir for config snippets (with same rules as for
> /etc/nova/conf.d/common/ ) but this dir is only used by $service (in
> this case nova-compute)
>
> - /usr/share/doc/packages/openstack-nova/nova.conf.sample
> The unmodified sample config from upstream
>
> - /etc/nova/README
> Explaining the configuration structure and where to find the whole
> sample config.
>

>From a packaging standpoint it's probably better to provide less and
not more default configuration files as the tooling usually has to go
an clean this stuff up as they have their own ways of configuring the
services.  Currently they may align with the existing packaging files
or they may completely remove what's provided via packaging and setup
their own structure.  Speaking from experience, having to cleanup
package provided configuration files is a pain[0][1].

[0] https://github.com/openstack/puppet-horizon/blob/master/manifests/wsgi/apache.pp#L115-L128
[1] https://github.com/openstack/puppet-tripleo/blob/master/manifests/ui.pp#L94-L107

>
> So starting nova-api would be:
> nova-api --config-file /usr/share/nova/nova-dist.conf
>          --config-file /etc/nova/nova.conf
>          --config-dir /etc/nova/conf.d/common/
>          --config-dir /etc/nova/conf.d/nova-api/
>

So from a puppet standpoint we actually go out of our way to default
to the python defaults and we may purge all values not explicitly set
via puppet. This would break our assumptions around this.  I
personally do not agree with forcing the *-dist.conf into the loading
path as this may also cause issues with unwanted values being injected
by packagers.  The danger for people who use puppet is that we
currently expected some values to not be defined in the configuration
files thus falling back to the python defaults. If they now get
defined in service-dist.conf and we undefine it in service.conf
(because that's what we currently configure),  the end user is now
going to get a value they did not expect or want.

>
> The order of command line switches (--config-file/--config-dir) is
> important here. Also --config-dir is ordering the files. So adding a
> config snippet to /etc/nova/conf.d/nova-api/ with
> e.g. [DEFAULT]bind_port would override the option from the previous
> configs (last found option wins).
>

This gets complicated to follow and may lead to issues on the user
side when ordering is a problem or attempting to debug issues. I think
this can lead to more issues than it's solving.

Personally unless this structure is also followed by the deb
packaging, I'd prefer not to switch to this as it may lead to even
more fragmentation when it comes to trying to configure OpenStack
deployed via packages.  Has this been requested by an end user to
solve a specific problem?  What exactly is the problem that's trying
to be solved other than trying to allow for two of the same project's
services being configured in (currently) conflicting fashions?

Thanks,
-Alex

>
> Feedback very welcome!
>
> Cheers,
>
> Tom
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list