Open Stack

On Tue, Oct 11, 2016 at 12:40 AM Thomas Bechtold <tbechtold at suse.com> wrote:

> Hi,
>
>
>
> On Mon, Oct 10, 2016 at 10:07:05AM -0600, Alex Schultz wrote:
>
> > On Mon, Oct 10, 2016 at 5:03 AM, Thomas Bechtold <tbechtold at suse.com>
> wrote:
>
> > > Hi,
>
> > >
>
> > > in the rpm-packaging project we started to package the services and are
>
> > > currently discussing a possible schema for configuration files and
>
> > > snippets used by the systemd .service files (but this would also affect
>
> > > OCF resource agents).
>
> > > This affects packagers, endusers and config management systems (Chef,
>
> > > Puppet, Ansible, Salt, ...) so I want to discuss the proposal here on
>
> > > the list.
>
> >
>
> > This also affects deployment tools so you may want to include tripleo,
>
> > kolla, fuel as they are downstream consumers and may have their own
>
> > assumptions about how services are launched.
>
>
>
> Done.
>
>
>
> > > Most services (at least for SUSE and RDO) are using a single config
>
> > > (e.g. /etc/nova/nova.conf) when starting the service. Some services
>
> > > (e.g. neutron services like neutron-l3-agent) use multiple config
> files.
>
> > >
>
> > > There are multiple problems with that approach:
>
> > > - when using a config-mgmt-system, users may want to override a config
>
> > > option (for a feature that is not yet supported) but the
>
> > > config-mgmt-system will override the config later again.
>
> >
>
> > Just to chime in here from a puppet standpoint, this is not a problem
>
> > because we provide a way for a user to add any extra options they wish
>
> > using the provider so it always ends up in the correct configuration
>
> > file.
>
>
>
> Does that also work if you need extra configuration files for 3rd party
>
> plugins (e.g. neutron plugins) ? I guess you could just copy the 3rd
>
> party config file content to the needed neutron config but that's ugly imo.
>
>
>
> > > - when users adjust /etc/$service/$service.conf and a release update is
>
> > > done (e.g. mitaka->newton) /etc/$service/$service.conf wil not be
>
> > > overridden. That's good because the user changed something but otoh the
>
> > > file (with all it's config options and comments) is no longer correct.
>
> >
>
> > Depending on the configuration management tool, the 'default' options
>
> > and comments may not even be there so I'm not sure this is actually
>
> > that much of a concern.  Also when you upgrade there is usually some
>
> > sort of upgrade process that has to go along with your configuration
>
> > management tool which should take care of this for you. So i'm not
>
> > sure this needs to be a packaging concern.
>
> >
>
> > > - when config-mgmt-systems use templates for the $service.conf file,
>
> > > updating theses templates is a lot of work.
>
> >
>
> > Yes which is why tools that don't use templates in the configuration
>
> > management tool makes this a non-issue.  I'm not sure this needs to be
>
> > a concern of packagers as it's an issue with the configuration
>
> > management tool of choice and many of these tools have switched away
>
> > from templates or are currently handling this.  If the configuration
>
> > management tool doesn't support this or is suffering from this, simply
>
> > adding conf.d support might help but then you also run into issues
>
> > about ensuring things are removed and cleaned up.
>
>
>
> There *are* config-mgmt-tools still using templates. And having the
>
> possibility to add config snippets simplifies the process here without
>
> any downside for available solutions. Just don't use it if you don't
>
> need it.
>
>
>
> > > - setting configuration options per service (let's say cinder-volume
>
> > > needs other config options than cinder-api) is not possible.
>
> >
>
> > So I agree this is more likely a real problem, but i'm not sure this
>
> > should be solved by packaging as this probably needs to be addressed
>
> > in upstream.  Unless this is already a thing and it's just never been
>
> > properly implemented when deployed via packages. The issue I have with
>
> > only solving this via rpm packaging is that for tools that support
>
> > both rpms and debs this would lead to 2 different configuration
>
> > mechanisms.  So I would vote not to do anything different then what
>
> > debs also do unless both packaging methods are updated at the same
>
> > time.
>
>
>
> Don't you have already a lot of different implementations for rpm
>
> vs. deb, different apache config dir styles, different package name, ...
>
> Improving the rpm side only if the deb side also changes is not going to
>
> work I think.
>
>
That isn't the issue here, both sides need to agree that this is a good
path forward and agree on some general structure here, and to a large
extent this is an upstream openstack issue to break the service configs
apart. As a config-mgmt-system collaborator, I can tell you that we have to
cover what ever deviation either side comes up with, and roughly gauging
what you are proposing to do here will have a massive impact to how we
configure services on RHEL derivatives which could easily take an entire
cycle for the Puppet team to implement support. If both sides aren't
supporting a similar structure, then things really start to become messy
for us, as we'd have multiple code paths to configure the two styles (and
all the issues that come from that). We can account for minor nuances in
the distros like package names, or config file locations quite easily, but
you are asking for us to put different config elements into entirely
different structures.


>
> > Do we have any examples or instances where an end user would
>
> > specifically want to configure two of the services in a conflicting
>
> > fashion?  Or are there configuration options that fall into this
>
> > pattern? I thought the service specific items where in their own
>
> > configuration namespace to allow for such things. I would assume that
>
> > the bigger issue would be wanting to run two of the same service on
>
> > the same host with different configurations I would think that's where
>
> > something like containers would handle this case better than trying to
>
> > have multiple configuration files.
>
>
>
> In HA case you may want to set the hostname for all cinder-volumes to
>
> the same name but to something different for cinder-api (if both
>
> services run on the same host). I'm sure there are other examples.
>
>
>
> > > So here's the proposal to fix theses problems. The proposal is based on
>
> > > what RDO is already doing with neutron and extends it a bit. Let's do
> it
>
> > > for Nova as an example:
>
> > >
>
> > > - /usr/share/nova/nova-dist.conf
>
> > > This is the configuration file a distribution (openSUSE, RDO, ...) can
>
> > > modify. It must not be modified by endusers and will be overridden with
>
> > > package updates
>
Seems to be a normal process, I'm not against it. I like to have the full,
and commented out config in /etc/nova/nova.conf to reference as it's the
expected location (ie nova-dist.conf with everything commented out) but
that is my preference

> > - /etc/nova/nova.conf
>
> > > This is an empty file . Users/config-mgmt-systems can modify it and it
>
> > > will not be overridden (if changed) with a package update.
>

The mandatory user needs to edit these values or the service can't start
type (mostly credentials, uri for keystone, etc) should be set here and not
be hidden in the -dist.conf files.

> >
>
> > > - /etc/nova/conf.d/common/
>
> > > In this directory, config snippets can be added. By convention,
>
> > > config-mgmt-systems would install files starting with 100-XXX.conf,
>
> > > endusers would install files starting with 500-XXX.conf . Also this
>
> > > directory is used by all services (nova-api, nova-compute, ...).
>

I'm against this, It makes no sense to config-mgmt-systems that aren't
encumbered by templates, and I see no acceptable path forward for
config-mgmt-systems to adopt an effective convention here. It makes sense
in an include-dir directive when the things being included are whole things
i.e. a whole vhost and what is supposed to be in it, and what is supposed
be be elsewhere but with out this I don't see a path forward. How do we
consistently identify what should be put where? Who/how do we ensure it's
codded correctly?

What use cases are you looking to solve by having this level of config
injection?

> > - /etc/nova/conf.d/$service/ (e.g. /etc/nova/conf.d/nova-compute/)
>
> > > Also a dir for config snippets (with same rules as for
>
> > > /etc/nova/conf.d/common/ ) but this dir is only used by $service (in
>
> > > this case nova-compute)
>

I'm for breaking each service into it's own file, as illustrated above, it
can be needed. I am however against directory include for the same issues
as the prior section. Also, by removing the directory include, proposed
structure, service config files could simply be /etc/nova/$service.conf

We should make per-service config files standard in the upstream
themselves, and this is an area where we need to collaborate with Debian as
well. At the same time, we should also be thinking about multiple instances
support for services (a.k.a cinder-volume) that may run multiple instances
but with different config files. Ceph does this quite well with the radosgw
service.


> > > - /usr/share/doc/packages/openstack-nova/nova.conf.sample
>
> > > The unmodified sample config from upstream
>
> > >
>
> > > - /etc/nova/README
>
> > > Explaining the configuration structure and where to find the whole
>
> > > sample config.
>
> > >
>
> >
>
> > From a packaging standpoint it's probably better to provide less and
>
> > not more default configuration files as the tooling usually has to go
>
> > an clean this stuff up as they have their own ways of configuring the
>
> > services.  Currently they may align with the existing packaging files
>
> > or they may completely remove what's provided via packaging and setup
>
> > their own structure.  Speaking from experience, having to cleanup
>
> > package provided configuration files is a pain[0][1].
>
>
>
> There are not more config files with this proposal. It's the same
>
> amount, just more possibilities. the nova-dist.conf is already in use
>
> for RDO and neutron [2]. This would be just used for all services.
>
> /etc/$service/$service.conf is the usual thing and also read by default
>
> by oslo.config . The conf.d directory delivered by a package would be
>
> completly empty. So nothing to cleanup there.
>
>
>
> [2]
>
>
> https://github.com/rdo-packages/neutron-distgit/blob/rpm-master/neutron-server.service
>
>
>
> >
>
> > [0]
> https://github.com/openstack/puppet-horizon/blob/master/manifests/wsgi/apache.pp#L115-L128
>
> > [1]
> https://github.com/openstack/puppet-tripleo/blob/master/manifests/ui.pp#L94-L107
>
> >
>
> > >
>
> > > So starting nova-api would be:
>
> > > nova-api --config-file /usr/share/nova/nova-dist.conf
>
> > >          --config-file /etc/nova/nova.conf
>
> > >          --config-dir /etc/nova/conf.d/common/
>
> > >          --config-dir /etc/nova/conf.d/nova-api/
>
> > >
>
> >
>
> > So from a puppet standpoint we actually go out of our way to default
>
> > to the python defaults and we may purge all values not explicitly set
>
> > via puppet. This would break our assumptions around this.  I
>
> > personally do not agree with forcing the *-dist.conf into the loading
>
> > path as this may also cause issues with unwanted values being injected
>
> > by packagers.
>
>
>
> This is already happening in RDO+neutron. See [2]. So nothing new here.
>
>
>
> > The danger for people who use puppet is that we
>
> > currently expected some values to not be defined in the configuration
>
> > files thus falling back to the python defaults. If they now get
>
> > defined in service-dist.conf and we undefine it in service.conf
>
> > (because that's what we currently configure),  the end user is now
>
> > going to get a value they did not expect or want.
>
>
>
> It's not about providing a complete config (which wouldn't work anyway)
>
> but setting for some values sane defaults. See [3].
>
>
>
> [3]
>
>
> https://github.com/rdo-packages/neutron-distgit/blob/rpm-master/neutron-dist.conf
>
>
>
> > >
>
> > > The order of command line switches (--config-file/--config-dir) is
>
> > > important here. Also --config-dir is ordering the files. So adding a
>
> > > config snippet to /etc/nova/conf.d/nova-api/ with
>
> > > e.g. [DEFAULT]bind_port would override the option from the previous
>
> > > configs (last found option wins).
>
> > >
>
> >
>
> > This gets complicated to follow and may lead to issues on the user
>
> > side when ordering is a problem or attempting to debug issues. I think
>
> > this can lead to more issues than it's solving.
>
>
>
> Most services print the complete config when running in debug mode. So
>
> getting th used config is not complicated. Also adding theses switches
>
> makes it more explicit when doing e.g. "ps awxu|grep nova-api" because
>
> you see then what Also knowing which files/dirs are used is just "ps
>
> awxu|grep $service".
>
> And afaik oslo.config already loads implicit config files if they are
>
> present.
>
>
>
> > Personally unless this structure is also followed by the deb
>
> > packaging, I'd prefer not to switch to this as it may lead to even
>
> > more fragmentation when it comes to trying to configure OpenStack
>
> > deployed via packages.  Has this been requested by an end user to
>
> > solve a specific problem?  What exactly is the problem that's trying
>
> > to be solved other than trying to allow for two of the same project's
>
> > services being configured in (currently) conflicting fashions?
>
>
>
> See my anwers above. It's not only about different configs for different
>
> service.
>
>
>
> Thanks for the feedback!
>
>
>
> Cheers,
>
>
>
> Tom
>
>
>
> __________________________________________________________________________
>
> OpenStack Development Mailing List (not for usage questions)
>
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> --
Andrew Woodward
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161011/c6db8774/attachment.html>