Open Stack

Hi,

On Tue, Oct 11, 2016 at 06:22:25PM +0000, Andrew Woodward wrote:
[snipped]
> > > So I agree this is more likely a real problem, but i'm not sure this
> >
> > > should be solved by packaging as this probably needs to be addressed
> >
> > > in upstream.  Unless this is already a thing and it's just never been
> >
> > > properly implemented when deployed via packages. The issue I have with
> >
> > > only solving this via rpm packaging is that for tools that support
> >
> > > both rpms and debs this would lead to 2 different configuration
> >
> > > mechanisms.  So I would vote not to do anything different then what
> >
> > > debs also do unless both packaging methods are updated at the same
> >
> > > time.
> >
> >
> >
> > Don't you have already a lot of different implementations for rpm
> >
> > vs. deb, different apache config dir styles, different package name, ...
> >
> > Improving the rpm side only if the deb side also changes is not going to
> >
> > work I think.
> >
> >
> That isn't the issue here, both sides need to agree that this is a good
> path forward and agree on some general structure here, and to a large
> extent this is an upstream openstack issue to break the service configs
> apart. As a config-mgmt-system collaborator, I can tell you that we have to
> cover what ever deviation either side comes up with, and roughly gauging
> what you are proposing to do here will have a massive impact to how we
> configure services on RHEL derivatives which could easily take an entire
> cycle for the Puppet team to implement support.

Hm. Maybe you missunderstand the proposal. You would need to change
absolutly *nothing*. Your config-mgmt-system would just work.
And in addition, people/config-mgmt-systems which want to use snippets
can simply do it.

> If both sides aren't
> supporting a similar structure, then things really start to become messy
> for us, as we'd have multiple code paths to configure the two styles (and
> all the issues that come from that). We can account for minor nuances in
> the distros like package names, or config file locations quite easily, but
> you are asking for us to put different config elements into entirely
> different structures.

I'm not asking you todo anything. The proposal is compatible with the
solutions that are already available.

Maybe as a small reminder - this proposal is from the rpm-packaging team
which *unifies* the packaging for SUSE, RDO and Miranis. So it will be
*simpler* for you because all theses distro will have the same package
structure.

> > > Do we have any examples or instances where an end user would
> >
> > > specifically want to configure two of the services in a conflicting
> >
> > > fashion?  Or are there configuration options that fall into this
> >
> > > pattern? I thought the service specific items where in their own
> >
> > > configuration namespace to allow for such things. I would assume that
> >
> > > the bigger issue would be wanting to run two of the same service on
> >
> > > the same host with different configurations I would think that's where
> >
> > > something like containers would handle this case better than trying to
> >
> > > have multiple configuration files.
> >
> >
> >
> > In HA case you may want to set the hostname for all cinder-volumes to
> >
> > the same name but to something different for cinder-api (if both
> >
> > services run on the same host). I'm sure there are other examples.
> >
> >
> >
> > > > So here's the proposal to fix theses problems. The proposal is based on
> >
> > > > what RDO is already doing with neutron and extends it a bit. Let's do
> > it
> >
> > > > for Nova as an example:
> >
> > > >
> >
> > > > - /usr/share/nova/nova-dist.conf
> >
> > > > This is the configuration file a distribution (openSUSE, RDO, ...) can
> >
> > > > modify. It must not be modified by endusers and will be overridden with
> >
> > > > package updates
> >
> Seems to be a normal process, I'm not against it. I like to have the full,
> and commented out config in /etc/nova/nova.conf to reference as it's the
> expected location (ie nova-dist.conf with everything commented out) but
> that is my preference
> 
> > > - /etc/nova/nova.conf
> >
> > > > This is an empty file . Users/config-mgmt-systems can modify it and it
> >
> > > > will not be overridden (if changed) with a package update.
> >
> 
> The mandatory user needs to edit these values or the service can't start
> type (mostly credentials, uri for keystone, etc) should be set here and not
> be hidden in the -dist.conf files.

Sure. That's the intention.

> 
> > >
> >
> > > > - /etc/nova/conf.d/common/
> >
> > > > In this directory, config snippets can be added. By convention,
> >
> > > > config-mgmt-systems would install files starting with 100-XXX.conf,
> >
> > > > endusers would install files starting with 500-XXX.conf . Also this
> >
> > > > directory is used by all services (nova-api, nova-compute, ...).
> >
> 
> I'm against this, It makes no sense to config-mgmt-systems that aren't
> encumbered by templates, and I see no acceptable path forward for
> config-mgmt-systems to adopt an effective convention here.

As I said - there are sytems that still use templates. I.e. Crowbar.

> It makes sense
> in an include-dir directive when the things being included are whole things
> i.e. a whole vhost and what is supposed to be in it, and what is supposed
> be be elsewhere but with out this I don't see a path forward. How do we
> consistently identify what should be put where? Who/how do we ensure it's
> codded correctly?
> 
> What use cases are you looking to solve by having this level of config
> injection?

As I already said, a) confgi-mgmt-systems that use templates and b)
users that want to add configs which are not supported by
$config-mgmt-system (e.g. 3rd party plugins with extra config files)

> 
> > > - /etc/nova/conf.d/$service/ (e.g. /etc/nova/conf.d/nova-compute/)
> >
> > > > Also a dir for config snippets (with same rules as for
> >
> > > > /etc/nova/conf.d/common/ ) but this dir is only used by $service (in
> >
> > > > this case nova-compute)
> >
> 
> I'm for breaking each service into it's own file, as illustrated above, it
> can be needed. I am however against directory include for the same issues
> as the prior section. Also, by removing the directory include, proposed
> structure, service config files could simply be /etc/nova/$service.conf
> 
> We should make per-service config files standard in the upstream
> themselves, and this is an area where we need to collaborate with Debian as
> well. At the same time, we should also be thinking about multiple instances
> support for services (a.k.a cinder-volume) that may run multiple instances
> but with different config files. Ceph does this quite well with the radosgw
> service.
> 
> 
> > > > - /usr/share/doc/packages/openstack-nova/nova.conf.sample
> >
> > > > The unmodified sample config from upstream
> >
> > > >
> >
> > > > - /etc/nova/README
> >
> > > > Explaining the configuration structure and where to find the whole
> >
> > > > sample config.
> >
> > > >
> >
> > >
> >
> > > From a packaging standpoint it's probably better to provide less and
> >
> > > not more default configuration files as the tooling usually has to go
> >
> > > an clean this stuff up as they have their own ways of configuring the
> >
> > > services.  Currently they may align with the existing packaging files
> >
> > > or they may completely remove what's provided via packaging and setup
> >
> > > their own structure.  Speaking from experience, having to cleanup
> >
> > > package provided configuration files is a pain[0][1].
> >
> >
> >
> > There are not more config files with this proposal. It's the same
> >
> > amount, just more possibilities. the nova-dist.conf is already in use
> >
> > for RDO and neutron [2]. This would be just used for all services.
> >
> > /etc/$service/$service.conf is the usual thing and also read by default
> >
> > by oslo.config . The conf.d directory delivered by a package would be
> >
> > completly empty. So nothing to cleanup there.
> >
> >
> >
> > [2]
> >
> >
> > https://github.com/rdo-packages/neutron-distgit/blob/rpm-master/neutron-server.service
> >
> >
> >
> > >
> >
> > > [0]
> > https://github.com/openstack/puppet-horizon/blob/master/manifests/wsgi/apache.pp#L115-L128
> >
> > > [1]
> > https://github.com/openstack/puppet-tripleo/blob/master/manifests/ui.pp#L94-L107
> >
> > >
> >
> > > >
> >
> > > > So starting nova-api would be:
> >
> > > > nova-api --config-file /usr/share/nova/nova-dist.conf
> >
> > > >          --config-file /etc/nova/nova.conf
> >
> > > >          --config-dir /etc/nova/conf.d/common/
> >
> > > >          --config-dir /etc/nova/conf.d/nova-api/
> >
> > > >
> >
> > >
> >
> > > So from a puppet standpoint we actually go out of our way to default
> >
> > > to the python defaults and we may purge all values not explicitly set
> >
> > > via puppet. This would break our assumptions around this.  I
> >
> > > personally do not agree with forcing the *-dist.conf into the loading
> >
> > > path as this may also cause issues with unwanted values being injected
> >
> > > by packagers.
> >
> >
> >
> > This is already happening in RDO+neutron. See [2]. So nothing new here.
> >
> >
> >
> > > The danger for people who use puppet is that we
> >
> > > currently expected some values to not be defined in the configuration
> >
> > > files thus falling back to the python defaults. If they now get
> >
> > > defined in service-dist.conf and we undefine it in service.conf
> >
> > > (because that's what we currently configure),  the end user is now
> >
> > > going to get a value they did not expect or want.
> >
> >
> >
> > It's not about providing a complete config (which wouldn't work anyway)
> >
> > but setting for some values sane defaults. See [3].
> >
> >
> >
> > [3]
> >
> >
> > https://github.com/rdo-packages/neutron-distgit/blob/rpm-master/neutron-dist.conf
> >
> >
> >
> > > >
> >
> > > > The order of command line switches (--config-file/--config-dir) is
> >
> > > > important here. Also --config-dir is ordering the files. So adding a
> >
> > > > config snippet to /etc/nova/conf.d/nova-api/ with
> >
> > > > e.g. [DEFAULT]bind_port would override the option from the previous
> >
> > > > configs (last found option wins).
> >
> > > >
> >
> > >
> >
> > > This gets complicated to follow and may lead to issues on the user
> >
> > > side when ordering is a problem or attempting to debug issues. I think
> >
> > > this can lead to more issues than it's solving.
> >
> >
> >
> > Most services print the complete config when running in debug mode. So
> >
> > getting th used config is not complicated. Also adding theses switches
> >
> > makes it more explicit when doing e.g. "ps awxu|grep nova-api" because
> >
> > you see then what Also knowing which files/dirs are used is just "ps
> >
> > awxu|grep $service".
> >
> > And afaik oslo.config already loads implicit config files if they are
> >
> > present.
> >
> >
> >
> > > Personally unless this structure is also followed by the deb
> >
> > > packaging, I'd prefer not to switch to this as it may lead to even
> >
> > > more fragmentation when it comes to trying to configure OpenStack
> >
> > > deployed via packages.  Has this been requested by an end user to
> >
> > > solve a specific problem?  What exactly is the problem that's trying
> >
> > > to be solved other than trying to allow for two of the same project's
> >
> > > services being configured in (currently) conflicting fashions?
> >
> >
> >
> > See my anwers above. It's not only about different configs for different
> >
> > service.
> >
> >
> >
> > Thanks for the feedback!
> >
> >
> >
> > Cheers,
> >
> >
> >
> > Tom
> >
> >
> >
> > __________________________________________________________________________
> >
> > OpenStack Development Mailing List (not for usage questions)
> >
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > --
> Andrew Woodward

> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev