[openstack-dev] [tc][pbr][packaging][all] Definition of Data Files (config) in setup.cfg

Monty Taylor mordred at inaugust.com
Sun Jun 12 20:45:07 UTC 2016

On 06/11/2016 10:13 PM, Robert Collins wrote:
> On 12 June 2016 at 11:31, Thomas Goirand <zigo at debian.org> wrote:
>> On 06/11/2016 05:02 AM, Robert Collins wrote:
> ...
>> Then keystone.conf ends up installed in /usr/etc/keystone when setup.py
>> install is used. With this configuration, it would be installed
>> correctly within a venv.
> Except that noone uses 'setup.py install' - *except* for rpm and dpkg
> packagers. Everyone else uses pip, and pip won't pass new options.
> Worse, pip creates wheels and then installs the wheels, and until PEP
> 491 [1] is finalised and implemented, wheels won't change their
> behaviour.
>> If instead we have:
>> [files]
>> data_files =
>>     /etc/keystone/keystone.conf = etc/keystone.conf.sample
>> [ notice the leading / for the destination... ]
>> Then keystone.conf gets installed on the root of the filesystem instead
>> of properly be installed within the venv. Though it's installed properly
>> for distributions in /etc/keystone.
>> To address this problem, Julien Danjou attempted to add a --sysconfdir
>> option in PBR. Though it was rejected by Robert, who wrote in the code
>> review that it should be addressed within setuptools or disutils.
> Not should. Must be. A local change to pbr won't solve the problem,
> except for the very narrow case of package creators (who already have
> excellent tools - its a single one-line file to fix it on Debian, and
> I presume roughly the same on RPM distributions). It certainly won't
> solve it for virtual envs.
>> If I got the above facts wrong, feel free to let everyone know. Now, my
>> reply to Robert...
>> === End of context ===
>> Robert,
>> I do not agree that we *must* wait until politics are discussed and a
> I'm going to quote what I said back in February - 4 months ago: "Ok,
> so here's where I got to:
> - wheel doesn't support or understand /etc per se
> - setuptools and distutils disagree on the meaning of data files paths
> - wheel takes the distutils value today
> - pep 491, which seems moribund, is an attempt to explicitly support
> sysconf dir's like /etc, and thats what is needed in general.
> I think the way forward to get this available as a sensible,
> supportable feature is to discuss it on the distutils-sig list, and
> that community can help figure out a design, including the backwards
> compat implications."
> I don't -2 lightly, and I did considerable research into this patch
> before leaving the -2 in place.
> With regards to politics and consensus in the upstream packaging
> community - well, it would move faster if folk who are distro folk and
> thus want to see the mess sorted were collaborating there: its been 4
> months since Julien's original patch and no discussion has taken
> place, nor has anyone volunteered in that community to do the work.
> Its not super hard, and I'd be happy to guide anyone interested in
> reviving PEP 491 (or drafting an alternative). What I'm not interested
> in doing is adding an ineffective hack to pbr which we'll have to
> maintain for an extended period, which isn't actually in pbr's domain
> (pbr exists to prevent duplication amongst OpenStack projects and
> provide solid and sane policy, not to replace wheel and setuptools).
> I'm sorry if that sounds like a hard line, but we had an ongoing
> series of bugs in pbr - the gift that keeps on giving - from the
> previous attempts to work around defects upstream of pbr rather than
> fixing those defects directly, which is a much better strategy : less
> cost, less technical debt, benefits the entire community, gets better
> outcomes.

I'm going to have to second what Robert says here.

Also - I would like to add the following.

Having pip or setup.py install install a config file is at best a
strange thing to want, as it does not have the mechanisms to deal with
templating in values, or dealing with conflicts with locally changed
file values on upgrades.

Consider the case where a person does "pip install keystone" to install

If that command installs keystone.conf into /etc/keystone - there is
_NO_ way that that config file will have any useful values. The user
will then edit the file to add the values into the file that need to be
added. Then, the user will, at a later point in time do "pip install -U
keystone" to upgrade the keystone installation.

What does pip do? Does it overwrite /etc/keystone/keystone.conf? What
about the user's values? Does it NOT overwrite
/etc/keystone/keystone.conf? Why not?

This is, btw, why dpkg and rpm and friends exist. It's their job.

Now, you might argue, there are files, such as the paste config files
which are configs that the software needs to install to be useful.

I will call hogwash on that. That we install files such as that which
are actually code has been a deployer nightmare since day one, and the
solution is not to make pip smarter, it's to stop installing such files.

NOW - if it's quite literally a file that just exists better as a file
and not direct python code - that's fine - just put it into the package
itself and then load it either by relative paths from the package or via
pkg_resources. For instance, one could put paste.conf into
/usr/local/lib/python2.7/site-packages/keystone/paste.ini instead of
into etc.

Which is my way of saying:

- If it's a file that one expects the user to edit, having pip/setup.py
install it is a bad idea
- If it's a file that one does NOT expect the user to edit, put it into
the package.

There is, of course, a class of files that meet neither condition:
manpages. Yup. Manpage handling is bong. But we already have as much
special casing in pbr as I think is reasonable to do the best we can
with manpages. That best is crappy anyway, which brings us back to
Robert's point about fixing things in distutils/setuptools.

But seriously - figuring out _why_ we want pip to be able to deal with
config files I think would be much better served by reducing the amount
of boilerplate config we expect users to plop down. No amount of code in
pbr is going to help that problem.


More information about the OpenStack-dev mailing list