Open Stack

Currently, the majority of OpenStack components make use of the Python
module "setuptools_git" in order to install additional configuration
files. This is basically the same functionality that the MANIFEST.in
file (setuptools/distribute) provides, but automatic. However, we
recently discovered that this approach has issues from a packaging
perspective. We weren't getting all the data/config files that the
python package usually gets even though we were running the same commands:

    $ python setup.py build

followed by:

    $ python setup.py install --skip-build

We are building RPM packages from release tarballs (such as [1]), which
of course don't include the .git directory. Therefore the setuptools_git
approach can't do its magic, thus our package builds get wrong results.
Having OpenStack components rely on setuptools_git at build time means
we have to distribute the whole git repository along with the source
code tarball. Of course this makes no sense, since it would increase the
size of release tarballs dramatically and won't get shipped in
distributions anyway.Therefore, we (and potentially other distribution
vendors) would have to track these files manually in our RPM spec files.
Some reviews have already been opened on the topic (admittedly before we
discovered the real issue). Given the different outcome of each review
it seems that not everybody is aware that setuptools_git is used or of
what it does.

https://review.openstack.org/#/c/17122/ (ceilometer) - this one got
accepted before we knew what was going on

https://review.openstack.org/#/c/17347/ (cinder) - abandoned until the
situation is clarified

https://review.openstack.org/#/c/17355/ (nova) - rejected

So the better solution would be to stop using setuptools_git and just
include all the data/config files that are meant to be distributed in
the MANIFEST.in file. This is what every Python developer should know
about and has the benefit of increased transparency about what gets
installed and what not. We created a blueprint to track this [2].

Thoughts?

Footnotes:
 [1]
https://launchpad.net/cinder/grizzly/grizzly-1/+download/cinder-2013.1%7Eg1.tar.gz
 [2] https://blueprints.launchpad.net/nova/+spec/drop-setuptools-git

Note: Resend after first sending (wrongly) to openstack@

-- 
With kind regards,
Sascha Peilicke
SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121204/295d6697/attachment.pgp>