Open Stack

On 12/04/2012 06:29 PM, Monty Taylor wrote:
>
> On 12/04/2012 07:04 AM, Mark McLoughlin wrote:
>> On Tue, 2012-12-04 at 15:43 +0100, Thierry Carrez wrote:
>>> Ionuț Arțăriși wrote:
>>>> On 12/04/2012 02:06 PM, Thierry Carrez wrote:
>>>>> In summary, I would hate it if we went back to the previous situation.
>>>>> I'm not personally attached to setuptools_git, but any proposed
>>>>> replacement solution should keep its simplicity.
>>>> Maybe a solution could be to add a Jenkins hook for re-generating the
>>>> MANIFEST.in file after each commit? Just like we currently have for
>>>> updating the contributors email list?
>>> I suppose that could work, although I'd wait on Monty's opinion before
>>> moving on that :)
>>>
>>> Note that it's not just MANIFEST.in that needs updating, but also the
>>> "scripts" entry in setup.py. Most developers just forgot to update it
>>> when they added a new executable under bin/... and it needs updating
>>> unless you rely on setuptools_git to handle that for you.
>>>
>>> Makes regenerating as a commit hook a bit more tricky (mix autogenerated
>>> with authored content in a single file).
>> (Excuse the hand-waving)
>>
>> If setuptools_git generates a bunch of info from git when creating a
>> tarball, you'd think it would include that info in the tarball itself.
>>
>> i.e. if setuptools_git replaces the need for MANIFEST.in to be kept up
>> to date, you'd expect that a generated MANIFEST.in would be included in
>> the generated tarball.
> Ok - I'm just going to respond down here instead of trying to respond to
> each of the suggestions (all of which I appreciate, btw)
>
> ttx is right, we did this INSTEAD of a job that did a tarball build and
> then compared the results of the tarball with the tree. You might think
> that people should care when they add stuff, but you'd be amazed at how
> wrong you would be.
>
> What markmc says is almost correct, except that it supplements
> MANIFEST.in to produce MANIFEST.
>
> The suggestion about the jenkins job is problematic because we would
> like for anyone to be able to re-build a tarball who isn't the openstack
> jenkins. We don't actually generate AUTHORS with Jenkins, it's done in
> setup.py directly.
>
> There are two potential bugs here. One is that we should probably not
> setup_requires setuptools_git. Lemme think about that one a bit more.
>
> For the other one, assuming that we can still meet our current usage
> patterns without injecting setuptools_git into setup_requires, can you
> please explain in more detail exactly what you are doing that is running
> in to problems? setuptools_git should only be affecting the manifest of
> things that go into the dist tarball. It should not have any effect on
> the build or install commands. So if you can explain a little bit more
> about your environment and exactly what you do so that I can reproduce
> it, I would be more than thrilled to work with you to help find a
> solution that works for all of us.
>
>

Actually setuptools_git *does* have an effect on the setup.py build and 
install commands. Here's how our two build methods differ:

This is building from git:

$ git clone git://github.com/openstack/nova.git
$ cd nova
$ python setup.py build
$ find build -name migrate.cfg
build/lib/nova/db/sqlalchemy/migrate_repo/migrate.cfg
$ grep migrate.cfg nova.egg-info/SOURCES.txt
nova/db/sqlalchemy/migrate_repo/migrate.cfg

This is a simplified version of what our rpm build tools do:

$ git clone git://github.com/openstack/nova.git
$ cd nova
$ rm -rf .git*
$ sed -i "/setup_requires=\['setuptools_git/d" setup.py
$ python setup.py build
$ find build -name migrate.cfg
$ grep migrate.cfg nova.egg-info/SOURCES.txt

The install command then just uses the filelist from the build command.

Thanks for helping out,
-Ionuț