Open Stack

On 08/08/2013 09:04 AM, Sean Dague wrote:
> On 08/08/2013 07:33 AM, Boris Pavlovic wrote:
>> Hi Mark,
>>
>>>> Sounds good. Just needs someone willing to implement it.
>>
>> This is very interesting for us.
>> What do you think if we create a fake project that use submodules as an
>> example, and then just discuss it?
> 
> How this is going to work in the gate is at least as important as how it
> works on developer's environments. For all the warts that the oslo
> process has, it does give us very fine grained control of which pieces
> come and land in projects, and when they do, and that it will pass the
> gate, and that they won't conflict with slightly different versions of
> oslo in other projects. Oslo is like our own venv for our own code.
> 
> So you really need to demonstrate the workflow in the CI system as well,
> with more than one project consuming the oslo bits at different levels.
> After the last 3 weeks unwinding our pip requires interrelations between
> projects, having another way in which we have interconnections between
> projects that has to be managed differently makes me kind at least a
> little nervous.
> 
> But, prototypes are good. It will at least let us discuss the
> practicalities of this instead of the theoreticals.

I believe zuul would also need to have knowledge of submodules added to
it before this would work.

The other issue that one will run in to with submodules is the
auto-module-renaming that update.py does. If the code in oslo-incubator
is openstack/common/foo.py and that's in nova as a submodule, then the
path in nova is going to be nova/openstack/common/foo - so we'd have to
start doing explicit relative paths in openstack/common modules.

GRANTED - the new version of pep8 calls this out as being a thing that's
ok - but it's another thing that's different from how we're operating
right now.

I have folks internally at HP just starting to do submodules in gerrit,
so I'd love it if we solved the general engineering issue. However - I
honestly am not sure that it's worth OpenStack's time and energy right
now given the amount of work I believe it's going to take. I'm also
concerned about doing it at this point in the cycle - it will be a
radically different dev workflow, and it will be a change to the zuul
merger pretty close to feature freeze. If it's something that we feel
strongly about - how about the POC is put together (and we can certainly
talk about the zuul changes that would be needed) and look at it at the
summit in detail. If we like where it's going, we can try to land it
early next cycle so that if we find places where it's crazy, they happen
at a time when we can handle the crazy better.

>> Best regards,
>> Boris Pavlovic
>> -- 
>> Mirantis Inc.
>>
>>
>>
>> On Thu, Aug 8, 2013 at 3:21 PM, Mark McLoughlin <markmc at redhat.com
>> <mailto:markmc at redhat.com>> wrote:
>>
>>     On Thu, 2013-08-08 at 15:11 +0400, Boris Pavlovic wrote:
>>      > Mark,
>>      >
>>      >
>>      > >> What do you mean by "dangerous code merging" in the subject?
>> The
>>      > body of
>>      > >> your mail doesn't make any reference to whatever "danger"
>> you're
>>      > seeing.
>>      >
>>      >
>>      >
>>      > I mean that cut and paste approach is really unsafe. For
>> example some
>>      > new member is able to change oslo code directly during syncing
>> with
>>      > some project,
>>      >  and nobody will be able to catch such things.
>>      >
>>      >
>>      > I didn't catch any of such situation, but I saw a lot of
>> attempts to
>>      > change openstack/common/* directly.
>>      > (and it is really close situation..)
>>
>>     Got examples?
>>
>>     It's a reviewer's responsibility to enforce that openstack/commmon/*
>>     code is just synced from oslo-incubator without modifications, but
>>     there's always an element of trust in reviewing - you can't
>> completely
>>     guard against people doing dumb or nefarious things.
>>
>>     We could come up with some automation where the oslo-incubator git
>>     commit ID corresponding to each file is included in each project's
>> repo
>>     and a test checks that the file does in fact correspond to that
>> commit
>>     ID. Needs someone willing to implement it.
>>
>>      > >> The idea of using submodules has come a few times. I don't
>> have a
>>      > >> fundamental objection to it, except any time I've seen
>> submodules
>>      > used
>>      > >> in a project they've been extremely painful for everyone
>> involved.
>>      >
>>      >
>>      >
>>      > oslo-incubator sync util and submodules solves the same problem,
>>      > almost in same way:
>>      > sync util -> copy paste code from <hash>
>>      > submodules -> just set <hash> of commit from what to use code
>>      >
>>      >
>>      > So I think the problem is not in submodules, problem is in
>>     approach of
>>      > common code for different projects.
>>      > But IMHO it is much better to have problems around creating common
>>      > code that is used by all projects, then to make
>>      > N different solutions for N different projects doing almost the
>> same
>>      > things.
>>      >
>>      >
>>      >
>>      >
>>      > >> I'd be happy to look at a demo of a submodule based system for
>>      > projects
>>      > >> to use code from oslo-incubator.
>>      >
>>      >
>>      >
>>      > Probably we should just try, and analyze what approach is better?
>>
>>     Sounds good. Just needs someone willing to implement it.
>>
>>     Cheers,
>>     Mark.
>>
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
>