Open Stack

On 08/27/2014 05:52 PM, Doug Hellmann wrote:
> 
> On Aug 27, 2014, at 11:14 AM, Flavio Percoco <flavio at redhat.com> wrote:
> 
>> On 08/27/2014 04:31 PM, Sean Dague wrote:
>>> So this change came in with adding glance.store -
>>> https://review.openstack.org/#/c/115265/5/lib/glance, which I think is a
>>> bad direction to be headed.
>>>
>>> Here is the problem when it comes to working with code from git, in
>>> python, that uses namespaces, it's kind of a hack that violates the
>>> principle of least surprise.
>>>
>>> For instance:
>>>
>>> cd /opt/stack/oslo.vmware
>>> pip install .
>>> cd /opt/stack/olso.config
>>> pip install -e .
>>> python -m olso.vmware
>>> /usr/bin/python: No module named olso.vmware
>>>
>>> In python 2.7 (using pip) namespaces are a bolt on because of the way
>>> importing modules works. And depending on how you install things in a
>>> namespace will overwrite the base __init__.py for the top level part of
>>> the namespace in such a way that you can't get access to the submodules.
>>>
>>> It's well known, and every conversation with dstuft that I've had in the
>>> past was "don't use namespaces".
>>>
>>> A big reason we see this a lot is due to the fact that devstack does
>>> 'editable' pip installs for most things, because the point is it's a
>>> development environment, and you should be able to change code, and see
>>> if live without having to go through the install step again.
>>>
>>> If people remember the constant issues with oslo.config in unit tests 9
>>> months ago, this was because of mismatch of editable vs. non editable
>>> libraries in the system and virtualenvs. This took months to get to a
>>> consistent workaround.
>>>
>>> The *workaround* that was done is we just gave up on installing olso
>>> libs in a development friendly way. I don't consider that a solution,
>>> it's a work around. But it has some big implications for the usefulness
>>> of the development environment. It also definitely violates the
>>> principle of least surprise, as changes to oslo.messaging in a devstack
>>> env don't immediately apply, you have to reinstall olso.messaging to get
>>> them to take.
>>>
>>> If this is just oslo, that's one thing (and still something I think
>>> should be revisted, because when the maintainer of pip says "don't do
>>> this" I'm inclined to go by that). But this change aims to start brining
>>> this pattern into other projects. Realistically I'm quite concerned that
>>> this will trigger more work arounds and confusion.
>>>
>>> It also means, for instance, that once we are in a namespace we can
>>> never decide to install some of the namespace from pypi and some of it
>>> from git editable (because it's a part that's under more interesting
>>> rapid development).
>>>
>>> So I'd like us to revisit using a namespace for glance, and honestly,
>>> for other places in OpenStack, because these kinds of violations of the
>>> principle of least surprise is something that I'd like us to be actively
>>> minimizing.
>>
>>
>> Sean,
>>
>> Thanks for bringing this up.
>>
>> To be honest, I became familiar with these namespace issues when I
>> started working on glance.store. That said, I realize how this can be an
>> issue even with the current "workaround".
>>
>> Unfortunately, it's already quite late in the release and starting the
>> rename process will delay Glance's migration to glance.store leaving us
>> with not enough time to test it and make sure things are working as
>> expected.
>>
>> With full transparency, I don't have an counter-argument for what you're
>> saying/proposing. I talked to Doug on IRC and he mentioned this is
>> something that won't be fixed in py27 so there's not even hope on seeing
>> it fixed/working soon. Based on that, I'm happy to rename glance.store
>> but, I'd like us to think in a good way to make this rename happen
>> without blocking the glance.store work in Juno.
> 
> When you asked me about namespace packages, I thought using them was fine because we had worked out how to do it for Oslo and the same approach worked for you in glance. I didn’t realize that was still considered an unresolved issue, so I apologize that my mistake has ended up causing you more work, Flavio.
> 

NP. Good thing is we now have a plan and a reference for future cases if
there happen to be any.

You've been way too helpful in this whole process. Thanks for all your
guidance and support, really.

>>
>> I've 2 ways to make this happen:
>>
>> 1. Do a partial rename and then complete it after the glance migration
>> is done. If I'm not missing anything, we should be able to do something
>> like:
>> 	- Rename the project internally
>> 	- Release a new version with the new name `glancestore`
>> 	- Switch glance over to `glancestore`
>> 	- Complete the rename process with support from infra
> 
> This seems like the best approach. If nothing is using the library now, all of the name changes would need to happen within the library. You can release “glancestore” from a git repo called “glance.store” and we can rename that repo somewhere down the line, so you shouldn’t be blocked on the infra team (who I expect are going to be really busy keeping an eye on the gate as we get close to feature freeze).
> 
> Was there anything else you were worried about being blocked on by the rename?

My only concern was related to the time the rename would required and
the fact that we don't want glancestore to miss Juno. I'll start working
on the above plan today. As soon as I have the internal rename done,
I'll propose a patch to infra. Hopefully, I'll manage to do all that
before the end of friday so that we can rename the project on satureday.
Otherwise, we can just delay the repo/gerrit rename until we have enough
time for it.

I'll keep you all posted.

> 
> We also need to figure out how to avoid problems like this in the future. We can’t expect everyone to be intimately familiar with all of the work going on in every project, but we also want to find ways to detect these sorts of issues earlier so they don’t have as much impact on the schedule. I don’t have any good suggestions, beyond soliciting reviews, but even then it’s not always clear what proposals might be “controversial” enough to warrant that extra step.
> 

Since it's hard to keep up with everything, I'd recommend pinging people
from each team that might be affected by something before going ahead
with it.

Unfortunately, I didn't do that for glance.store. I did send a couple of
emails to the mailing list but again, there are too many emails and it's
easy to miss things.

I had pingged folks from different teams in other occasions and it
worked fine.

Thanks again,
Flavio


> Doug
> 
>>
>> 2. Let this patch land, complete Glance's switch-over using namespaces
>> and then do the rename all together.
>>
>> Do you have any other suggestion that would help avoiding namespaces
>> without blocking glance.store?
>>
>> Thanks,
>> Flavio
>>
>>
>> -- 
>> @flaper87
>> Flavio Percoco
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
@flaper87
Flavio Percoco