Open Stack

On 06/23/2014 10:30 AM, Monty Taylor wrote:
> On 06/23/2014 11:24 AM, Ben Nemec wrote:
>> On 06/23/2014 10:02 AM, Doug Hellmann wrote:
>>> On Mon, Jun 23, 2014 at 10:38 AM, Ben Nemec <openstack at nemebean.com> wrote:
>>> On 06/23/2014 08:41 AM, Julien Danjou wrote:
>>>>>> Hi there,
>>>>>>
>>>>>> We discovered a problem in pylockfile recently, and after
>>>>>> discussing with its current maintainer, it appears that more help
>>>>>> and workforce would be require:
>>>>>>
>>>>>> https://github.com/smontanaro/pylockfile/issues/11#issuecomment-45634012
>>>>>>
>>>>>>  Since we are using it via oslo lockutils module, I proposed to
>>>>>> adopt this project under the Oslo program banner. The review to
>>>>>> copy the repository to our infrastructure is up at:
>>>>>>
>>>>>> https://review.openstack.org/#/c/101911/
>>>
>>> We actually don't use this in lockutils - we use our own
>>> implementation of LockFile because there was some sort of outstanding
>>> bug in pylockfile that made it not work for us.  The only place I can
>>> see that we do use that project is in the oslo.db code because we
>>> didn't want to depend on incubator modules there, but once
>>> oslo.concurrency graduates we can switch to using our own locking
>>> implementation again.
>>>
>>> Basically I think this would be duplicating what we're already doing
>>> in lockutils, so I'm -1 on it.  I'd rather focus on getting
>>> oslo.concurrency graduated and remove pylockfile from
>>> global-requirements to make sure no one is using it anymore.
>>>
>>>> Which makes more sense, continuing to maintain our library or fixing
>>>> that bug and maintaining pylockfile? How big is pylockfile compared to
>>>> what we have? Does it solve problems our existing locking code doesn't
>>>> solve (and that we care about)?
>>
>> It looks to me like pylockfile would provide a subset of the
>> functionality in lockutils (for example, I don't see anything to replace
>> the @lock decorator).  So I don't think we could just drop lockutils and
>> switch to it.  We'd just be switching out the underlying lock mechanism,
>> and we know how well that has gone in the past. ;-)
> 
> But if we had originally thought to use pylockfile except for the bug,
> and if oslo.lockutils brings in oslo.config making it not suitable for
> general usage - it seems like it would be a good thing for the wider
> community if we 'fix' pylockfile and make oslo.lockutils the
> oslo-ification of it?
> 
> I mean, ultimately like everything else in OpenStack we don't REALLY
> want to just have our own set of parallel libs to what the rest of
> python uses, do we?

Fair point.  I guess I've just been scarred by the fact that every
single time we've tried to change the underlying lock mechanics in
lockutils we've found some edge case that gets broken.

But that said, we could limit the changes to lockutils by simply
contributing our lockfile code as an alternative implementation in
pylockfile (it looks like there are already several options there) and
using it from there instead.  Then it's more of a refactoring with the
side benefit that anyone else can use the code too, and we have the
option of using other pylockfile implementations if need be.  I could
get behind that, so consider my objection to adopting this withdrawn.

> 
>>>
>>>
>>> This also makes me wonder if oslo.concurrency should not be an oslo.*
>>> library since this functionality is more generally applicable outside
>>> OpenStack.  Something to discuss anyway.
>>>
>>>> That makes sense. When I made the list of libraries to release this
>>>> time, I called them all "oslo.foo" because I wasn't digging into the
>>>> code deep enough to figure out whether they could be something else. I
>>>> expected the person managing the spec for the release to do that step,
>>>> but I may not have made that clear.
>>>
>>>> The main thing I would be concerned with about using a non-oslo name
>>>> for oslo.concurrency is whether or not it uses another oslo library
>>>> like oslo.config. If we can completely avoid those dependencies, then
>>>> it should be safe to release it under a name other than
>>>> oslo.concurrency.
>>
>> Oh, that's probably why I didn't suggest this in the first place.
>> lockutils uses oslo.config, so it does need to be in the oslo namespace.
>>
>> I don't think we can drop the oslo.config dep, but we might be able to
>> decouple it like oslo.db did.  I think that would be messy though
>> because Windows is where problems would come up and we don't test
>> Windows in the gate. :-/
>>
>>>
>>>> Doug
>>>
>>>
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________ OpenStack-dev
>>>>>> mailing list OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>