Open Stack

In-case folks are not following comments on all the various forums for this
discussion. We've got changes up to address the concerns raised so far on
the immediate problem:

Devstack (changing default config option to be shared):
https://review.openstack.org/341744
Cinder (release note):  https://review.openstack.org/354501
Nova (release note):  https://review.openstack.org/354502
oslo.concurrency (updated config description):
https://review.openstack.org/355269

For addressing concerns like

I have enough experience to know that the notes will not be read.


Any suggestions on where we should document it? I am also not entirely
convinced a release note is the best, especially since it isn't really new
for this release, and may be a thing for future ones too. Theoretically
this has been a problem since Cinder and Nova split off way back when, any
of the volume attach/detach operations have been at risk for this when run
on the same host. With os-brick in liberty and its named locks we got a
mechanism to prevent it, but this wasn't really documented anywhere (or
known to be an issue, as it turns out). I'm not sure what the normal
strategy is to give a heads up to folks using older releases that there
might be issues like this. Suggestions welcome.

Clint Byrum wrote:
>
>> Excerpts from Joshua Harlow's message of 2016-08-13 20:04:13 -0700:
>>
>>> The larger issue here IMHO is that there is now a<between-project>  API
>>> around locking that might be better suited targeting an actual lock
>>> management system (say redis or zookeeper or etcd or ...).
>>>
>>
>> The more I look at this, the more I think this is just evidence that
>> the compute node itself needs to be an API unto itself. Whether it's
>> Neutron agents, cinder volumes, or what, nova-compute has a bunch of
>> under-the-covers interactions with things like this. It would make more
>> sense to put that into its own implementation behind a real public API
>> than what we have now: processes that just magically expect to be run
>> together with shared filesystems, lock dirs, network interfaces, etc.
>>
>> That would also go a long way to being able to treat the other components
>> more like microservices.
>>
>>
> I very much agree, the amount of interactions 'under-the-covers' makes it
> really hard to do many things (including understanding what those
> interactions even are). For example, how does someone even install
> 'os-brick' at this point, if it requires as a prerequisite that cinder and
> nova-compute be pre-setup with the <same lock dir>? Sucks I guess for
> people/operators/anyone using both components, that are already running
> those with different lock directories...
>
> IMHO the amount of time done 'hacking in solutions' like a shared lock
> directory (or moving both projects to share the same configuration somehow)
> would be better spent on an actual locking solution/service and thinking
> about microservices and ... but meh, what can u do...


I like the sound of a more unified way to interact with compute node
services. Having a standardized approach for inter-service synchronization
for controlling system resources would be sweet (even if it is just a more
sane way of using local file locks). Anyone know if there is existing work
in this area we can build off of? Or is the path forward a new
cross-project spec to try and lock down some requirements, use-cases, etc.?

As far as spending time to hack together solutions via the config settings
for this.. we'll its pretty minimal wrt size of effort compared to solving
the large issue. Don't get me wrong though, I'm a fan of doing both in
parallel. Even if we have resources jump on board immediately I'm not
convinced we have a great chance to "fix" this for N in a more elegant
fashion, much less any of the older releases affected by this. That leads
me to believe we still need the shared config setting for at least a little
while in Devstack, and documentation for existing deployments or ones going
up with N.

-Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160814/07ef8c30/attachment.html>