Open Stack

On 08/28/2014 12:34 PM, Doug Hellmann wrote:
> 
> On Aug 28, 2014, at 8:36 AM, Mark McLoughlin <markmc at redhat.com> wrote:
> 
>> On Thu, 2014-08-28 at 13:24 +0200, Flavio Percoco wrote:
>>> On 08/27/2014 03:35 PM, Ken Giusti wrote:
>>>> Hi All,
>>>>
>>>> I believe Juno-3 is our last chance to get this feature [1] included
>>>> into olso.messaging.
>>>>
>>>> I honestly believe this patch is about as low risk as possible for a
>>>> change that introduces a whole new transport into oslo.messaging.  The
>>>> patch shouldn't affect the existing transports at all, and doesn't
>>>> come into play unless the application specifically turns on the new
>>>> 'amqp' transport, which won't be the case for existing applications.
>>>>
>>>> The patch includes a set of functional tests which exercise all the
>>>> messaging patterns, timeouts, and even broker failover. These tests do
>>>> not mock out any part of the driver - a simple test broker is included
>>>> which allows the full driver codepath to be executed and verified.
>>>>
>>>> IFAIK, the only remaining technical block to adding this feature,
>>>> aside from core reviews [2], is sufficient infrastructure test coverage.
>>>> We discussed this a bit at the last design summit.  The root of the
>>>> issue is that this feature is dependent on a platform-specific library
>>>> (proton) that isn't in the base repos for most of the CI platforms.
>>>> But it is available via EPEL, and the Apache QPID team is actively
>>>> working towards getting the packages into Debian (a PPA is available
>>>> in the meantime).
>>>>
>>>> In the interim I've proposed a non-voting CI check job that will
>>>> sanity check the new driver on EPEL based systems [3].  I'm also
>>>> working towards adding devstack support [4], which won't be done in
>>>> time for Juno but nevertheless I'm making it happen.
>>>>
>>>> I fear that this feature's inclusion is stuck in a chicken/egg
>>>> deadlock: the driver won't get merged until there is CI support, but
>>>> the CI support won't run correctly (and probably won't get merged)
>>>> until the driver is available.  The driver really has to be merged
>>>> first, before I can continue with CI/devstack development.
>>>>
>>>> [1] https://blueprints.launchpad.net/oslo.messaging/+spec/amqp10-driver-implementation
>>>> [2] https://review.openstack.org/#/c/75815/
>>>> [3] https://review.openstack.org/#/c/115752/
>>>> [4] https://review.openstack.org/#/c/109118/
>>>
>>>
>>> Hi Ken,
>>>
>>> Thanks a lot for your hard work here. As I stated in my last comment on
>>> the driver's review, I think we should let this driver land and let
>>> future patches improve it where/when needed.
>>>
>>> I agreed on letting the driver land as-is based on the fact that there
>>> are patches already submitted ready to enable the gates for this driver.
>>
>> I feel bad that the driver has been in a pretty complete state for quite
>> a while but hasn't received a whole lot of reviews. There's a lot of
>> promise to this idea, so it would be ideal if we could unblock it.
>>
>> One thing I've been meaning to do this cycle is add concrete advice for
>> operators on the state of each driver. I think we'd be a lot more
>> comfortable merging this in Juno if we could somehow make it clear to
>> operators that it's experimental right now. My idea was:
>>
>>  - Write up some notes which discusses the state of each driver e.g.
>>
>>      - RabbitMQ - the default, used by the majority of OpenStack 
>>        deployments, perhaps list some of the known bugs, particularly 
>>        around HA.
>>
>>      - Qpid - suitable for production, but used in a limited number of 
>>        deployments. Again, list known issues. Mention that it will 
>>        probably be removed with the amqp10 driver matures.
>>
>>      - Proton/AMQP 1.0 - experimental, in active development, will
>>        support  multiple brokers and topologies, perhaps a pointer to a
>>        wiki page with the current TODO list
>>
>>      - ZeroMQ - unmaintained and deprecated, planned for removal in
>>        Kilo
>>
>>  - Propose this addition to the API docs and ask the operators list 
>>    for feedback
>>
>>  - Propose a patch which adds a load-time deprecation warning to the 
>>    ZeroMQ driver
>>
>>  - Include a load-time experimental warning in the proton driver
>>
>> Thoughts on that?
> 
> By "API docs" do you mean the ones in the oslo.messaging repository? Would it be better to put this information in the operator’s guide?

I was talking to Ken a little about this today and came up with
http://docs.openstack.org/icehouse/config-reference/content/configuring-rpc.html

That seems like a reasonable place to put information like this (in
fact, there's already some there about rabbit being the default).  I
wasn't sure exactly where those docs are generated from, so I suggested
he talk to Anne Gentle about it.

-Ben

> 
> Other than the question of where to put it, I definitely think this is the sort of guidance we should document, including through the logged warnings.
> 
> Doug
> 
>>
>> (I understand the ZeroMQ situation needs further discussion - I don't
>> think that's on-topic for the thread, I was just using it as example of
>> what kind of advice we'd be giving in these docs)
>>
>> Mark.
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>