Open Stack

Hello folks,

I was curious if there is an OpenStack project that would be a good example to follow as we convert Barbican over to oslo messaging. 

I've been examining existing OpenStack projects such as Ceilometer and Keystone to see how they are utilizing oslo messaging. These projects appear to be utilizing packages such as 'rpc' and 'notifier' from the oslo-incubator project. It seems that the oslo.messaging project's structure is different than the messaging structure of oslo-incubator (there are newer classes such as Transport now for example). Is there an example OpenStack project utilizing the oslo.messaging structure that might be better for us to follow?

The RPC approach of Ceilometer in particular seems well suited to Barbican's use case, so seems to be a good option for us to follow, unless there are better options folks can suggest.

Thanks,
John

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


________________________________________
From: Russell Bryant [rbryant at redhat.com]
Sent: Monday, December 02, 2013 8:08 PM
To: Dolph Mathews
Cc: Monty Taylor; OpenStack Development Mailing List (not for usage questions); openstack-tc at lists.openstack.org; cloudkeep at googlegroups. com; barbican at lists.rackspace.com
Subject: Re: [openstack-tc] [openstack-dev] Incubation Request for Barbican

On 12/02/2013 06:46 PM, Dolph Mathews wrote:
>
>
>
> On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant <rbryant at redhat.com
> <mailto:rbryant at redhat.com>> wrote:
>
>     On 12/02/2013 12:46 PM, Monty Taylor wrote:
>     > On 12/02/2013 11:53 AM, Russell Bryant wrote:
>     >>>  * Scope
>     >>>  ** Project must have a clear and defined scope
>     >>
>     >> This is missing
>     >>
>     >>>  ** Project should not inadvertently duplicate functionality
>     present in other
>     >>>     OpenStack projects. If they do, they should have a clear
>     plan and timeframe
>     >>>     to prevent long-term scope duplication.
>     >>>  ** Project should leverage existing functionality in other
>     OpenStack projects
>     >>>     as much as possible
>     >>
>     >> I'm going to hold off on diving into this too far until the scope is
>     >> clarified.
>     >
>     > I'm not.
>     >
>     > *snip*
>     >
>
>     Ok, I can't help it now.
>
>     >>
>     >> The list looks reasonable right now.  Barbican should put
>     migrating to
>     >> oslo.messaging on the Icehouse roadmap though.
>     >
>     > *snip*
>
>     Yeahhh ... I looked and even though rpc and notifier are imported, they
>     do not appear to be used at all.
>
>     >>
>     >>
>     http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires
>     >>
>     >> It looks like the only item here not in the global requirements is
>     >> Celery, which is licensed under a 3-clause BSD license.
>     >
>     > I'd like to address the use of Celery.
>     >
>     > WTF
>     >
>     > Barbican has been around for 9 months, which means that it does not
>     > predate the work that has become oslo.messaging. It doesn't even
>     try. It
>     > uses a completely different thing.
>     >
>     > The use of celery needs to be replaced with oslo. Full stop. I do not
>     > believe it makes any sense to spend further time considering a project
>     > that's divergent on such a core piece. Which is a shame - because I
>     > think that Barbican is important and fills an important need and I
>     want
>     > it to be in. BUT - We don't get to end-run around OpenStack project
>     > choices by making a new project on the side and then submitting it for
>     > incubation. It's going to be a pile of suck to fix this I'm sure, and
>     > I'm sure that it's going to delay getting actually important stuff
>     done
>     > - but we deal with too much crazy as it is to pull in a non-oslo
>     > messaging and event substrata.
>     >
>
>     Yeah, I'm afraid I agree with Monty here.  I didn't really address this
>     because I was trying to just do a first pass and not go too far into the
>     tech bits.
>
>     I think such a big divergence is going to be a hard sell for a number of
>     reasons.  It's a significant dependency that I don't think is justified.
>      Further, it won't work in all of the same environments that OpenStack
>     works in today.  You can't use Celery with all of the same messaging
>     transports as oslo.messaging (or the older rpc lib).  One example is
>     Qpid.
>
>
> I feel like I'm trying to read past the rant :) so I'd like to stop an
> ask for clarification on the exact argument being made. Is the *only*
> reason that celery should not be used in openstack because it is
> incapable of being deployed against amqp 1.0 brokers (i.e. qpid).
>
> I'm really trying to understand if the actual objection is over the use
> (or not) of oslo (which seems like something the TC should express an
> opinion on, if that's the case), but rather about limiting OpenStack's
> deployment options.

There are two big parts to this, I think.  One is techincal - a
significant portion of OpenStack deployments will not work with this
because Celery does not work with their deployed messaging architecture.
 See another reply in this thread for an example of someone that sees
the inability to use Qpid as a roadblock for an example.  This is
solvable, but not quickly.

The other is somewhat technical, but also a community issue.  Monty
articulated this well in another reply.  Barbican has made a conflicting
library choice with what every other project using messaging is using.
With the number of projects we have, it is in our best interest to
strive for consistency where we can.  Differences should not be
arbitrary.  The differences should only be where an exception is well
justified.  I don't see that as being the case here.  Should everyone
using oslo.messaging (or its predecessor rpc in oslo-incubator) be using
Celery?  Maybe.  I don't know, but that's the question at hand.  Ideally
this would have come up with a more broad audience sooner.  If it did,
I'm sorry I missed it.

--
Russell Bryant