Open Stack

This thread [1] has turned more “future focused", so I’m moving the conversation to the -dev list where we usually have those sorts of discussions.

[1] http://lists.openstack.org/pipermail/openstack/2014-September/009253.html

On Sep 17, 2014, at 7:54 AM, James Page <james.page at ubuntu.com> wrote:

> Signed PGP part
> Hi Li
> 
> On 17/09/14 11:58, Li Ma wrote:
> >> The scale potential is very appealing and is something I want to
> >> test - - hopefully in the next month or so.
> >>
> >> Canonical are interested in helping to maintain this driver and
> >> hopefully we help any critical issues prior to Juno release.
> >>
> >>
> > That sounds good. I just went through all the bugs reported in the
> > community.
> >
> > The only critical bug which makes ZeroMQ malfunction is
> > https://bugs.launchpad.net/oslo.messaging/+bug/1301723 and the
> > corresponding review is under way:
> > https://review.openstack.org/#/c/84938/
> 
> Agreed
> 
> > Others are tagged to 'zmq' in
> > https://bugs.launchpad.net/oslo.messaging
> 
> Looking through Doug's suggested list of information and collating
> what I know of from our work last week:
> 
> 1) documentation for how to configure and use zeromq with
> oslo.messaging (note, not the version in oslo-incubator, the version
> in the messaging library repository)
> 
> As part of our sprint, I worked on automating deployment of OpenStack
> + 0MQ using Ubuntu + Juju (service orchestration tool). I can re-jig
> that work into some general documentation on how best to configure
> ZeroMQ with OpenStack - the current documentation is a bit raw and
> does not talk about how to configure the oslo-messaging-zmq-receiver
> at all.
> 
> I also plan some packaging updates for Debian/Ubuntu in our next dev
> cycle to make this a little easier to configure and digest - for
> example, right now no systemd unit/upstart configuration/sysv init
> script is provided to manage the zmq-receiver.
> 
> I'd also like to document the current design of the ZMQ driver - Doug
> - where is the best place todo this? I thought in the source tree
> somewhere.

The documentation in the oslo.messaging repository [2] would be a good place to start for that. If we decide deployers/operators need the information we can either refer to it from the guides managed by the documentation team or we can move/copy the information. How about if you start a new drivers subdirectory there, and add information about zmq. We can have other driver authors provide similar detail about their drivers in the same directory.

[2] http://git.openstack.org/cgit/openstack/oslo.messaging/tree/doc/source

> 
> 2) a list of the critical bugs that need to be fixed + any existing
> patches associated with those bugs, so they can be reviewed early in kilo
> 
> This blocks operation of nova+neutron environements:
> 
> https://bugs.launchpad.net/oslo.messaging/+bug/1301723
> 	Summary: Message was sent to wrong node with zmq as rpc_backend
> 	Patch: https://review.openstack.org/84938
> 
> Also notifcations are effectively unimplemented which prevents use
> with Ceilometer so I'd also add:
> 
> https://bugs.launchpad.net/oslo.messaging/+bug/1368154
> 	Summary: https://bugs.launchpad.net/oslo.messaging/+bug/
> 	Patch: https://review.openstack.org/120745

That’s a good list, and shorter than I expected. I have added these bugs to the next-kilo milestone.

> 
> 3) an analysis of what it would take to be able to run functional
> tests for zeromq on our CI infrastructure, not necessarily the full
> tempest run or devstack-gate job, probably functional tests we place
> in the tree with the driver (we will be doing this for all of the
> drivers) + besides writing new functional tests, we need to bring the
> unit tests for zeromq into the oslo.messaging repository
> 
> Kapil Thangavelu started work on both functional tests for the ZMQ
> driver last week; the output from the sprint is here:
> 
>    https://github.com/ostack-musketeers/oslo.messaging
> 
> it covers the ZMQ driver (including messaging through the zmq-receiver
> proxy) and the associated MatchMakers (local, ring, redis) at a
> varying levels of coverage, but I feel it moves things in the right
> direction - Kapil's going to raise a review for this in the next
> couple of days.
> 
> Doug - has any structure been agreed within the oslo.messaging tree
> for unit/functional test splits? Right now we have them all in one place.

I think we will want them split up, but we don’t have an agreed existing structure for that. I would like to see a test framework of some sort that defines the tests in a way that can be used to run the same functional for all of the drivers as separate jobs (with appropriate hooks for ensuring the needed services are running, etc.). Setting that up warrants its own spec, because there are going to be quite a few details to work out. We will also need to participate in the larger conversation about how to set up those functional test jobs to be consistent with the other projects.

> 
> Edward Hope-Morley also worked on getting devstack working with ZMQ:
> 
>    https://github.com/ostack-musketeers/devstack
> 
> that's still WIP but again we'll get any changes submitted for review
> ASAP.

That’s good to have, but I don’t necessarily consider it a requirement for in-project functional tests.

> 
> 4) and some improvements that we would like to make longer term
> 
> a) Connection re-use on outbound messaging avoiding the current tcp
> setup overhead for every sent message.  This may also bring further
> performance benefits due to underlying messaging batching in ZMQ.

This sounds like it would be a good thing to do, but making what we have work correctly and testing it feels more important for now.

> 
> b) Moving from tcp PUSH/PULL sockets between servers to DEALER/DEALER
> (or something similar) to allow for heartbeating and more immediate
> failure detection

I would need to understand how much of a rewrite that represents before commenting further.

> 
> c) Crypto support

There are some other discussions about adding crypto to messaging, and I hope we can do that without having to touch each driver, if possible.

> 
> Cheers
> 
> James
> 
> --
> James Page
> Ubuntu and Debian Developer
> james.page at ubuntu.com
> jamespage at debian.org
>