Open Stack

 Flavio Percoco wrote:

> Still, I'd like us to learn from
> previous experiences and have a better plan for this driver (and
> future cases like this one).


Hi, all!

As one of a just joined ZeroMQ maintainers I have a growing plan of ZeroMQ
refactoring and development. At the most abstract view our plan is to
remove single broker and implement peer-2-peer model in the messaging
driver. Now exists a blueprint with this goal
https://blueprints.launchpad.net/oslo.messaging/+spec/reduce-central-broker.
I maintain a patch and a spec which I had inherited from Aleksey Kornienko.
For now this blueprint is the first step in the planning process. I believe
we can split this big work in a set of specs and if is needed in several
related blueprints. With these specs and BPs our plan should become
obvious. I wrote a mail in the dev mail list with short overview to the
coming work.

Please, feel free to discuss it all with me and correct me on this big road.

On Mon, Nov 17, 2014 at 4:45 PM, Doug Hellmann <doug at doughellmann.com>
wrote:

> Thanks, Josh, I’ll subscribe to the issue to keep up to date.
>
> On Nov 16, 2014, at 6:58 PM, Joshua Harlow <harlowja at outlook.com> wrote:
>
> > I started the following issue on kombu's github page (to see if there is
> any interest on there side to such an effort):
> >
> > https://github.com/celery/kombu/issues/430
> >
> > It's about seeing if the kombu folks would be ok with a 'rpc' subfolder
> in there repository that can start to contain 'rpc' like functionality that
> now exists in oslo.messaging (I don't see why they would be against this
> kind of idea, since it seems to make sense IMHO).
> >
> > Let's see what happens,
> >
> > -Josh
> >
> > Doug Hellmann wrote:
> >>
> >> On Nov 13, 2014, at 7:02 PM, Joshua Harlow <harlowja at yahoo-inc.com
> >> <mailto:harlowja at yahoo-inc.com>> wrote:
> >>
> >>> Don't forget my executor which isn't dependent on a larger set of
> >>> changes for asyncio/trollious...
> >>>
> >>> https://review.openstack.org/#/c/70914/
> >>>
> >>> The above will/should just 'work', although I'm unsure what thread
> >>> count should be by default (the number of green threads that is set at
> >>> like 200 shouldn't be the same number used in that executor which uses
> >>> real python/system threads). The neat thing about that executor is
> >>> that it can also replace the eventlet one, since when eventlet is
> >>> monkey patching the threading module (which it should be) then it
> >>> should behave just as the existing eventlet one; which IMHO is pretty
> >>> cool (and could be one way to completely remove the eventlet usage in
> >>> oslo.messaging).
> >>
> >> Good point, thanks for reminding me.
> >>
> >>>
> >>> As for the kombu discussions, maybe its time to jump on the #celery
> >>> channel (where the kombu folks hang out) and start talking to them
> >>> about how we can work better together to move some of our features
> >>> into kombu (and also depreciate/remove some of the oslo.messaging
> >>> features that now are in kombu). I believe
> >>> https://launchpad.net/~asksol is the main guy there (and also the main
> >>> maintainer of celery/kombu?). It'd be nice to have these
> >>> cross-community talks and at least come up with some kind of game
> >>> plan; hopefully one that benefits both communities…
> >>
> >> I would like that, but won’t have time to do it myself this cycle. Maybe
> >> we can find another volunteer from the team?
> >>
> >> Doug
> >>
> >>>
> >>> -Josh
> >>>
> >>> <https://launchpad.net/~asksol>
> >>>
> ------------------------------------------------------------------------
> >>> *From:* Doug Hellmann <doug at doughellmann.com
> >>> <mailto:doug at doughellmann.com>>
> >>> *To:* OpenStack Development Mailing List (not for usage questions)
> >>> <openstack-dev at lists.openstack.org
> >>> <mailto:openstack-dev at lists.openstack.org>>
> >>> *Sent:* Wednesday, November 12, 2014 12:22 PM
> >>> *Subject:* [openstack-dev] [oslo] oslo.messaging outcome from the
> summit
> >>>
> >>> The oslo.messaging session at the summit [1] resulted in some plans to
> >>> evolve how oslo.messaging works, but probably not during this cycle.
> >>>
> >>> First, we talked about what to do about the various drivers like
> >>> ZeroMQ and the new AMQP 1.0 driver. We decided that rather than moving
> >>> those out of the main tree and packaging them separately, we would
> >>> keep them all in the main repository to encourage the driver authors
> >>> to help out with the core library (oslo.messaging is a critical
> >>> component of OpenStack, and we’ve lost several of our core reviewers
> >>> for the library to other priorities recently).
> >>>
> >>> There is a new set of contributors interested in maintaining the
> >>> ZeroMQ driver, and they are going to work together to review each
> >>> other’s patches. We will re-evaluate keeping ZeroMQ at the end of
> >>> Kilo, based on how things go this cycle.
> >>>
> >>> We also talked about the fact that the new version of Kombu includes
> >>> some of the features we have implemented in our own driver, like
> >>> heartbeats and connection management. Kombu does not include the
> >>> calling patterns (cast/call/notifications) that we have in
> >>> oslo.messaging, but we may be able to remove some code from our driver
> >>> and consolidate the qpid and rabbit driver code to let Kombu do more
> >>> of the work for us.
> >>>
> >>> Python 3 support is coming slowly. There are a couple of patches up
> >>> for review to provide a different sort of executor based on greenio
> >>> and trollius. Adopting that would require some application-level
> >>> changes to use co-routines, so it may not be an optimal solution even
> >>> though it would get us off of eventlet. (During the Python 3 session
> >>> later in the week we talked about the possibility of fixing eventlet’s
> >>> monkey-patching to allow us to use the new eventlet under python 3.)
> >>>
> >>> We also talked about the way the oslo.messaging API uses URLs to get
> >>> some settings and configuration options for others. I thought I
> >>> remembered this being a conscious decision to pass connection-specific
> >>> parameters in the URL, and “global” parameters via configuration
> >>> settings. It sounds like that split may not have been implemented as
> >>> cleanly as originally intended, though. We identified documenting URL
> >>> parameters as an issue for removing the configuration object, as well
> >>> as backwards-compatibility. I don’t think we agreed on any specific
> >>> changes to the API based on this part of the discussion, but please
> >>> correct me if your recollection is different.
> >>>
> >>> We also learned that there is a critical bug [2] related to heartbeats
> >>> for RabbitMQ, and we have a few patches up to propose fixes in
> >>> different ways (see the bottom of [1]). This highlighted again the
> >>> fact that we have a shortage of reviewers for oslo.messaging.
> >>>
> >>> Doug
> >>>
> >>> [1] https://etherpad.openstack.org/p/kilo-oslo-oslo.messaging
> >>> [2] https://bugs.launchpad.net/nova/+bug/856764
> >>>
> >>>
> >>> _______________________________________________
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev at lists.openstack.org
> >>> <mailto:OpenStack-dev at lists.openstack.org>
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>> _______________________________________________
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev at lists.openstack.org
> >>> <mailto:OpenStack-dev at lists.openstack.org>
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > --
> > Sent with Postbox <http://www.getpostbox.com>
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141117/7248f7df/attachment.html>