Open Stack

On Mon, May 20, 2013 at 4:45 PM, Flavio Percoco <flavio at redhat.com> wrote:

> On 20/05/13 16:11 -0400, Doug Hellmann wrote:
>
>>   On Mon, May 20, 2013 at 12:48 PM, Flavio Percoco <[1]flavio at redhat.com>
>>
>>   wrote:
>>
>>   The problem with that drivers specify all sorts of different options,
>>   so we can't have any sort of consistent API for creating a Transport.
>>   We need the constructors for driver classes to all take the same
>>   parameters, and for those parameters to encode everything about the
>>   transport -- no peeking at the config from within the driver -- so that
>>   applications that need to connect to multiple transports and targets
>>   (ceilometer, cells, etc.) can pass the required parameter(s) to create
>>   a new connection no matter which driver is being used.
>>   IOW, instead of eliminating the URL option, the goal is to eliminate
>>   *all of the other* driver-specific options, and encode their values in
>>   the URLs. That leaves deployers able to configure everything that they
>>   can today, while Oslo only has to know about the "scheme" portion of
>>   the URL to figure out which driver to load. The drivers are also then
>>   free to interpret the other fields of the URL in whatever way makes the
>>   most sense for the driver.
>>   For backwards compatibility, we would obviously need to have a way to
>>   read the existing configuration format and create the URLs to pass to
>>   the drivers, but that's not difficult.
>>
>
> But all drivers have a conf instance, right? I was thinking that using
> that conf instance, drivers could register their options and read them
> from there. Drivers constructors would be consistent, no custom args /
> kwargs, everything read from the config instance.
>

A single conf instance does not solve the problem of connecting to multiple
transports that need different configurations. For example, in ceilometer,
we listen to notifications on the exchanges for all of the other projects.
Right now we create separate connections where only the exchange varies.
However, we would prefer for ceilometer not to know that exchanges are a
meaningful concept to the driver at all, since they *aren't* for some
drivers. In the case of cells, the IP where the exchange is located also
changes (assuming each cell has its own rabbit/qpid cluster). That might be
the case for ceilometer, as well, if a single ceilometer cluster is being
used to collect data about several cells (or even deployments).

Eric will need to weigh in, but I think this is especially a problem with
0mq because of the exchange vs. port question.

One thing we explored early on was having separate configuration sections
for each connection, and reusing the same option names. The config parser
doesn't like that, and I'm not really a fan either. It also doesn't solve
the problem for cells, which need to store the connection parameters
somewhere other than the config file. A single URL specifying all of the
connection settings supports all of the use cases.


>
> More complex drivers will, most likely, have to specify custom options
> anyway, I doubt we'll be able to completely get rid of those. My
> concern is that those "more complex" drivers will have to create a
> custom connection url - comma separate netlocs for qpid's case for
> example - which, IMHO, makes the implementation less cleaner and
> deployers will have to know how to encode that URL with the accepted
> parameters anyway.
>

Yes, I agree, that's a problem we need to solve. I haven't done an
exhaustive analysis, but it seems like anything set in an individual option
now could potentially be made a query parameter. I could see some of the
drivers that have required settings using the "path" in the URL for those.
For example, an rabbit URL might look like:

  rabbitmq://user:password@host:port/exchange?param_only_rabbit_uses=value

QPid might do something like:

  qpid://user:password@primary_host
:port/exchange?alternate_host=ip&alternate_host=ip

or

  qpid://user:password@primary_host:port/exchange?alternate_host=ip,ip

This is very similar to the way we're already specifying the storage
settings in ceilometer.

If there are driver-specific parameters that control how the driver works
but not where it talks (timeouts, etc.) those could conceivably be kept as
separate options. I would prefer, though, that drivers not define options
at all because it introduces a dependency between the driver and the config
library that isn't really necessary.

Doug


>
> I see the benefit of having a "common" url but we might be
> complicating things instead of simplifying them, configuration wise.


> Thoughts?
>
> FF
>
>
> --
> { name: "Flavio Percoco",
>   gpg: "87112EC1",   internal: "8261386",
>   phone: "+390687502386",
>   irc: ["fpercoco", "flaper87"]}
>
> ______________________________**_________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.**org <OpenStack-dev at lists.openstack.org>
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev<http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130520/15908019/attachment.html>