Open Stack

On Fri, Sep 9, 2011 at 1:45 PM, Brian Lamar <brian.lamar at rackspace.com> wrote:
> Topic #1 - Separating API projects from OpenStack Nova. I would push for separate projects for the EC2-compatible API, the Rackspace-compatible API, and the OpenStack Official API. Nova is not the place to stick things which deal with cross-project public APIs. Nova, in my opinion, needs to be more focused and not so much of the place where everything gets put.

Not sure I would agree with adding a separate project for the
OpenStack official API, which I assume you are actually referring to
the *Compute* API, not all OpenStack APIs (Identity, Compute, Images,
Objects, etc..). The other two, which are pure translation shims, I
think could be extensions that live either in an extensions repo or in
an extensions folder in Nova...

> Topic #2 - Standardizing on and documenting programmatic interfaces to each OpenStack project. This means that the Compute API would be documented with all methods/input parameters so that it could be implemented in any other language. This creates clean coupling points for the future and so higher-level APIs can be sufficiently stable. If Topic #1 were to become a reality, we would absolutely need to depend on the interfaces of the Compute/Volume/Network APIs. If you're looking at the current Compute API [1] you'll see that it might need a little cleanup for consistency and coherency.

I've been saying this for over a year now, that the API and the
implementation should be separated entirely, including having the PPB
(or some other focused group) vote *separately* on the API of a layer
separately from the implementation of that layer. This would make it
much easier for competing implementations to be written and "blessed"
as OpenStack compliant. The Object Storage API could then have
non-Python implementations, such as what has been done by the folks at
GlusterFS and is being worked on for two other storage
systems/vendors. Likewise, if Rackspace IT or some other organization
(just using Rackspace IT as an example since I recently spoke with
them.. ;) ) have a Java implementation of the Images API, they could
go ahead and use that without worrying that their implementation would
break other parts of OpenStack. In addition, these folks could propose
their implementation for inclusion into "core Openstack".

That said, if you separate out the API from the implementation, the
idea of "core OpenStack" might be able to change from "a set of mostly
Python projects (because that's what things were implemented in...)"
to "a set of projects that implement the core OpenStack APIs".

Another way to think of it: We don't say that RabbitMQ is "core AMQP".
We say that RabbitMQ "implements the AMQP standard". OpenStack should
be similar: a standard set of APIs for cloud components, with a set of
implementations of those APIs.

> Topic #3 - Use the database as a *cache* and nothing more. For a while we've talked about no-db-messaging and if you've ever followed Udi Dahan [2] or seen slides on CQRS [3] you can see we're eerily close to making a CQRS-ish system here. Maybe it's a buzz-word [4] architecture, but we can at least discuss the benefits/pitfalls of having the API have read-only access to the database and having the Managers have write-access to the database.

Sure, good topic for discussion.

> Topic #4 - As far as I know Glance doesn't follow some of the same patterns that Nova does. Nova pioneered the API/Manager paradigm and since Glance doesn't use AMQP it never really needed a Manager process. As such there is, from what I can tell, not a clear delineation between it's "internal" API (python) and it's "external" API (http). I feel these should be two distinct well-defined entities and as such we might look at making Glance follow some of the design principles found in Nova. This would also allow us to remove GlanceImageService from Nova.

Hmm, two things on this one. First, I agree that it would good to have
Glance have an internal API layer separate from its HTTP API. Brian
Waldon already started the work on this; the first step was to get the
output of the Controller classes to be non-HTTP encoded results. The
second step would be to remove the raising of HTTP-centric errors.

The second point, about this being able to remove GlanceImageService
from Nova, I'm not sure I buy that. I still think that, since the
Images API is an entirely separate endpoint from Nova (as it should
be, IMO, just like the Network (Quantum), IP Management (Melange),
Volume Management, etc).

Now, should Glance be able to respond on other protocols than HTTP?
Absolutely! :) Just not sure that doing so will remove the need for
the GlanceImageService (or *some* sort of connector code..) in Nova.

> Topic #5 - Standardize and document all messages which are sent to AMQP from each project. This has been brought up on the message list a couple times and I'm in agreement that we really need to document and standardize these messages so that other projects can potentially write to our queues.

++

Also, same point for outbound notification events. We need to codify
the event type codes as standaridized integer values that are in the
API documentation for Compute and Images. People shouldn't need to
grep through strings to figure out what type of event is being pushed.
An event code should be all that is needed for aggregation and
categorization.

> So basically I'd love to know if anyone has interest in any or all of these topics. If you're interested, want more information, or think think that my ideas are just bad feel free to respond to me or the list.

Great stuff, looking forward to participating in some of the above
sessions, Brian!

Cheers,
-jay