Open Stack

On Jul 8, 2011, at 6:06 AM, Soren Hansen wrote:

> 2011/7/7 Vishvananda Ishaya <vishvananda at gmail.com>:
>> On Jul 7, 2011, at 11:35 AM, Soren Hansen wrote:
>>> 2011/7/7 Trey Morris <trey.morris at rackspace.com>:
>>>> The goal isn't for ec2 api to be a "second class citizen", but to keep it
>>>> from being a limiting factor since we don't have control over it. How does
>>>> the compatibility layer make it second class?
>>> Well, for one thing because you'll be limiting the EC2 API to what the
>>> OpenStack API is capable of doing and/or expressing.
>> Actually, we can add any code that is necessary to make the ec2 api work properly as extensions to the os api.
> 
> Maybe it's functionality that we don't want anywhere near the OpenStack API?
> 
>> My main reason for suggesting the switch over to a compatibility layer is so that we don't have multiple code paths into the heart of nova.
> 
> Both frontend types will be speaking to the same backend API. That's
> where we translate the user's requests into actions that the backend
> can perform. This sounds like a very reasonable split to me. I don't
> understand why we'd change that. Building a backend that is reasonably
> frontend-API-agnostic is a good thing. It keeps us honest and helps
> maintain a clear separation of the various components. Eric did a lot
> of excellent work to make this split cleaner. I'd hate it if we
> blurred the lines between frontend and backend again.

Agreed

> 
>> This has been a pain point already, and it will only get more painful as we move forward.
> 
> IMO, reworking things to achieve looser coupling is tricky and
> painful, but time well spent.
> 
>>  If all the stuff that makes ec2 compatibility work is in one place at the top layer, it is easier to maintain.
> 
> I'll see your unsubstantiated claim, and raise you another one: It'll
> simply make other parts harder to maintain.

I'm willing to accept your hypothesis.  My inclination is to keep ec2 stuff in one place as much as possible, but I can't substantiate it with any real evidence.

> 
>> If we need to maintain entirely disparate code paths from the api all the way down to the hypervisor, things will continue to be very fragile.
> 
> It's not at all down the hypervisor. Not even close. The (currently)
> two different API frontends will interact with the relevant services
> (network, compute or scheduler), all of which should have a
> well-defined, flexible API. So far, these API's have been extended on
> a very ad-hoc basis to support some new functionality in either of the
> frontend API's. Just to be clear: I'm guilty of this too!  What we
> should be doing is defining sensiblem coherent API's for these backend
> services, document their interfaces and treat them with respect, be
> aware when/how they change, attempt to maintain backwards/forwards
> compatiblity.  When we start to think about how to upgrade Nova
> deployments, this will be very helpful. Having different things
> talking to the same API's keeps us more honest in this regard.

Agreed

> 
> So, I feel the exact opposite. A lot of the current fragility of Nova
> stems from the fact that we've focused a lot on specific features and
> not very much on these grander, more holistic things. When something
> is tightly coupled, and you try to bolt something else onto it, yes,
> it becomes fragile. Not because you're bolting things on, but because
> you didn't consider early enough that other things might be bolted on
> further down the road.

This is an interesting point.  I guess I see the issues stemming from the fact that nova supports many different configurations and most developers working on a feature are only concerned with one of them.  Someone focuses on live migration, and it ends up being implemented for kvm but no other hypervisor.  Someone else adds boot from volume support, but only through the ec2 api.

My hope was that turning ec2 into a compatibility api would force people adding the ec2 features to make them work in the openstack api as well.  I really feel like we're failing in producing a cloud standard api if we don't have all of the features.  It is quite sad that we are still missing security group support in the os api for example.

That said, I'm not wedded to this approach;  I don't mind ec2 talking directly to the service apis. I just want to make sure that we understand only way we are going to get to the goal of a standard ubiquitous os api is if we make sure to add the features there as well.

Said another way, I have two goals:

Goal 1. OS api becomes a complete standard that supports the entire nova feature set.
Goal 2. EC2 api implements as close to amazon's apis as possible and supports as many features as possible without breaking compatibility.

I think that there is little disagreement in the goals, it is just how to get there that we are quibbling over.

Option 1
    EC2
       |
     OS
       |
   Internal

Advantage: forces us to add extensions to the OS api to support ec2 features, which makes the OS api more complete.
Disadvantage: harder to add features to the EC2 api

Option 2
    EC2  OS
       \      /
      Internal
Advantage: easier to add the ec2 features, so we end up with a more feature-rich ec2 api
Disadvantage: OS api can get skipped when new features are added


I feel that goal 1 is slightly more important than goal 2 so I'm leaning towards implementation option 1.  This is not a strong preference, though.  I think with the right focus we can achieve both goals in either scenario.

Vish