Open Stack

Thanks Andy,

Yes, after the discussions we had I dug into that approach and ran into some issues that suggested it would be difficult to implement for cactus. I summarized our discussion in this thread for the community to review and it flagged concerns. It also highlighted other issues that need to be addressed (pagination, for example).

>From this it I decided to push on with the "turtles all the way down" approach (great description btw :) ... just to get something working. As we discussed, code > talk.

I'm still not convinced we can't get the benefits of single database through separate caching schemes; and easy to layer on to the existing solution.

Certainly a great topic for the design summit, imho.

-S


________________________________
From: Andy Smith [andyster at gmail.com]
Sent: Wednesday, March 23, 2011 10:23 PM
To: Sandy Walsh
Cc: openstack at lists.launchpad.net
Subject: Re: [Openstack] A single cross-zone database?

Coming back from a long break and getting back up to speed.

I believe Sandy and I spoke about this at decent length before, the proposal that I understood we both walked away happy with was this:

1. As a first step, implement it with a single db, because that is what we already have and what is the easiest first step. Zones are an additional column or join table and implemented as a filter on the query for instances and the like. Zones in this case being used largely to mean "grouping of servers with some common, possibly distinct, property." We only support a single level of 'zones' for the moment, no arbitrary nesting.
2. See whether that works, it probably will for the current scaling targets and use cases.
3. To implement bursting/hybrid stuff, we then work on a data abstraction for aggregating the results of remote calls, understanding that we expect remote calls to have lower performance.

The blueprint as it stands (http://wiki.openstack.org/MultiClusterZones) is following a 'turtles all the way down' approach that probably isn't really appropriate for real world usage. In reality there are things that are global level, things that are region level and things that are cluster level.
 * A global service will be in charge of specific requests that need to be global and have very specific and constrained scaling properties, perhaps global dns management.
 * A region represents the largest unit of scaling, the maximum number of nodes and services that can managed at time while meeting SLAs, including the public endpoints, most things above region level need to be managed client side: one does not expect to list all instances across all regions in a single call, one expects to make three requests to the three regions one's instances are in.
 * A cluster represents a grouping unit with common properties, either rack commonality, capability commonality, or simply special case customers. Subgroupings here may still be interesting or useful but are unlikely to be necessary. Clusters are likely to share many things at the region level as well as have individual cluster level concerns.

This design handles the needs of plenty of large organizations already and is easy to move to piecemeal from our current codebase, the first step being that zones are clusters and can share a regional registry (database) and public endpoint while still having per-zone services (scheduler, et al).

--andy

On Wed, Mar 16, 2011 at 4:51 PM, Sandy Walsh <sandy.walsh at rackspace.com<mailto:sandy.walsh at rackspace.com>> wrote:
Yup, that's a fair assessment.

That said, even without SDB and caching it's going to be tight for Cactus anyway. There are lots of little issues that are cropping up once I got down to 100'.

-S

________________________________
From: Justin Santa Barbara <justin at fathomdb.com<mailto:justin at fathomdb.com>>

Sandy: Have I got the wrong end of the stick here?  Are these our choices?

Justin


Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com<mailto:abuse at rackspace.com>, and delete the original message.
Your cooperation is appreciated.


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack at lists.launchpad.net<mailto:openstack at lists.launchpad.net>
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20110324/e1eb945f/attachment.html>