[Openstack] Queue Service

Glen Campbell glen.campbell at RACKSPACE.COM
Tue Feb 15 15:45:37 UTC 2011


Having the <account> in the URL makes the resource cacheable (via HTTP, I
mean; you can always manually cache it, but that's not RESTful); without
it, you would need to require unique <container>s to uniquely reference a
cached copy.

On 2/14/11 7:57 PM, "Paul Voccio" <paul.voccio at rackspace.com> wrote:

>Looking at the swift docs, they reference a container like so:
>
> METHOD /v1/<account>/<container> HTTP/1.1
>
>http://docs.rackspacecloud.com/files/api/v1/cf-devguide-20110112.pdf
>
>For the openstack api, it also includes the account id in the request:
>
>POST /v1.1/214412/images HTTP/1.1
>Host: servers.api.openstack.org
>Content-Type: application/json
>Accept: application/xml
>X-Auth-Token: eaaafd18-0fed-4b3a-81b4-663c99ec1cb
>
>
>
>This seems a bit different than what you're suggesting. What am I missing?
>Shouldn't the account id be in the request with the auth headers to stay
>in line with the other specs?
>
>
>On 2/14/11 6:56 PM, "Eric Day" <eday at oddments.org> wrote:
>
>>On Tue, Feb 15, 2011 at 12:49:01AM +0000, Paul Voccio wrote:
>>> Dropping the account_id, would this also assume that there can be more
>>> than one queue per account?
>>
>>Yeah. Think of the queue name as an extra namespace layer much like
>>a swift container, except you don't create or delete them, they just
>>exist when there is an active message in it.
>>
>>-Eric
>>
>>> On 2/14/11 5:54 PM, "Eric Day" <eday at oddments.org> wrote:
>>> 
>>> >Yeah, for anonymous access that would be the case. Those are not
>>>needed
>>> >when the request comes in with OpenStack Auth headers (like nova).
>>> >
>>> >Do you think we should be repeating the account id in the URI when
>>> >the auth headers are present?
>>> >
>>> >-Eric
>>> >
>>> >On Mon, Feb 14, 2011 at 11:44:58PM +0000, Paul Voccio wrote:
>>> >> Eric,
>>> >> 
>>> >> Just looking at the http operations. Shouldn't the calls be around
>>>the
>>> >> account then the queue?
>>> >> 
>>> >> GET /$account_id/queue/id to list all the queues
>>> >> GET /$account_id/queue/id/message/id
>>> >> 
>>> >> Am I off here? Thoughts?
>>> >> 
>>> >> pvo
>>> >>   
>>> >> 
>>> >> On 2/14/11 5:07 PM, "Eric Day" <eday at oddments.org> wrote:
>>> >> 
>>> >> >I've summarized the original email and added more sections for
>>>review
>>> >> >and discussion here:
>>> >> >
>>> >> >http://wiki.openstack.org/QueueService
>>> >> >
>>> >> >In particular there are details on the various components of the
>>> >> >queue service, a draft at what the REST operations will look like,
>>> >> >and a couple brief examples.
>>> >> >
>>> >> >Please let me know if any clarification is needed and I'll update
>>> >> >the wiki. Feedback and discussion on use cases and what you think
>>> >> >the service should look like is very appreciated.
>>> >> >
>>> >> >Thanks,
>>> >> >-Eric
>>> >> >
>>> >> >On Mon, Feb 14, 2011 at 09:51:42AM -0800, Eric Day wrote:
>>> >> >> Hi everyone,
>>> >> >> 
>>> >> >> When looking at other services to include as part of OpenStack,
>>>the
>>> >> >> first that comes to mind for many is a queue. A queue service can
>>> >> >> not only provide a useful public cloud service, but can also
>>>provide
>>> >> >> one of the building blocks for other services. I've been leading
>>>an
>>> >> >> effort to research and gather requirements for a queue service
>>>and
>>> >> >> I'd like to share the current state and get community feedback. I
>>> >> >> expect real development to begin very soon, and would also like
>>>to
>>> >> >> identify developers who will have time to dedicate to this
>>>project.
>>> >> >> 
>>> >> >> I'd like to note this is not an official OpenStack project yet.
>>>The
>>> >> >> intention is once we have the community support and a simple
>>> >> >> implementation, we will submit the project to the OpenStack
>>>Project
>>> >> >> Oversight Committee for approval.
>>> >> >> 
>>> >> >> The reason we are initiating our own project vs using an existing
>>>one
>>> >> >> is due to simplicity, modularity, and scale. Also, very few (if
>>>any)
>>> >> >> existing queue systems out there were built with multi-tenant
>>>cloud
>>> >> >> use cases in mind. Very few also have a simple and extensible
>>>REST
>>> >> >> API. There are possible solutions to build an AMQP based service,
>>> >> >> but AMQP brings complexity and a protocol not optimized for high
>>> >> >> latency and intermittent connectivity.
>>> >> >> 
>>> >> >> The primary goals of the queue service are:
>>> >> >> 
>>> >> >> * Simple - Think simple REST based queues for most use cases.
>>>Easy
>>> >> >>   to access and use from any language. It should not require much
>>> >> >>   setup, if any, before you can start pushing messages into it.
>>> >> >> 
>>> >> >> * Modular API - Initially we'll focus on a simple REST API,
>>> >> >>   but this will not in any way be a first-class API. It should be
>>> >> >>   possible to add other protocols (AMQP, protocol buffers,
>>>Gearman,
>>> >> >>   etc) for other use cases. Note that the internal service API
>>>will
>>> >> >>   not always provide a 1-1 mapping with the external API, so some
>>> >> >>   features with advanced protocols may be unavailable.
>>> >> >> 
>>> >> >> * Fast - Since this will act as a building block for other
>>>services
>>> >> >>   that my drive heavy throughput, performance will have a focus.
>>>This
>>> >> >>   mostly comes down to implementation language and how clients
>>>and
>>> >> >>   workers interact with the broker to reduce network chatter.
>>> >> >> 
>>> >> >> * Multi-tenant - Support multiple accounts for the service, and
>>>since
>>> >> >>   this will also be a public service for some deployments,
>>>protect
>>> >> >>   against potentially malicious users.
>>> >> >> 
>>> >> >> * Persistent - Allow messages to optionally be persistent. For
>>> >> >>   protocols that can support it, this can be an optional flag
>>>while
>>> >> >>   the message is submitted. The persistent storage should also be
>>> >> >>   modular so we can test various data stores and accommodate
>>> >>different
>>> >> >>   deployment options.
>>> >> >> 
>>> >> >> * Zones and locality awareness - As we've been discussing in
>>>other
>>> >> >>   threads, locality in cloud services is an important feature.
>>>When
>>> >> >>   dealing with where messages should be processed, we need to
>>>have
>>> >> >>   location awareness to process data where it exists to reduce
>>> >>network
>>> >> >>   overhead and processing time.
>>> >> >> 
>>> >> >> Before diving down into implementation details, I would like to
>>>hear
>>> >> >> what folks have to say about the initial requirements above. Once
>>> >> >> there is something along the lines of agreement, I'll be sending
>>>out
>>> >> >> other topics for discussion dealing with implementation.
>>> >> >> 
>>> >> >> I'm looking forward to your feedback. Thanks!
>>> >> >> 
>>> >> >> -Eric
>>> >> >
>>> >> >_______________________________________________
>>> >> >Mailing list: https://launchpad.net/~openstack
>>> >> >Post to     : openstack at lists.launchpad.net
>>> >> >Unsubscribe : https://launchpad.net/~openstack
>>> >> >More help   : https://help.launchpad.net/ListHelp
>>> >> 
>>> >> 
>>> >> 
>>> >> Confidentiality Notice: This e-mail message (including any attached
>>>or
>>> >> embedded documents) is intended for the exclusive and confidential
>>>use
>>> >>of the
>>> >> individual or entity to which this message is addressed, and unless
>>> >>otherwise
>>> >> expressly indicated, is confidential and privileged information of
>>> >>Rackspace.
>>> >> Any dissemination, distribution or copying of the enclosed material
>>>is
>>> >>prohibited.
>>> >> If you receive this transmission in error, please notify us
>>>immediately
>>> >>by e-mail
>>> >> at abuse at rackspace.com, and delete the original message.
>>> >> Your cooperation is appreciated.
>>> 
>>> 
>>> 
>>> Confidentiality Notice: This e-mail message (including any attached or
>>> embedded documents) is intended for the exclusive and confidential use
>>>of the
>>> individual or entity to which this message is addressed, and unless
>>>otherwise
>>> expressly indicated, is confidential and privileged information of
>>>Rackspace. 
>>> Any dissemination, distribution or copying of the enclosed material is
>>>prohibited.
>>> If you receive this transmission in error, please notify us immediately
>>>by e-mail
>>> at abuse at rackspace.com, and delete the original message.
>>> Your cooperation is appreciated.
>
>
>
>Confidentiality Notice: This e-mail message (including any attached or
>embedded documents) is intended for the exclusive and confidential use of
>the
>individual or entity to which this message is addressed, and unless
>otherwise
>expressly indicated, is confidential and privileged information of
>Rackspace.
>Any dissemination, distribution or copying of the enclosed material is
>prohibited.
>If you receive this transmission in error, please notify us immediately
>by e-mail
>at abuse at rackspace.com, and delete the original message.
>Your cooperation is appreciated.
>
>
>_______________________________________________
>Mailing list: https://launchpad.net/~openstack
>Post to     : openstack at lists.launchpad.net
>Unsubscribe : https://launchpad.net/~openstack
>More help   : https://help.launchpad.net/ListHelp





More information about the Openstack mailing list