[Openstack] Chef Deployment System for Swift - a proposed design - feedback?

Jay Payne letterj at gmail.com
Fri May 6 17:29:11 UTC 2011


Judd,

Proxy servers with the allow_account_management flag will accept PUT
and DELETE requests on accounts.   On most deployments I would think
that this functionality would have limited access and locked down.
That is why I was suggesting it be broken out into a different roll.

--J

On Fri, May 6, 2011 at 10:35 AM, Judd Maltin <judd at newgoliath.com> wrote:
> Hi Joe -
>
> Answers inline:
>
> On Wed, Apr 27, 2011 at 9:55 PM, Jay Payne <letterj at gmail.com> wrote:
>>
>> Judd,
>>
>> I'm not that familiar with Chef (I'll do some research) but I have a
>> couple of questions and some thoughts:
>>
>> 1.  Is this for a multi-server environment?
>
> Yes.
>>
>> 2.  Are all your proxy nodes going to have "allow_account_management =
>> true" in the configs?   It might be a good idea to have a second proxy
>> config for account management only
>
> I'm not sure - I don't know about the performance impact of such a config.
>>
>> 3.  Have you looked at using swauth instead of auth?
>
> Not yet.
>>
>> 4.  Have you thought about an admin or client node that has utilities
>> on it like st and stats-report?
>
> Good suggestion.  An admin node which is part of the cluster and not subject
> to the performance sensitivity of line of business nodes.
>>
>> 5.  How where will you do on-going ring management or changes?
>
> Rings are defined in a Chef "DataBag".  When the databag is updated, Chef
> detects the file change.  When the chef-client runs on the nodes, they pick
> up the config changes.  Scheduling and orchestrating the chef-client runs
> will also be expressed in the data-bag (or from an orchestration server)
>>
>> 6.  I would think about including some type of functional test at the
>> end of the deployment process to verify everything was created
>> properly and that all nodes can communicate.
>
> Good point - adding in validation steps and final functional test.
>>
>>
>>
>> --J
>>
>> On Wed, Apr 27, 2011 at 6:18 PM, Judd Maltin <judd at newgoliath.com> wrote:
>> > Hi Folks,
>> >
>> > I've been hacking away at creating an automated deployment system for
>> > Swift
>> > using Chef.  I'd like to drop a design idea on you folks (most of which
>> > I've
>> > already implemented) and get feedback from this esteemed group.
>> >
>> > My end goal is to have a "manifest" (apologies to Puppet) which will
>> > define
>> > an entire swift cluster, deploy it automatically, and allow edits to the
>> > ingredients to manage the cluster.  In this case, a "manifest" is a
>> > combination of a chef databag describing the swift settings, and a
>> > spiceweasel infrastructure.yaml file describing the OS configuration.
>> >
>> > Ingredients:
>> > - swift cookbook with base, proxy and server recipes.  proxy nodes also
>> > (provisionally) contain auth services. storage nodes handle object,
>> > container and account services.
>> > -- Base recipe handles common package install, OS user creation.  Sets
>> > up
>> > keys.
>> > -- Proxy recipe handles proxy nodes: network config, package install,
>> > memcache config, proxy and auth package config, user creation, ring
>> > management (including builder file backup), user management
>> > -- Storage recipe handles storage nodes: network config, storage device
>> > config, package install, ring management.
>> >
>> > - chef databag that describes a swift cluster (eg:
>> > mycluster_databag.json)
>> > -- proxy config settings
>> > -- memcached settings
>> > -- settings for all rings and devices
>> > -- basic user settings
>> > -- account management
>> >
>> > - chef "spiceweasel" file that auto-vivifies the infrastructure: (eg:
>> > mycluster_infra.yaml)
>> > -- uploads cookbooks
>> > -- uploads roles
>> > -- uploads the cluster's databag
>> > -- kicks off node provisioning by requesting from infrastructure API
>> > (ec2 or
>> > what have you) the following:
>> > --- chef roles applied (role[swift:proxy] or role[swift:storage])
>> > --- server flavor
>> > --- storage device configs
>> > --- hostname
>> > --- proxy and storage network details
>> >
>> > By calling this spiceweasel file, the infrastructure can leap into
>> > existence.
>> >
>> > I'm more or less done with all this stuff - and I'd really appreciate
>> > conceptual feedback before I take out all the non-sense code I have in
>> > the
>> > files and publish.
>> >
>> > Many thanks!  Happy spring, northern hemispherians!
>> > -judd
>> >
>> > Judd Maltin
>> > T: 917-882-1270
>> > F: 501-694-7809
>> > A loving heart is never wrong.
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Mailing list: https://launchpad.net/~openstack
>> > Post to     : openstack at lists.launchpad.net
>> > Unsubscribe : https://launchpad.net/~openstack
>> > More help   : https://help.launchpad.net/ListHelp
>> >
>> >
>
>
>
> --
> Judd Maltin
> T: 917-882-1270
> F: 501-694-7809
> A loving heart is never wrong.
>
>
>
>




More information about the Openstack mailing list