[openstack-dev] [TripleO] Is Swift a good choice of database for the TripleO API?
Fox, Kevin M
Kevin.Fox at pnnl.gov
Thu Jan 7 18:16:06 UTC 2016
I don't think the radosgw supports bulk yet. They haven't for a very long time. :/
From: Dan Prince [dprince at redhat.com]
Sent: Thursday, January 07, 2016 9:03 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [TripleO] Is Swift a good choice of database for the TripleO API?
On Tue, 2015-12-22 at 15:36 +0000, Dougal Matthews wrote:
> Hi all,
> This topic came up in the 2015-12-15 meeting, and again briefly
> After working with the code that came out of the deployment library
> spec I
> had some concerns with how we are storing the templates.
> Simply put, when we are dealing with 100+ files from tripleo-heat-
> how can we ensure consistency in Swift without any atomicity or
> I think this is best explained with a couple of examples.
> - When we create a new deployment plan (upload all the templates to
> how do we handle the case where there is an error? For example, if
> we are
> uploading 10 files - what do we do if the 5th one fails for some
> There is a patch to do a manual rollback, but I have concerns
> doing this in Python. If Swift is completely inaccessible for a
> period the rollback wont work either.
Would Swift's Bulk middleware help us here:
We don't currently have that middleware enabled but we certainly could
try it out. NOTE the "Expand tar files into a swift account" feature...
> - When deploying to Heat, we need to download all the YAML files
> from Swift.
> This can take a couple of seconds. What happens if somebody starts
> upload a new version of the plan in the middle? We could end up
> trying to
> deploy half old and half new files. We wouldn't have a consistent
> view of
> the database.
> We had a few suggestions in the meeting:
> - Add a locking mechanism. I would be concerned about deadlocks or
> having to
> lock for the full duration of a deploy.
> - Store the files in a tarball (so we only deal with one file). I
> think we
> could still hit issues with multiple operations unless we
> guarantee that
> only one instance of the API is ever running.
> I think these could potentially be made to work, but at this point
> are we
> using the best tool for the job?
> For alternatives, I see a can think of a couple of options:
> - Use a database, like we did for Tuskar and most OpenStack API's do.
> - Invest time in building something on Swift.
> - Glance was transitioning to be a Artifact store. I don't know the
> status of
> this or if it would meet out needs.
> Any input, ideas or suggestions would be great!
> : http://eavesdrop.openstack.org/meetings/tripleo/2015/tripleo.201
> : https://specs.openstack.org/openstack/tripleo-specs/specs/mitaka
> : https://review.openstack.org/#/c/257481/
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev