Open Stack

thats somewhat limiting though...

We use ceph here because we can pool our storage between the swift protocol endpoint, and the block storage system. It doesn't make us partition up our storage. I can totally see a case where you'd maybe just scale out your undercloud ceph for use in the overcloud, rather then stand up a whole other storage system and have to partition your storage.

Thanks,
Kevin
________________________________________
From: Dan Prince [dprince at redhat.com]
Sent: Thursday, January 07, 2016 11:02 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [TripleO] Is Swift a good choice of database for the TripleO API?

On Thu, 2016-01-07 at 18:16 +0000, Fox, Kevin M wrote:
> I don't think the radosgw supports bulk yet. They haven't for a very
> long time. :/

We use pure Swift (not Ceph backed radosgw) in our Undercloud. So I'm
not sure this would be a factor for us.

Dan

>
> Thanks,
> Kevin
> ________________________________________
> From: Dan Prince [dprince at redhat.com]
> Sent: Thursday, January 07, 2016 9:03 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [TripleO] Is Swift a good choice of
> database for the TripleO API?
>
> On Tue, 2015-12-22 at 15:36 +0000, Dougal Matthews wrote:
> > Hi all,
> >
> > This topic came up in the 2015-12-15 meeting[1], and again briefly
> > today.
> > After working with the code that came out of the deployment library
> > spec[2] I
> > had some concerns with how we are storing the templates.
> >
> > Simply put, when we are dealing with 100+ files from tripleo-heat-
> > templates
> > how can we ensure consistency in Swift without any atomicity or
> > transactions.
> > I think this is best explained with a couple of examples.
> >
> >  - When we create a new deployment plan (upload all the templates
> > to
> > swift)
> >    how do we handle the case where there is an error? For example,
> > if
> > we are
> >    uploading 10 files - what do we do if the 5th one fails for some
> > reason?
> >    There is a patch to do a manual rollback[3], but I have concerns
> > about
> >    doing this in Python. If Swift is completely inaccessible for a
> > short
> >    period the rollback wont work either.
>
> Would Swift's Bulk middleware help us here:
>
> https://github.com/openstack/swift/blob/master/swift/common/middlewar
> e/
> bulk.py
>
> We don't currently have that middleware enabled but we certainly
> could
> try it out. NOTE the "Expand tar files into a swift account"
> feature...
>
> Dan
>
> >
> >  - When deploying to Heat, we need to download all the YAML files
> > from Swift.
> >    This can take a couple of seconds. What happens if somebody
> > starts
> > to
> >    upload a new version of the plan in the middle? We could end up
> > trying to
> >    deploy half old and half new files. We wouldn't have a
> > consistent
> > view of
> >    the database.
> >
> > We had a few suggestions in the meeting:
> >
> >  - Add a locking mechanism. I would be concerned about deadlocks or
> > having to
> >    lock for the full duration of a deploy.
> >  - Store the files in a tarball (so we only deal with one file). I
> > think we
> >    could still hit issues with multiple operations unless we
> > guarantee that
> >    only one instance of the API is ever running.
> >
> > I think these could potentially be made to work, but at this point
> > are we
> > using the best tool for the job?
> >
> > For alternatives, I see a can think of a couple of options:
> >
> > - Use a database, like we did for Tuskar and most OpenStack API's
> > do.
> > - Invest time in building something on Swift.
> > - Glance was transitioning to be a Artifact store. I don't know the
> > status of
> >   this or if it would meet out needs.
> >
> > Any input, ideas or suggestions would be great!
> >
> > Thanks,
> > Dougal
> >
> >
> > [1]: http://eavesdrop.openstack.org/meetings/tripleo/2015/tripleo.2
> > 01
> > 5-12-15-14.03.log.html#l-89
> > [2]: https://specs.openstack.org/openstack/tripleo-specs/specs/mita
> > ka
> > /tripleo-overcloud-deployment-library.html
> > [3]: https://review.openstack.org/#/c/257481/
> > ___________________________________________________________________
> > __
> > _____
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsu
> > bs
> > cribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev