Open Stack

On Thu, 2016-01-07 at 18:16 +0000, Fox, Kevin M wrote:
> I don't think the radosgw supports bulk yet. They haven't for a very
> long time. :/

We use pure Swift (not Ceph backed radosgw) in our Undercloud. So I'm
not sure this would be a factor for us.

Dan

> 
> Thanks,
> Kevin
> ________________________________________
> From: Dan Prince [dprince at redhat.com]
> Sent: Thursday, January 07, 2016 9:03 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [TripleO] Is Swift a good choice of
> database for the TripleO API?
> 
> On Tue, 2015-12-22 at 15:36 +0000, Dougal Matthews wrote:
> > Hi all,
> > 
> > This topic came up in the 2015-12-15 meeting[1], and again briefly
> > today.
> > After working with the code that came out of the deployment library
> > spec[2] I
> > had some concerns with how we are storing the templates.
> > 
> > Simply put, when we are dealing with 100+ files from tripleo-heat-
> > templates
> > how can we ensure consistency in Swift without any atomicity or
> > transactions.
> > I think this is best explained with a couple of examples.
> > 
> >  - When we create a new deployment plan (upload all the templates
> > to
> > swift)
> >    how do we handle the case where there is an error? For example,
> > if
> > we are
> >    uploading 10 files - what do we do if the 5th one fails for some
> > reason?
> >    There is a patch to do a manual rollback[3], but I have concerns
> > about
> >    doing this in Python. If Swift is completely inaccessible for a
> > short
> >    period the rollback wont work either.
> 
> Would Swift's Bulk middleware help us here:
> 
> https://github.com/openstack/swift/blob/master/swift/common/middlewar
> e/
> bulk.py
> 
> We don't currently have that middleware enabled but we certainly
> could
> try it out. NOTE the "Expand tar files into a swift account"
> feature...
> 
> Dan
> 
> > 
> >  - When deploying to Heat, we need to download all the YAML files
> > from Swift.
> >    This can take a couple of seconds. What happens if somebody
> > starts
> > to
> >    upload a new version of the plan in the middle? We could end up
> > trying to
> >    deploy half old and half new files. We wouldn't have a
> > consistent
> > view of
> >    the database.
> > 
> > We had a few suggestions in the meeting:
> > 
> >  - Add a locking mechanism. I would be concerned about deadlocks or
> > having to
> >    lock for the full duration of a deploy.
> >  - Store the files in a tarball (so we only deal with one file). I
> > think we
> >    could still hit issues with multiple operations unless we
> > guarantee that
> >    only one instance of the API is ever running.
> > 
> > I think these could potentially be made to work, but at this point
> > are we
> > using the best tool for the job?
> > 
> > For alternatives, I see a can think of a couple of options:
> > 
> > - Use a database, like we did for Tuskar and most OpenStack API's
> > do.
> > - Invest time in building something on Swift.
> > - Glance was transitioning to be a Artifact store. I don't know the
> > status of
> >   this or if it would meet out needs.
> > 
> > Any input, ideas or suggestions would be great!
> > 
> > Thanks,
> > Dougal
> > 
> > 
> > [1]: http://eavesdrop.openstack.org/meetings/tripleo/2015/tripleo.2
> > 01
> > 5-12-15-14.03.log.html#l-89
> > [2]: https://specs.openstack.org/openstack/tripleo-specs/specs/mita
> > ka
> > /tripleo-overcloud-deployment-library.html
> > [3]: https://review.openstack.org/#/c/257481/
> > ___________________________________________________________________
> > __
> > _____
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsu
> > bs
> > cribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _____________________________________________________________________
> _____
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev