Open Stack

On 23 December 2015 at 12:01, Steven Hardy <shardy at redhat.com> wrote:

> On Tue, Dec 22, 2015 at 03:36:02PM +0000, Dougal Matthews wrote:
> >    Hi all,
> >
> >    This topic came up in the 2015-12-15 meeting[1], and again briefly
> today.
> >    After working with the code that came out of the deployment library
> >    spec[2] I
> >    had some concerns with how we are storing the templates.
> >
> >    Simply put, when we are dealing with 100+ files from
> >    tripleo-heat-templates
> >    how can we ensure consistency in Swift without any atomicity or
> >    transactions.
> >    I think this is best explained with a couple of examples.
> >
> >    Â - When we create a new deployment plan (upload all the templates to
> >    swift)
> >    Â Â  how do we handle the case where there is an error? For example,
> if we
> >    are
> >    Â Â  uploading 10 files - what do we do if the 5th one fails for some
> >    reason?
> >    Â Â  There is a patch to do a manual rollback[3], but I have concerns
> >    about
> >    Â Â  doing this in Python. If Swift is completely inaccessible for a
> short
> >    Â Â  period the rollback wont work either.
>
> How does using a different data store help fix this error path?
>
> Regardless of the Swift/DB/Git choice, you have to detect something failed,
> and mark the plan as in a failed state.
>
> >    Â - When deploying to Heat, we need to download all the YAML files
> from
> >    Swift.
> >    Â Â  This can take a couple of seconds. What happens if somebody
> starts to
> >    Â Â  upload a new version of the plan in the middle? We could end up
> >    trying to
> >    Â Â  deploy half old and half new files. We wouldn't have a consistent
> >    view of
> >    Â Â  the database.
>
> If this happens, the API design is wrong - we should have a plan reference
> a unique identifier, including a version (e.g through swift versioned
> objects, git tags/branches or whatever), then on deployment heat should
> download exactly one version of those artefacts (e.g via a swift tempurl or
> a git URL, or whatever).
>

Swift gives us no way to get the current version. In a versioned container,
the
current version is referenced by the file path only, the previous versions
are
the file path with a timestamp. So there is no way to reference an object
and
not see updates unless we start tracking timestamps etc. in code.

I hope I am understanding it correctly, this was one of the primary reasons
that we discounted Swift as an option for Tuskar when I was looking into
versioning around a year ago. I've had a look again and it seems the
situation
hasn't changed (but I will double check now).


>
> FYI heatclient already supports downloading template objects directly from
> swift, and heat/heatclient already support downloading from http URLs, so
> all we have to do AFAICS is generate a URL pointing to a consistent
> version of the templates.
>
> >    We had a few suggestions in the meeting:
> >
> >    Â - Add a locking mechanism. I would be concerned about deadlocks or
> >    having to
> >    Â Â  lock for the full duration of a deploy.
>
> I think we need to create a new version of a plan every time it's modified,
> then on deployment the concern around locking is much reduced, or even
> eliminated?
>
> >    Â - Store the files in a tarball (so we only deal with one file). I
> think
> >    we
> >    Â Â  could still hit issues with multiple operations unless we
> guarantee
> >    that
> >    Â Â  only one instance of the API is ever running.
> >
> >    I think these could potentially be made to work, but at this point
> are we
> >    using the best tool for the job?
> >
> >    For alternatives, I see a can think of a couple of options:
> >
> >    - Use a database, like we did for Tuskar and most OpenStack API's do.
>
> Storing template files in a database with Tuskar was a mistake - we should
> *not* do that again IMO - it was a very opaque interface and caused a lot
> of confusion in use.
>
> Storing code (in this case yaml files) in some sort of versioned repository
> is *such* a solved problem - it'll be total wheel-reinventing if we
> implement template revision control inside a bespoke DB IMHO.
>
> I think swift is probably a workable solution (using versioning and
> tempurl's), but we should really consider making the template store
> pluggable, as having the API backed by an (operator visible) git repo is
> likely to be a nicer and more familiar solution.
>
> The question around validation data I'm less clear on - we already store
> discovery data in swift, which seems to work OK - how is the validation
> data different, in that it warrants a bespoke DB data store?
>
> Steve
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151223/829fd3be/attachment.html>