<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On 23 December 2015 at 12:01, Steven Hardy <span dir="ltr"><<a href="mailto:shardy@redhat.com" target="_blank">shardy@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Tue, Dec 22, 2015 at 03:36:02PM +0000, Dougal Matthews wrote:<br>
> Hi all,<br>
><br>
> This topic came up in the 2015-12-15 meeting[1], and again briefly today.<br>
> After working with the code that came out of the deployment library<br>
> spec[2] I<br>
> had some concerns with how we are storing the templates.<br>
><br>
> Simply put, when we are dealing with 100+ files from<br>
> tripleo-heat-templates<br>
> how can we ensure consistency in Swift without any atomicity or<br>
> transactions.<br>
> I think this is best explained with a couple of examples.<br>
><br>
</span>> Â - When we create a new deployment plan (upload all the templates to<br>
> swift)<br>
> Â Â how do we handle the case where there is an error? For example, if we<br>
> are<br>
> Â Â uploading 10 files - what do we do if the 5th one fails for some<br>
> reason?<br>
> Â Â There is a patch to do a manual rollback[3], but I have concerns<br>
> about<br>
> Â Â doing this in Python. If Swift is completely inaccessible for a short<br>
> Â Â period the rollback wont work either.<br>
<br>
How does using a different data store help fix this error path?<br>
<br>
Regardless of the Swift/DB/Git choice, you have to detect something failed,<br>
and mark the plan as in a failed state.<br>
<br>
> Â - When deploying to Heat, we need to download all the YAML files from<br>
> Swift.<br>
> Â Â This can take a couple of seconds. What happens if somebody starts to<br>
> Â Â upload a new version of the plan in the middle? We could end up<br>
> trying to<br>
> Â Â deploy half old and half new files. We wouldn't have a consistent<br>
> view of<br>
> Â Â the database.<br>
<br>
If this happens, the API design is wrong - we should have a plan reference<br>
a unique identifier, including a version (e.g through swift versioned<br>
objects, git tags/branches or whatever), then on deployment heat should<br>
download exactly one version of those artefacts (e.g via a swift tempurl or<br>
a git URL, or whatever).<br></blockquote><div><br></div><div>Swift gives us no way to get the current version. In a versioned container, the<br></div><div>current version is referenced by the file path only, the previous versions are<br>the file path with a timestamp. So there is no way to reference an object and<br>not see updates unless we start tracking timestamps etc. in code.<br></div><div><br></div><div>I hope I am understanding it correctly, this was one of the primary reasons<br></div><div>that we discounted Swift as an option for Tuskar when I was looking into <br></div><div>versioning around a year ago. I've had a look again and it seems the situation<br></div><div>hasn't changed (but I will double check now).<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
FYI heatclient already supports downloading template objects directly from<br>
swift, and heat/heatclient already support downloading from http URLs, so<br>
all we have to do AFAICS is generate a URL pointing to a consistent<br>
version of the templates.<br>
<span class=""><br>
> We had a few suggestions in the meeting:<br>
><br>
</span>> Â - Add a locking mechanism. I would be concerned about deadlocks or<br>
> having to<br>
> Â Â lock for the full duration of a deploy.<br>
<br>
I think we need to create a new version of a plan every time it's modified,<br>
then on deployment the concern around locking is much reduced, or even<br>
eliminated?<br>
<br>
> Â - Store the files in a tarball (so we only deal with one file). I think<br>
> we<br>
> Â Â could still hit issues with multiple operations unless we guarantee<br>
> that<br>
> Â Â only one instance of the API is ever running.<br>
<span class="">><br>
> I think these could potentially be made to work, but at this point are we<br>
> using the best tool for the job?<br>
><br>
> For alternatives, I see a can think of a couple of options:<br>
><br>
> - Use a database, like we did for Tuskar and most OpenStack API's do.<br>
<br>
</span>Storing template files in a database with Tuskar was a mistake - we should<br>
*not* do that again IMO - it was a very opaque interface and caused a lot<br>
of confusion in use.<br>
<br>
Storing code (in this case yaml files) in some sort of versioned repository<br>
is *such* a solved problem - it'll be total wheel-reinventing if we<br>
implement template revision control inside a bespoke DB IMHO.<br>
<br>
I think swift is probably a workable solution (using versioning and<br>
tempurl's), but we should really consider making the template store<br>
pluggable, as having the API backed by an (operator visible) git repo is<br>
likely to be a nicer and more familiar solution.<br>
<br>
The question around validation data I'm less clear on - we already store<br>
discovery data in swift, which seems to work OK - how is the validation<br>
data different, in that it warrants a bespoke DB data store?<br>
<br>
Steve<br>
<div class="HOEnZb"><div class="h5"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div></div>