[openstack-dev] [heat] [glance] Heater Proposal

Tim Schnell tim.schnell at RACKSPACE.COM
Thu Dec 5 21:07:17 UTC 2013

On 12/5/13 12:17 PM, "Clint Byrum" <clint at fewbar.com> wrote:

>Excerpts from Tim Schnell's message of 2013-12-05 09:49:03 -0800:
>> On 12/5/13 11:33 AM, "Randall Burt" <randall.burt at RACKSPACE.COM> wrote:
>> >On Dec 5, 2013, at 11:10 AM, Clint Byrum <clint at fewbar.com>
>> > wrote:
>> >
>> >> Excerpts from James Slagle's message of 2013-12-05 08:35:12 -0800:
>> >>> On Thu, Dec 5, 2013 at 11:10 AM, Clint Byrum <clint at fewbar.com>
>> >>>> Excerpts from Monty Taylor's message of 2013-12-04 17:54:45 -0800:
>> >>>>> Why not just use glance?
>> >>>>> 
>> >>>> 
>> >>>> I've asked that question a few times, and I think I can collate the
>> >>>> responses I've received below. I think enhancing glance to do these
>> >>>> things is on the table:
>> >>> 
>> >>> I'm actually interested in the use cases laid out by Heater from
>> >>> a template perspective and image perspective.  For the templates, as
>> >>> Robert mentioned, Tuskar needs a solution for this requirement,
>> >>> it's deploying using templates.  For the images, we have the concept
>> >>> of a "golden" image in TripleO and are heavily focused on image
>> >>> deployments.  Therefore, it seems to make sense that TripleO also
>> >>> needs a way to version/tag known good images.
>> >>> 
>> >>> Given that, I think it makes sense  to do this in a way so that it's
>> >>> consumable for things other than just templates.  In fact, you can
>> >>> almost s/template/image/g on the Heater wiki page, and it pretty
>> >>> lays out what I'd like to see for images as well.
>> >>> 
>> >>>> 1. Glance is for big blobs of data not tiny templates.
>> >>>> 2. Versioning of a single resource is desired.
>> >>>> 3. Tagging/classifying/listing/sorting
>> >>>> 4. Glance is designed to expose the uploaded blobs to nova, not
>> >>>> 
>> >>>> My responses:
>> >>>> 
>> >>>> 1: Irrelevant. Smaller things will fit in it just fine.
>> >>>> 
>> >>>> 2: The swift API supports versions. We could also have git as a
>> >>>> backend.
>> >>> 
>> >>> I would definitely like to see a git backend for versioning.  No
>> >>> reason to reimplement a different solution for what already works
>> >>> well.  I'm not sure we'd want to put a whole image into git though.
>> >>> Perhaps just it's manifest (installed components, software versions,
>> >>> etc) in json format would go into git, and that would be associated
>> >>> back to the binary image via uuid.  That would even make it easy to
>> >>> diff changes between versions, etc.
>> >>> 
>> >> 
>> >> Right, git for a big 'ol image makes little sense.
>> >> 
>> >> I'm suggesting that one might want to have two glances, one for
>> >> which just uses swift versions and would just expose a list of
>> >> and one for templates which would use git and thus expose more
>> >> like a git remote for the repo. I'm not sure if glance has embraced
>> >> extension paradigm yet, but this would fall nicely into it.
>> >
>> >Alternatively, Glance could have configurable backends for each image
>> >type allowing for optimization without the (often times messy)
>> >mechanism? This is assuming it doesn't do this already - I really need
>> >start digging here. In the spirit of general OpenStack architectural
>> >paradigms, as long as the service exposes a consistent interface for
>> >templates that includes versioning support, the back-end store and
>> >(possibly) the versioning "engine" should certainly be configurable.
>> >Swift probably makes a decent first/default implementation.
>> I'm not sure why we are attempting to fit a round peg in a square hole
>> here. Glance's entire design is built around serving images and there
>> absolutely is a difference between serving BLOBs versus relatively small
>> templates. Just look at the existing implementations, Heat already
>> the template directly in the database while Glance stores the blob in an
>> external file system.
>What aspects of circles and squares do you find match glance and the
>Heater problem space? :-P
>If this were as simple as geometric shape matching, my 4-year old could
>do it, right? :) Let's use analogies, I _love_ analogies, but perhaps
>lets be more careful about whether they actually aid the discussion.
>The blobs in the heat database are going to be a problem actually. I've
>dealt with a very similar relatively large system, storing millions
>of resumes in a single table for a job candidate database. It performs
>horribly even if you shard. These weren't 50MB resumes, these were 30 -
>50 kB resumes. That is because every time you need to pull all the rows
>you have to pull giant amounts of data and generally just blow out all
>of the caches, buffers, etc. Keystone's token table for the sql token
>backend has the same problem. RDBMS's are good at storing and retrieving
>rows of relatively predictably sized data. They suck for documents.
>Heat would do well to just store template references and fetch them in
>the rare cases where the raw templates are needed (updates, user asks to
>see it, etc). That could so easily be a glance reference.  And if Glance
>was suddenly much smarter and more capable of listing/searching/etc. the
>things in its database, then Heat gets a win, and so does Nova.
>> I'm not opposed to having Glance as an optional configurable backend for
>> Heater but if you look at the existing database models for Glance, an
>> entirely separate schema would have to be created to support templates.
>> also think that having Heater as a layer above Glance or whatever other
>> backend would allow for the flexibility for Heater's requirements to
>> diverge in the future from requirements that might make sense for images
>> but not templates.
>Huh? How are templates any different from images? Stick the template
>in as the blob that is currently the image (note that there are
>relatively small images too, like aki's for kernels which are just a
>few MB). Add in some meta-data for finding/listing/displaying template
>data efficiently. Bada-boom-bada-bing, you're done, right?
>I would love to hear about "future requirements" for Heater that won't
>fit into Glance. But thus far, I am struggling to see where this could
>go that wouldn't be a place Glance also wants to go.

I know we covered this a bit in the hangout but one possibility that I
think I failed to articulate is the fact that the template is a parseable
data representation while an image is a binary object. In my original
email I mentioned having a discussion about moving some of the "template"
metadata like application information and documentation into the HOT

This has important ramifications for the user experience as well as
whether or not abstracting Glance to "one-size-fits-all" is actually
feasible at an implementation level. Ideally the Heat template would serve
as the source of truth for information like "application-version" and
"template documentation", in the original Heater proposal this is
important because this data would have to be extracted from the template
and stored in a metadata field if it is relevant to the way that Heater
allows you to search or index different templates.

If we had to pass a duplicate version of this metadata in the HTTP Request
we could avoid grabbing this information from the template but we open up
data integrity issues. What happens when the end-user updates the metadata
but not the template? At the same time, I would reiterate that we can't
just make this data a "Heater/Template Catalog" metadata issue because we
want the experience of the template to be consistent regardless of how the
end-user obtains the template. This is why I would consider this
information to belong as part of the HOT specification.

This is relevant to the Glance conversation because, just by the template
existing in a different format than an image, we have already introduced
logic in Glance that cannot be abstracted. We could most likely build an
abstraction layer that covers some large percentage of the overlap in CRUD
operations between images and templates but I do not think that we can
expect to do this for every use case with Heat templates.

I know that I cannot provide a future use case that would exemplify this
problem but I can imagine that if we choose this design path going
forward, then we will continue to increase the scope of Glance with every
new type of object (maybe Solum assemblies?).

I do agree that in general, we should continue to attempt to drive
consistency and congruency across Openstack projects when possible. I am
just concerned that this design will take us down rabbit holes both now
and in the future when it comes to the implementation details. In the
bigger picture I am attempting to weigh the value of building this
complexity into Glance against putting Heater into its own project or Heat.

>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org

More information about the OpenStack-dev mailing list