Open Stack

Excerpts from Tim Schnell's message of 2013-12-05 13:07:17 -0800:
> On 12/5/13 12:17 PM, "Clint Byrum" <clint at fewbar.com> wrote:
> 
> >Excerpts from Tim Schnell's message of 2013-12-05 09:49:03 -0800:
> >> 
> >> On 12/5/13 11:33 AM, "Randall Burt" <randall.burt at RACKSPACE.COM> wrote:
> >> 
> >> >On Dec 5, 2013, at 11:10 AM, Clint Byrum <clint at fewbar.com>
> >> > wrote:
> >I would love to hear about "future requirements" for Heater that won't
> >fit into Glance. But thus far, I am struggling to see where this could
> >go that wouldn't be a place Glance also wants to go.
> 
> I know we covered this a bit in the hangout but one possibility that I
> think I failed to articulate is the fact that the template is a parseable
> data representation while an image is a binary object. In my original
> email I mentioned having a discussion about moving some of the "template"
> metadata like application information and documentation into the HOT
> specification.
> 

I've gone on record before as saying that having things in the template
schema that should be in the catalog is a mistake. I stand by that
assertion. However that is just for HOT, I think for a full blown
packaging format, that may not be the case and so I agree there may be
a case for parsing the contents of the uploaded object.

> This has important ramifications for the user experience as well as
> whether or not abstracting Glance to "one-size-fits-all" is actually
> feasible at an implementation level. Ideally the Heat template would serve
> as the source of truth for information like "application-version" and
> "template documentation", in the original Heater proposal this is
> important because this data would have to be extracted from the template
> and stored in a metadata field if it is relevant to the way that Heater
> allows you to search or index different templates.
> 

Given a separate service, how wold you design things differently than
glance to make this parsing work. As I recall, when you create an
image for glance, it just makes a record in the registry and a place to
upload it to in the storage. Would you do this differently? Does that
invalidate parsing?

Presumably after uploading your template to storage, you would need a
process which inspects the data and populates the registry. How would
glance's design preclude this given the more generic object schema that
we discussed.

> If we had to pass a duplicate version of this metadata in the HTTP Request
> we could avoid grabbing this information from the template but we open up
> data integrity issues. What happens when the end-user updates the metadata
> but not the template? At the same time, I would reiterate that we can't
> just make this data a "Heater/Template Catalog" metadata issue because we
> want the experience of the template to be consistent regardless of how the
> end-user obtains the template. This is why I would consider this
> information to belong as part of the HOT specification.
> 

Don't take this the wrong way, but that would be absurd. Either you
maintain it as the source of truth, or it is derived from the source
of truth. "What if" I upload lolcatz instead of a template? I can
has meta-data where there is none? Let's stick to the main question:
Can Glance's scope be expanded to have data derived from the objects
themselves. I think the answer is a resounding yes.

> This is relevant to the Glance conversation because, just by the template
> existing in a different format than an image, we have already introduced
> logic in Glance that cannot be abstracted. We could most likely build an
> abstraction layer that covers some large percentage of the overlap in CRUD
> operations between images and templates but I do not think that we can
> expect to do this for every use case with Heat templates.
> 

Who said that there would not be logic per object type? The point was to
have a generic object registry, but not an abstract object registry. You
still instantiate them by uploading content and in doing so the objects
take on all of the attributes of their class. So images are opaque.
Things that allow parsing are not.

> I know that I cannot provide a future use case that would exemplify this
> problem but I can imagine that if we choose this design path going
> forward, then we will continue to increase the scope of Glance with every
> new type of object (maybe Solum assemblies?).
> 

If Heater were its own service, and Solum assemblies were added,
requiring some new things like multi-file objects, would you support
Solum developers wanting to write a new registry because their use-case
is slightly different?

> I do agree that in general, we should continue to attempt to drive
> consistency and congruency across Openstack projects when possible. I am
> just concerned that this design will take us down rabbit holes both now
> and in the future when it comes to the implementation details. In the
> bigger picture I am attempting to weigh the value of building this
> complexity into Glance against putting Heater into its own project or Heat.
> 

Writing a new service will take you down much deeper rabbit holes that
look a lot like the rabbit holes the Glance developers already went down,
only you won't have all of the collected wisdom that is wrapped up in
a service that has existed and been in use doing 90% of what you are
suggesting must be re-done in a slightly different way.