Open Stack

Excerpts from Jay Pipes's message of 2013-12-06 16:46:24 -0800:
> On 12/06/2013 01:38 PM, Clint Byrum wrote:
> > Excerpts from Jay Pipes's message of 2013-12-05 21:32:54 -0800:
> >> On 12/05/2013 04:25 PM, Clint Byrum wrote:
> >>> Excerpts from Andrew Plunk's message of 2013-12-05 12:42:49
> >>> -0800:
> >>>>> Excerpts from Randall Burt's message of 2013-12-05 09:05:44
> >>>>> -0800:
> >>>>>> On Dec 5, 2013, at 10:10 AM, Clint Byrum <clint at
> >>>>>> fewbar.com> wrote:
> >>>>>>
> >>>>>>> Excerpts from Monty Taylor's message of 2013-12-04
> >>>>>>> 17:54:45 -0800:
> >>>>>>>> Why not just use glance?
> >>>>>>>>
> >>>>>>>
> >>>>>>> I've asked that question a few times, and I think I can
> >>>>>>> collate the responses I've received below. I think
> >>>>>>> enhancing glance to do these things is on the table:
> >>>>>>>
> >>>>>>> 1. Glance is for big blobs of data not tiny templates. 2.
> >>>>>>> Versioning of a single resource is desired. 3.
> >>>>>>> Tagging/classifying/listing/sorting 4. Glance is designed
> >>>>>>> to expose the uploaded blobs to nova, not users
> >>>>>>>
> >>>>>>> My responses:
> >>>>>>>
> >>>>>>> 1: Irrelevant. Smaller things will fit in it just fine.
> >>>>>>
> >>>>>> Fitting is one thing, optimizations around particular
> >>>>>> assumptions about the size of data and the frequency of
> >>>>>> reads/writes might be an issue, but I admit to ignorance
> >>>>>> about those details in Glance.
> >>>>>>
> >>>>>
> >>>>> Optimizations can be improved for various use cases. The
> >>>>> design, however, has no assumptions that I know about that
> >>>>> would invalidate storing blobs of yaml/json vs. blobs of
> >>>>> kernel/qcow2/raw image.
> >>>>
> >>>> I think we are getting out into the weeds a little bit here. It
> >>>> is important to think about these apis in terms of what they
> >>>> actually do, before the decision of combining them or not can
> >>>> be made.
> >>>>
> >>>> I think of HeatR as a template storage service, it provides
> >>>> extra data and operations on templates. HeatR should not care
> >>>> about how those templates are stored. Glance is an image
> >>>> storage service, it provides extra data and operations on
> >>>> images (not blobs), and it happens to use swift as a backend.
> >>>>
> >>>> If HeatR and Glance were combined, it would result in taking
> >>>> two very different types of data (template metadata vs image
> >>>> metadata) and mashing them into one service. How would adding
> >>>> the complexity of HeatR benefit Glance, when they are dealing
> >>>> with conceptually two very different types of data? For
> >>>> instance, should a template ever care about the field "minRam"
> >>>> that is stored with an image? Combining them adds a huge
> >>>> development complexity with a very small operations payoff, and
> >>>> so Openstack is already so operationally complex that HeatR as
> >>>> a separate service would be knowledgeable. Only clients of Heat
> >>>> will ever care about data and operations on templates, so I
> >>>> move that HeatR becomes it's own service, or becomes part of
> >>>> Heat.
> >>>>
> >>>
> >>> I spoke at length via G+ with Randall and Tim about this earlier
> >>> today. I think I understand the impetus for all of this a little
> >>> better now.
> >>>
> >>> Basically what I'm suggesting is that Glance is only narrow in
> >>> scope because that was the only object that OpenStack needed a
> >>> catalog for before now.
> >>>
> >>> However, the overlap between a catalog of images and a catalog
> >>> of templates is quite comprehensive. The individual fields that
> >>> matter to images are different than the ones that matter to
> >>> templates, but that is a really minor detail isn't it?
> >>>
> >>> I would suggest that Glance be slightly expanded in scope to be
> >>> an object catalog. Each object type can have its own set of
> >>> fields that matter to it.
> >>>
> >>> This doesn't have to be a minor change to glance to still have
> >>> many advantages over writing something from scratch and asking
> >>> people to deploy another service that is 99% the same as Glance.
> >>
> >> My suggestion for long-term architecture would be to use Murano
> >> for catalog/metadata information (for images/templates/whatever)
> >> and move the block-streaming drivers into Cinder, and get rid of
> >> the Glance project entirely. Murano would then become the
> >> catalog/registry of objects in the OpenStack world, Cinder would be
> >> the thing that manages and streams blocks of data or block devices,
> >> and Glance could go away. Imagine it... OpenStack actually
> >> *reducing* the number of projects instead of expanding! :)
> >
> > Have we not learned our lesson with Nova-Net/Neutron yet? Rewrites
> > of existing functionality are painful.
> 
> I said *long-term* above, Clint.
> 
> > The Murano-concerned people have already stated they are starting
> > over on that catalog.
> >
> > I suggest they start over by expanding Glance's catalog. If the
> > block streaming bits of Glance need to move somewhere else, that
> > sounds like a completely separate concern that distracts from this
> > point.
> 
> Yes, the block streaming bits of Glance are separate -- sorry for
> distracting from your point. However, Glance's architecture actually
> went from having the registry and streaming pieces separated to having
> the two pieces more closely interwoven in the last two release cycles.
> It's harder now, IMO, to separate out the "registry" aspects of Glance 
> from the block management pieces. Which is why I suggested using Murano 
> as the catalog since there is already a separate catalog sub-project in 
> Murano and the Murano devs are familiar with this space.
> 
> Please try not to shoot the messenger here -- I'm only bringing up my 
> thoughts on where the various related projects seem to be moving.
> 
> > And to be clear, (I think I will just stop talking as I think I've
> > made this point), my point is, we have a catalog, let's make it
> > better.
> 
> Well, we have multiple catalogs... which one shall we make better? 
> Sounds like contributors are coalescing around using Glance as the 
> catalog -- which is fine I suppose -- but I think it would have been 
> easier to use the old Glance Registry server (which now does not exist) 
> as this catalog service, than the newer Glance API, which has gotten 
> sidetracked with things like image tasks (which IMO should never have 
> been added to the Images API; the taskflow or Mistral projects should 
> have handled this functionality).
> 
> Anyway, just figured I'd offer some commentary. Sorry if you think I was 
> causing trouble.
> 

Your commentary is much appreciated, and I apologize for shooting you,
Mr. Messenger. It really is your fault for being such an easy target! ;)