Open Stack

-----Original Message-----
From: Clint Byrum <clint at fewbar.com>
Reply: OpenStack Development Mailing List (not for usage questions)
<openstack-dev at lists.openstack.org>
Date: February 13, 2017 at 13:41:00
To: openstack-dev <openstack-dev at lists.openstack.org>
Subject:  Re: [openstack-dev] [tc][glance][glare][all]
glance/glare/artifacts/images at the PTG

> Excerpts from Mikhail Fedosin's message of 2017-02-13 18:23:19 +0300:
> > Hello!
> >
> >
> > Let me quickly describe my vision of the problem. I asked this question to
> > Brian last Friday, because it is evident that the projects have the
> > intersection in functionality. For this reason, I proposed to bring Glare
> > back and develop it as a new generation of Glance service. Perhaps such a
> > solution would be more correct from my point of view.
> >
> > Moving away from Glance, let me remind you why we created Glare service.
> >
> > Almost every project works with some binary data and must store it
> > somewhere, and almost always storage itself is not the part of the
> > project's mission. This issue has often been neglected. For this reason
> > there is no single recommended method for storing of binary data, which
> > would have a unified public api and hide all the things of the internal
> > storage infrastructure.
> >
>
> We have an awesome service for storing binary data in a hierarchical
> format in Swift. But it's so generic, it can't really just be the image
> service. But something like Glare is just a way to scope it down and
> give us a way to ask for "just the images" or "just the heat templates",
> which I think is a natural thing for cloud users to want.
>
> > These questions were answered by Glare. First of all, the service allows to
> > use different storages for various types of artifacts - an operator can
> > assign the storage of large files, such as virtual machine images, to
> > Swift, and for relatively small ones, such as Heat templates, use a mysql
> > database.
> >
>
> Meh. Swift isn't exactly slow, or cumbersome for small files.
>
> > Then, we have to admit that data tends to change, so we added a versioning
> > of artifacts and dependencies between them, that the user was convenient to
> > take the data of required version.
> >
>
> Any attempt at versioning that is not git, will frustrate any git user.
> This cat's already out of the bag, but I'd suggest adding git repositories
> as a blob container type and finding a way to allow git to push/pull
> to/from swift. That would be an amazing feature _for swift_ anyway
> (maybe it already exists?) but it would allow Glare to piggy back on all
> of the collective versioning capabilities in Git rather than having to
> chase git.

So the versioning that's present will frustrate everyone. The
reasoning for it is that the original Glare developers found a hack
online to convert the version string into something that a database
can sort (by turning it into one giant integer basically). (I'm
certain that's not the only reason, but when challenged with several
other options they said they couldn't find anyone who had already
found a way to make it sortable on the version.)

That aside, I'm not sure anyone wants git (even git-lfs) managing 50GB
images for them.

> > Often a "binary data" refers to more than one specific object, but a whole
> > lot of files. Therefore, we have implemented the ability to create
> > arbitrary nested folders per one artifact and store multiple files there.
> > And for sure users can receive any file with a single API request.
> >
>
> See above: IMO, use git for this as well and just teach Glare to
> understand git repos rather than having to implement folders in databases.
>
> > For validation and conversion of uploaded data Glare introduces the concept
> > of hooks for the operation. Thus the operator can extend the basic
> > functionality of the system and add integration with third-party systems
> > for each artifact type. For example, for Nokia we implemented integration
> > with custom TOSCA validator.
> >
>
> At first this set off my interop alarm, but it was a false alarm as long
> as this is always limited to 3rd party systems. What worries me is when
> somebody adds one of these for something already in OpenStack, and now
> suddenly a perfectly interoperable app only works right on that one
> special cloud.
>
> > This is just a small overview of the key features of the service. For sure,
> > at the moment Glare is able to do all that Glance can do (except maybe a
> > sharing of artifacts), on the other hand we have added a number of new
> > features, that were requested by cloud operators for a long time.
> >
> > Fyi, now we in Nokia are preparing additional API, which corresponds to the
> > ETSI VNF Packaging Specification [1]. So support of Image v2 API is not an
> > impossible task, and we may implement it as an alternative way of
> > interaction with "Images" artifact type. In this case Nova and other
> > services using Glance are absolutely indifferent to what service provides
> > Image API.
> >
>
> If you can make it 100% API compatible with Image v2, you'll go a long way
> to helping users smoothly switch over.
>
> > All tasks related to documentation and packaging are solvable. We’re
> > working on them together with Nokia, so I can assure you that the documents
> > and packages will be available this spring. The same story is for Ansible
> > and Puppet.
> >
> > Now back again to our question. What I'd like is that Glare will receive
> > due recognition. Doing a project on the outskirts of OpenStack is not I
> > really want to. Therefore, it would be nice to develop Glare as a natural
> > evolution of Glance, associated with the requirements of operators and the
> > market in general. For Glance team is a good chance to try something new
> > and interesting, and of course gain new experience.
> >
>
> I support you in your attempt to have a natural evolution. I think
> it's going to be harder and harder the longer you're developing Glare's
> features without pushing for a transition to it. The heavier weight it
> gets, the less people will want it (again, remember how hard it was to
> get the community behind Neutron?)
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

--
Ian Cordasco