Open Stack

Hi Nikhil,

Thanks so much for you response.

1) No, this is a private cloud.
2) Glance v1 (this problem has manifested itself in one of our oldest
deployments, which is running Icehouse).
3) No, location is not exposed.
4) Glance is setup with the filesystem backend drive, using a Gluster
volume mounted on the host..
5.1) Images were in active state, even though the image file had zero bytes.
5.2) very low, it may have happened only twice in the last year.

Even if the location is not exposed, there are a number of things that can
happen to the actual images files after they've been uploaded to Glance,
without Glance noticing, depending how reliable your storage backend is.
That's why I thought, in some circumstances, it would be useful to have
some sort of background service checking that image files haven't been
corrupted or gone missing altogether.

Sergio


On Mon, Sep 12, 2016 at 7:27 PM, Nikhil Komawar <nik.komawar at gmail.com>
wrote:

>
> Hi Sergio,
>
> Thanks for reaching out. And this is an excellent question.
>
> Firstly, I'd like to mention that Glance is built-in (and if deployed
> correctly) is self-resilient in ensuring that you do NOT need an audit
> of such files. In fact, if any operator (particularly large scale
> operator) needs such a system we have a serious issue where potentially
> important /user/ data is likely to be lost resulting in legal issues (so
> please beware).
>
> Having said that, I'd like to start investigating more into your
> particular issue and see where we may be missing out in ensuring data
> integrity in Glance. Let me ask you a first few set of questions that
> will help us get an initial understanding:-
>
> 1) Are you a public cloud vendor; in particular, have you deployed
> glance to potentially non-trusted users? or is the case otherwise?
> 2) Are you deploying Glance v1 or Glance v2?
> 3) Have you exposed the "location" feature set (CRUD) to regular users?
> (if using API v2, have you enabled ``show_multiple_locations``
> configuration)
> 4) What backends have you configured Glance with and who has access to
> them? What is the resiliency or rotation (of disks) (for say capacity
> management) of your backend store system?
> 5) Sanity check on your issue:-
> 5.1) What are the image statues for which the image data files are missing?
> 5.2) What is the rate of error approximately (if you don't have
> specifics, info like rare, medium, often will help)
>
>
> We may have to dig a bit further into the issue but this set of info
> should help us narrow down the issue and determine if there are any gaps
> in Glance.
>
> P.S. Please use the tag "[glance]" in the subject line to help us get to
> your email faster.
>
> On 9/12/16 12:48 PM, Sergio A. de Carvalho Jr. wrote:
> > Hi all,
> >
> > Is there (or was there ever) any plans to implement in Glance a
> > service that would periodically check that the image files are still
> > available on the file system (or in whatever storage system being
> > used) and have the correct checksum?
> >
> > We had a few issues where an image file was removed from the
> > filesystem and that can go undetected for a long time until someone
> > tries to access that image, so we were wondering if it would be
> > possible (and if it would make sense) to implement some sort of
> > background service to periodically check if all images found in the
> > database can be retrieved successfully.
> >
> > Thoughts?
> >
> > Sergio
> >
> >
> >
> >
> > ____________________________________________________________
> ______________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
>
>
> --
>
> Thanks,
> Nikhil
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160912/048a73ef/attachment.html>