<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Sep 13, 2016 at 7:16 AM, Nikhil Komawar <span dir="ltr"><<a href="mailto:nik.komawar@gmail.com" target="_blank">nik.komawar@gmail.com</a>></span> wrote:</div><div class="gmail_quote"><span style="color:rgb(80,0,80)">> Firstly, I'd like to mention that Glance is built-in (and if deployed</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> correctly) is self-resilient in ensuring that you do NOT need an audit</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> of such files. In fact, if any operator (particularly large scale</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> operator) needs such a system we have a serious issue where</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> potentially</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> important /user/ data is likely to be lost resulting in legal</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> issues (so</span><br style="color:rgb(80,0,80)"><span style="color:rgb(80,0,80)">> please beware).</span><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">Can you please elaborate on how Glance is self-resilient?</div><div class="gmail_quote"><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hey Sergio,<br>
<br>
<br>
Glad to know that you're not having any feature related issues (to me<br>
this is a good sign). Based on your answers, it makes sense to require a<br>
reliability solution for backend data (or some sort of health monitoring<br>
for the user data).<br></blockquote><div><br></div><div>All backends will at some point lose some data. The ask is for reflecting the image's "health" to the user.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
So, I wonder what your thoughts are for such an audit system. At a first<br>
glance, this looks rather not scalable, at least if you plan to do the<br>
audit on all of the active images. Consider a deployment trying to run<br>
this for around 100-500K active image records. This will need to be run<br>
in batches, thus completing the list of records and saying that you've<br>
done a full audit of the active image -- is a NP-complete problem (new<br>
images can be introduced, some images can be updated in the meantime, etc.)<br></blockquote><div><br></div><div>NP-complete? Really? Every storage system scrubs all data periodically to protect from disk errors. Glance images should be relatively static anyway.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
The failure rate is low, so a random (sparse check) on the image data<br>
won't help either. Would a cron job setup to do the audit for smaller<br>
deployments work? May be we can look into some known cron solutions to<br>
do the trick?<br></blockquote><div><br></div><div>How about letting the backend report the health? S3, for example, reports an <a href="http://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html#supported-notification-event-types">event on object loss</a>. The S3 driver could monitor those events and update status. Swift performs scrubbing to determine object health - I haven't checked if it reports an event on object loss, but don't see any reason not to. For local filesystem, it would need its own scrubbing process (e.g., recalculate hash for each object every N days). On the other hand if it is a mount of some filer, the filer should be able to report on health.</div><div> </div><div>Thanks,</div><div>Avishay</div></div><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><b style="font-size:12.7273px"><font color="#666666">Avishay Traeger, PhD</font></b><br></div><div><font color="#666666"><i>System Architect</i></font></div><div><span style="color:rgb(102,102,102);font-size:12.7273px"><br></span></div><div><span style="color:rgb(102,102,102)">Mobile:</span><span style="color:rgb(102,102,102)"> </span><a value="+972524317955" style="color:rgb(17,85,204)">+972 54 447 1475</a><br></div><div><font color="#666666">E-mail: <a href="mailto:avishay@stratoscale.com" style="color:rgb(17,85,204)" target="_blank">avishay@stratoscale.com</a></font></div><div><font color="#666666"><br></font></div><div><img src="http://www.stratoscale.com/wp-content/uploads/Logo-Signature-Stratoscale-230.jpg"><br></div><div><font color="#666666"><br></font></div><div><p style="margin:0in"><a href="http://www.stratoscale.com/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:arial;font-size:9.75pt">Web</span></a><span style="font-family:arial;font-size:9.75pt"> | </span><a href="http://www.stratoscale.com/blog/" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:arial;font-size:9.75pt">Blog</span></a><span style="font-family:arial;font-size:9.75pt;color:rgb(108,163,214)"> | </span><a href="https://twitter.com/Stratoscale" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:arial;font-size:9.75pt">Twitter</span></a><span style="font-family:arial;font-size:9.75pt;color:rgb(108,163,214)"> | <a href="https://plus.google.com/u/1/b/108421603458396133912/108421603458396133912/posts" style="color:rgb(17,85,204)" target="_blank">Google+</a> | </span><span style="font-family:arial;font-size:9.75pt"><a href="https://www.linkedin.com/company/stratoscale" style="color:rgb(17,85,204)" target="_blank">Linkedin</a></span></p></div></div></div></div></div></div></div></div></div>
</div></div>