<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Hi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Thanks for the follow-up. I was able to find this cause in the IRC channel. I ultimately upgraded the other nodes to 20.04 in our test clusters and moved the quarantined objects back to where they belonged. From there the files were downloadable again.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
We are going to try to create a new liberasurecode package 1.6.2 for 20.04 so we can set the environment variable to write legacy CRC headers until all the nodes in the cluster can be upgraded. It is hard to find the information about the bug pre-upgrade. I
didn't see it in the release notes for 2.25.2 (well they don't exist) and I don't see anything about it in the main Ubuntu Release notes. This is why we have testing environments.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">Reid</span><br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Clay Gerrard <clay.gerrard@gmail.com><br>
<b>Sent:</b> Sunday, October 2, 2022 17:41<br>
<b>To:</b> Pete Zaitcev <zaitcev@redhat.com><br>
<b>Cc:</b> Reid Guyett <rguyett@datto.com>; openstack-discuss@lists.openstack.org <openstack-discuss@lists.openstack.org>; Matthew Grinnell <mgrinnell@datto.com><br>
<b>Subject:</b> Re: [Swift][Ussuri] Erasure Coding Quarantines</font>
<div> </div>
</div>
<div>
<table id="x_graphusbanner" align="center" cellspacing="0" cellpadding="0" style="width:100%; background-color:#fcfa7e">
<tbody>
<tr>
<td valign="middle" style="">
<table align="left" cellspacing="0" cellpadding="0" style="background-color:#fcfa7e; color:#000000; width:98%; margin-bottom:5px; margin-top:5px; margin-left:1%">
<tbody>
<tr>
<td><b>KASEYA</b></td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="width:100%; background-color:#fcfa7e">
<div style="text-align:left; vertical-align:middle; zoom:1; margin-left:1%; margin-right:1%; color:#000000">
<b>Warning:</b> Sender <font style="display:none">@</font>clay.gerrard@gmail.com is not yet trusted by your organization.
<br>
Please be careful before replying or clicking on the URLs. <br>
</div>
</td>
</tr>
<tr>
<td style="width:100%; background-color:#fcfa7e">
<div style="text-align:center; vertical-align:middle; zoom:1"><a href="https://cloud.graph.us/feedback?msgId=PENBK19KS3pyYjdEeUVweTE3c01UcUxqbVhMckJoSjRhV0NuUVk1TERzVzRYV0V6WFdiUUBtYWlsLmdtYWlsLmNvbT4%3D&orgDomain=a2FzZXlhLmNvbQ%3D%3D&opt=unsafe" title="Delete & Report Message to IT" style="color:#FF0000; margin-right:5px">Report
Phishing</a> <a href="https://cloud.graph.us/feedback?msgId=PENBK19KS3pyYjdEeUVweTE3c01UcUxqbVhMckJoSjRhV0NuUVk1TERzVzRYV0V6WFdiUUBtYWlsLmdtYWlsLmNvbT4%3D&orgDomain=a2FzZXlhLmNvbQ%3D%3D&opt=safe" title="Remove Banner" style="color:#228B22; margin-right:5px">
Mark as Safe</a> </div>
</td>
</tr>
<tr>
<td style="width:100%; background-color:#fcfa7e">
<div style="text-align:right; vertical-align:middle; zoom:1; margin-left:1%; margin-right:1%; margin-bottom:2px; color:#000000">
powered by Graphus® </div>
</td>
</tr>
</tbody>
</table>
<p></p>
<div style="">
<p class="x_MsoNormal" style="background:#FFEB9C’"><b><span style="color:#9C6500′"></span></b><span style="color:black’">[EXTERNAL]</span></p>
</div>
<div>
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Fri, Sep 30, 2022 at 4:56 PM Pete Zaitcev <
<a href="mailto:zaitcev@redhat.com">zaitcev@redhat.com</a>> wrote: <br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<br>
Unfortunately, I'm not familiar with the exact details of this. <br>
There was a window where depending on how linker worked, our <br>
code could get linked with an incorrect zlib crc routine randomly. <br>
<br>
</blockquote>
<div><br>
</div>
<div>
<div class="x_gmail_quote">
<div># When upgrading from liberasurecode<=1.5.0, you may want to continue writing
<br>
# legacy CRCs until all nodes are upgraded and capabale of reading fragments <br>
# with zlib CRCs. liberasurecode>=1.6.2 checks for the environment variable <br>
# LIBERASURECODE_WRITE_LEGACY_CRC; if set (value doesn't matter), it will use <br>
# its legacy CRC. Set this option to true or false to ensure the environment <br>
# variable is or is not set. Leave the option blank or absent to not touch <br>
# the environment (default). For more information, see <br>
# <a href="https://bugs.launchpad.net/liberasurecode/+bug/1886088">https://bugs.launchpad.net/liberasurecode/+bug/1886088</a>
<br>
# write_legacy_ec_crc = <br>
<br>
</div>
<div><a href="https://github.com/NVIDIA/swift/blob/master/etc/proxy-server.conf-sample#L326-L334">https://github.com/NVIDIA/swift/blob/master/etc/proxy-server.conf-sample#L326-L334</a>
<br>
</div>
<div><br>
</div>
<div>set it in your object-server [DEFAULT] confs too </div>
</div>
<div></div>
</div>
<div><br>
</div>
<div>-- <br>
</div>
</div>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">Clay Gerrard </div>
</div>
</div>
</div>
</div>
<p style="font-size:8pt; line-height:10pt; font-family: 'Verdana','Verdana',serif; color: gray">
<b>Important Notice:</b> This email is intended to be received only by persons entitled to receive the confidential and legally privileged information it presumptively contains, and this notice constitutes identification as such. Any reading, disclosure, copying,
distribution or use of this information by or to someone who is not the intended recipient, is prohibited. If you received this email in error, please notify us immediately at legal@kaseya.com, and then delete it. To opt-out of receiving emails
<a href="https://info.kaseya.com/email-subscription-center.html">Please click here</a>. The term 'this e-mail' includes any and all attachments.</p>
</body>
</html>