<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Hello,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I'm not sure if it is related, but I am testing reimaging a node from Ubuntu 18.04 to 20.04 (Cloud Archive Ussuri 2.25.1 to Focal Repo Ussuri 2.25.2) in a 3-node cluster. I have a lot of servers to upgrade, and the environments will be mixed for a while, so
I have left this mixed for a week. Now the 18.04 servers are starting to accumulate quarantined EC objects.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
The 20.04 server has log messages indicating that it is unable to get enough responses from the reconstructor.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
The 18.04 servers have a couple EC messages.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
"...liberasurecode[30807]: Invalid fragment header information!" followed by "...object-server: Quarantined object /srv/node/d9/objects-4/10321/b2e/50a3dce515aafc6f222824ec5c7dfb2e/1664002866.33100#8#d.data: Invalid EC metadata at offset 0x0<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Is there any thing you can think of that would cause this issue?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
My steps to reproduce:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<ol>
<li><span>Create a container in an EC policy</span></li><li><span>Upload object to EC container</span></li><li>Wait a bit of time (It looks like the object-server is quarantining but not sure on the trigger)</li><li>Try to download the file and it will give a 504</li></ol>
<div>Looking at python3-swift dependencies and the package versions I thought may affect this:</div>
<div>20.04</div>
<div>libjerasure2 = 2.0.0+2017.04.10.git.de1739cc84-1<br>
</div>
<div>python3-pyeclib = 1.6.0-6build1<br>
</div>
<div>python3-xattr = 0.9.6-1.1<br>
</div>
<div><br>
</div>
<div>18.04</div>
<div>libjerasure2 = 2.0.0+2017.04.10.git.de1739cc84-1<br>
</div>
<div>python3-pyeclib = 1.3.1-1ubuntu3<br>
</div>
<div>python3-xattr = 0.9.2-0ubuntu1<br>
</div>
<div><br>
</div>
<div>Thanks!</div>
<div>Reid</div>
</div>
<p style="font-size:8pt; line-height:10pt; font-family: 'Verdana','Verdana',serif; color: gray">
<b>Important Notice:</b> This email is intended to be received only by persons entitled to receive the confidential and legally privileged information it presumptively contains, and this notice constitutes identification as such. Any reading, disclosure, copying,
distribution or use of this information by or to someone who is not the intended recipient, is prohibited. If you received this email in error, please notify us immediately at legal@kaseya.com, and then delete it. To opt-out of receiving emails
<a href="https://info.kaseya.com/email-subscription-center.html">Please click here</a>. The term 'this e-mail' includes any and all attachments.</p>
</body>
</html>