<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="color: rgb(0, 0, 0);">I’ve got Glance running on a Ceph Swift backend store (NOT the OpenStack implementation of Swift.) I’m noticing a problem around large images and the checksums/ETags on the manifest object. I’m seeing a 422 Unprocessable
Entity response from Swift on those PUTs.</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div style="color: rgb(0, 0, 0);">(* Background on large objects in Swift below.)</div>
<div style="color: rgb(0, 0, 0);"><br>
</div>
<div><span style="line-height: 21px;">I</span><font color="#000000"><span style="background-color: rgb(254, 253, 253);"><span style="line-height: 21px;"> figured out this is due to a manifest object ETag verification implementation difference between Ceph Swift
and OS Swift.</span></span></font></div>
<div><font color="#000000"><span style="background-color: rgb(254, 253, 253);"><span style="line-height: 21px;"><br>
</span></span></font></div>
<div><font color="#000000"><span style="background-color: rgb(254, 253, 253);"><span style="line-height: 21px;">OS Swift verifies it just like any other object, md5’ing the content of the object - </span></span></font><a href="https://github.com/openstack/swift/blob/master/swift/obj/server.py#L439-L459">https://github.com/openstack/swift/blob/master/swift/obj/server.py#L439-L459</a></div>
<div><br>
</div>
<div>Ceph Swift actually does the full DLO checksum across all the component objects - <a href="https://github.com/ceph/ceph/blob/master/src/rgw/rgw_op.cc#L1765-L1781">https://github.com/ceph/ceph/blob/master/src/rgw/rgw_op.cc#L1765-L1781</a></div>
<div><br>
</div>
<div>The problem comes into play in the Glance Swift store driver. It assumes the OS Swift behavior, and sends an ETag of md5(“”) in the PUT request - <a href="https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L552">https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/swift/store.py#L552</a></div>
<div><br>
</div>
<div>TBH, I don’t understand why the Swift store driver is even sending an ETag there. It would function just as well without sending an ETag at all.</div>
<div><br>
</div>
<div>Wondering if anyone else had bumped up against this? I did a basic search over the Glance bugs and I did't see anything around this, so I opened <a href="https://bugs.launchpad.net/glance/+bug/1387311">https://bugs.launchpad.net/glance/+bug/1387311</a>.
I’m surprised this hasn’t surfaced before.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Mike</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>* See <a href="http://docs.openstack.org/developer/swift/overview_large_objects.html">[1]</a> and <a href="http://docs.openstack.org/api/openstack-object-storage/1.0/content/dynamic-large-object-creation.html">[2]</a>. But basically you chunk up the object,
upload those separately, and then PUT a special manifest object which ties it all together. The Etag/checksum on the manifest object is the "<span style="background-color: rgb(255, 255, 255);"><font color="#3e4349"><span style="line-height: 21.5999984741211px;">MD5
sum of the concatenated string of ETags for each of the segments in the manifest</span><span style="line-height: 21px;">”</span><span style="line-height: 21.5999984741211px;">.</span></font></span></div>
<div><span style="background-color: rgb(255, 255, 255);"><font color="#3e4349"><span style="line-height: 21.5999984741211px;"><br>
</span></font></span></div>
<div><span style="background-color: rgb(255, 254, 254);"><font style="color: rgb(62, 67, 73);"><span style="line-height: 21.5999984741211px;">When the client provides an </span></font><span style="color: rgb(62, 67, 73); line-height: 21px;">E</span><font color="#000000"><font color="#3e4349"><span style="line-height: 21.5999984741211px;">Tag
header in the PUT request, the Swift API checks that against what it determines the </span></font><span style="line-height: 21px;">checksum for the object to be. If there is a mismatch, the response is a 422 Unprocessable Entity.</span></font></span></div>
</body>
</html>