Open Stack

Over the holidays the infra team noticed that some of our release artifacts were in the wrong file format. On further investigation we discovered the reason for this was some swift implementations were inflating compressed tarballs when retrieved by clients not setting accept-encoding: gzip. We then ended up with tar files when we expected .tar.gz format files.

This behavior seems to be controlled by the content-encoding we set on object upload. If we tell swift that the object is a gzip'd file on upload then the webservers helpfully inflate them when a client retrieves them. In order to fix our release artifacts we have updated our swift upload tooling to stop setting content-type on gzip files. This forces the swift implementation to return the files in the same format they are uploaded when retrieved.

A side effect of this is that any gzip'd files (like testr_results.html.gz) are no longer automatically decompressed for you when you retrieve them. We have fixes up to handle common occurrences of this at https://review.opendev.org/#/c/701578/ and https://review.opendev.org/701282 (note the second case is already handled on OpenDev's Zuul).

If you run into other files you expect to be browesable but get compressed instead, the fix is to stop compressing the files explicitly in the job. We will still upload files in compressed form to swift for efficiency but we should operate on these files as if they were uncompressed. Then for files that should be compressed (like .tar.gz files) we can pass them through as is avoiding any format change problems.

Clark