Open Stack

On 2017-04-28 14:39:51 +0200 (+0200), Luigi Toscano wrote:
> The Sahara project has been providing pre-built images containing
> the Hadoop/ Spark/$bigdata frameworks since the beginning of the
> project, so that users can be immediately productive.
> 
> The generated qcow2 images have been living so far here:
> http://sahara-files.mirantis.com/images/upstream/
> 
> As a team we were wondering whether we could store those images on
> some shared and publicly accessible space on openstack.org (like
> tarballs.openstack.org).
> 
> I guess that the main concern could be the disk usage. Currently
> the space used for the older releases (from kilo to newton) is
> around ~110GB. The estimate for Ocata is ~35GB and the number is
> going to grow. Of course we can drop old images when a certain
> release reaches its end-of- life (unless there is a place to store
> some archived artifacts).
> 
> About the update frequency: the images are currenctly rebuilt with
> with every commit in sahara-image-elements (and soon in sahara
> with a different build method) by the tests. I don't think that we
> would need to update the images in this stored space with every
> commit, but at most once every month or, even better, when a new
> release of sahara-image-elements is tagged.
> 
> Please note that we already store some artifacts on
> tarballs.openstack.org, even if their size is not definitely not
> the same of those disk images.
> https://review.openstack.org/#/c/175395/
> https://review.openstack.org/#/c/367271/
> 
> To summarize: would it be possible for us to use some shared
> space, and if yes, which are the conditions?

Apologies for the delay in responding. We don't currently have
sufficient free space to store the quantity of data you're talking
about (some projects like Ironic, Trove and Kolla do or have in the
past uploaded guest images there, but those are far fewer and much
smaller than what you're requesting). We can see about extending the
available free space for tarballs.openstack.org after we relocate it
off the same server where we store our job logs, but we don't have a
timeline for when that will be. I'm sorry I don't have better news
on that front.

On a separate note, what degree of security support is being
provided for those images (as far as known vulnerabilities in
non-OpenStack software aggregated within them)? There is still some
concern expressed within the community around producing images of
this nature for any purpose other than use within our CI system, in
which case long-term archiving of release versions of those images
is unnecessary. If you need a periodic job to upload and replace
pregenerated branch-tip snapshot images for consumption by other CI
jobs, we should be able to work something out pretty easily (that's
what the other projects I mentioned above have been doing).
-- 
Jeremy Stanley