Open Stack

To be clear, Randall is referring to a swift container (directory).

Murali has a good idea of attempting to use swift client first, as it has performance optimizations that can speed up the process more than naive file transfer tools. I did mention to him that wget does have a retiree feature, and that we could see about using curl instead to allow for chunked encoding as additional optimizations.

Randall, are you suggesting that we could use swift client for both private and public LP uses? That sounds like a good suggestion to me.

Adrian

On Jun 17, 2015, at 11:10 AM, Randall Burt <randall.burt at RACKSPACE.COM<mailto:randall.burt at RACKSPACE.COM>> wrote:

Can't an operator make the target container public therefore removing the need for multiple access strategies?

-------- Original message --------
From: Murali Allada
Date:06/17/2015 11:41 AM (GMT-06:00)
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: [openstack-dev] [Solum] Supporting swift downloads for operator languagepacks

Hello Solum Developers,

When we were designing the operator languagepack feature for Solum, we wanted to make use of public urls to download operator LPs, such as those available for CDN backed swift containers we have at Rackspace, or any publicly accessible url. This would mean that when a user chooses to build applications on to​​p of a languagepack provided by the operator, we use a url to 'wget' the LP image.

Recently, we have started noticing a number of failures because of corrupted docker images downloaded using 'wget'. The docker images work fine when we download them manually with a swift client and use them. The corruption seem to be happening when we try to download a large image using 'wget' and there are dropped packets or intermittent network issues.

My thinking is to start using the swift client to download operator LPs by default instead of wget. The swift client already implements retry logic, downloading large images in chunks, etc. This means we would not get the niceties of using publicly accessible urls. However, the feature will be more reliable and robust.

The implementation would be as follows:

  *   ​We'll use the existing service tenant configuration available in the solum config file to authenticate and store operator languagepacks using the swift client. We were using a different tenant to build and host LPs, but now that we require the tenants credentials in the config file, it's best to reuse the existing service tenant creds. Note: If we don't, we'll have 3 separate tenants to maintain.
     *   ​Service tenant
     *   Operator languagepack tenant
     *   Global admin tenant
  *   I'll keep the option to download the operator languagepacks from a publicly available url. I'll allow operators to choose which method they want to use by changing a setting in the solum config file.

FYI: In my tests, I've noticed that downloading an image using the swift client is twice as fast as downloading the same image using 'wget' from a CDN url.

Thanks,
Murali

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150617/e743a6f4/attachment.html>