<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div>Hello,</div>
<div><br>
</div>
<div>Good question, I forgot to include that. SSL is not being offloaded in any environment and is being handled via the glance-api and glance-registry services. We increased the number of workers to 40, to better handle multiple downloads. In production
we are using F5’s or A10’s for load balancing in our dev/test/stage environments we are using haproxy. Issue exists in all environments. Also, in testing it did not matter the number of glance-api servers we had in rotation. To simplify troubleshooting,
I had disabled glance-api on all but one server. So most of the testing was done from a single compute node using multiple clients to a single glance-api instance (with 40 workers). To add some additional detail I am running on Centos 6.5, and I have already
tried upgrading eventlet, greenlet, pyOpenSSL, pycryptography to their latest versions on both the client and the server and it did not help.</div>
<div><br>
</div>
<div>If we turn off ssl in glance-api and the client, then 3 downloads work without issue.</div>
<div>
<div>____________________________________________</div>
<div> </div>
<div>Kris Lindgren</div>
<div>Senior Linux Systems Engineer</div>
<div>GoDaddy, LLC.</div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>John Dewey <<a href="mailto:john@dewey.ws">john@dewey.ws</a>><br>
<span style="font-weight:bold">Date: </span>Friday, July 11, 2014 at 10:22 PM<br>
<span style="font-weight:bold">To: </span>"Kris G. Lindgren" <<a href="mailto:klindgren@godaddy.com">klindgren@godaddy.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a>" <<a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Openstack-operators] Glance + SSL - Image download issues?<br>
</div>
<div><br>
</div>
<div>
<div>
<div>What are you offloading SSL to (haproxy, pound, hw lb)? If you turn off SSL, and traverse the same load balanced path do you still run into problems with three simultaneous downloads?
</div>
<div></div>
<p style="color: #A0A0A8;">On Friday, July 11, 2014 at 3:33 PM, Kris G. Lindgren wrote:</p>
<blockquote type="cite" style="border-left-style:solid;border-width:1px;margin-left:0px;padding-left:10px;">
<span>
<div>
<div>
<div>Hello,</div>
<div><br>
</div>
<div>Wondering if anyone is running glance+ssl in production? I am running on havana 2013.2.3 code base and I am having intermittent issues with backing files not downloading for deploying vms. To trouble shoot the issue some more I create some scripts and
I have found that with 3 parallel image downloads on the same compute node with ssl enabled in glance typically one or two of the images will fail (most of the time 2) to successful download. I have filed bug: <a href="https://bugs.launchpad.net/glance/+bug/1340993">https://bugs.launchpad.net/glance/+bug/1340993</a>
which includes links to the scripts. Is anyone else running glance+ssl in production and have vm’s that get stuck in spawning state – and when you investigate the backing file is only partially downloaded and hasn’t been modified in a very long time?</div>
<div><br>
</div>
<div>If so can you please try to run either: <a href="https://gist.github.com/krislindgren/fc519aa03d350f42e9e6#file-multiboot-sh">https://gist.github.com/krislindgren/fc519aa03d350f42e9e6#file-multiboot-sh</a> or <a href="https://gist.github.com/krislindgren/fc519aa03d350f42e9e6#file-multi-img-download-sh">https://gist.github.com/krislindgren/fc519aa03d350f42e9e6#file-multi-img-download-sh</a> and
see if your results duplicate my own?</div>
<div><br>
</div>
<div>
<div>
<div>____________________________________________</div>
<div> </div>
<div>Kris Lindgren</div>
<div>Senior Linux Systems Engineer</div>
<div>GoDaddy, LLC.</div>
<div>Email: <a href="mailto:klindgren@godaddy.com">klindgren@godaddy.com</a></div>
<div><br>
</div>
<div>This email message and any attachment(s) hereto are intended for use only by its intended recipient(s) and may contain confidential information. If you have received this email in error, please immediately notify the sender and permanently delete the original
and any copy of this message and its attachments.</div>
</div>
<div><br>
</div>
</div>
</div>
<div>
<div>_______________________________________________</div>
<div>OpenStack-operators mailing list</div>
<div><a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a></div>
<div><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a></div>
</div>
</div>
</span></blockquote>
<div><br>
</div>
</div>
</div>
</span>
</body>
</html>