<div dir="ltr"><div><div><div>In response to Kris,<br><br></div>On ctrl1: glance:x:112:120::/var/lib/glance:/bin/false<br></div>On ctrl2: glance:x:112:119::/var/lib/glance:/bin/false<br><br></div>120 on ctrl2 is nova, so on ctrl1 the perms would be glance:glance and on ctrl2 it would glance:nova. I would suspect this as the issue if the issue has happening on ctrl2 but it is happening on ctrl1<br></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 3 Feb 2016 at 10:34 Kris G. Lindgren <<a href="mailto:klindgren@godaddy.com">klindgren@godaddy.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>
<div>
<div>Not related to your issue, but something to keep an eye out for, is that you need to keep the uid for glance synced across your glances servers when using an nfsv3 store. Since nfsv3 stores the uid & gid for the file perms. You can run into weird issues
if glance is uid/gid 501 on one glance server and 502 on another. We had that problem crop up in production when packages were doing "useradd" without specifying a uid/gid. So you could end up with systems with different id's and permissions that are all
screwed up between multiple servers. </div>
<div><br>
</div>
<div>So related to your question .. If I remember correctly you need read/execute permissions to list the contents/enter a directory under linux.</div>
<div>
<div>
<div><font color="#000000"><font face="Calibri"><span style="font-size:14px"><br>
</span></font></font></div>
<div><font color="#000000"><font face="Calibri"><span style="font-size:14px">___________________________________________________________________</span></font></font></div>
<div><font color="#000000"><font face="Calibri"><span style="font-size:14px">Kris Lindgren</span></font></font></div>
<div><font color="#000000"><font face="Calibri"><span style="font-size:14px">Senior Linux Systems Engineer</span></font></font></div>
<div><font color="#000000"><font face="Calibri"><span style="font-size:14px">GoDaddy</span></font></font></div>
</div>
</div>
</div>
</div>
<div><br>
</div>
<span>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span style="font-weight:bold">From: </span>Liam Haworth <<a href="mailto:liam.haworth@bluereef.com.au" target="_blank">liam.haworth@bluereef.com.au</a>><br>
<span style="font-weight:bold">Date: </span>Tuesday, February 2, 2016 at 4:25 PM<br>
<span style="font-weight:bold">To: </span>Abel Lopez <<a href="mailto:alopgeek@gmail.com" target="_blank">alopgeek@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a>" <<a href="mailto:openstack-operators@lists.openstack.org" target="_blank">openstack-operators@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Openstack-operators] [glance] Image enters "killed" state on upload<br>
</div></span></div><div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif"><span>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div>
<div>Here is a output from my system instead of my blabbering in a long winded email<br>
<br>
root@ctrl1:~# uname -a<br>
Linux ctrl1 3.19.0-43-generic #49~14.04.1-Ubuntu SMP Thu Dec 31 15:44:49 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux<br>
<br>
root@ctrl1:~# df -h <br>
Filesystem Size Used Avail Use% Mounted on
<br>
udev 7.9G 4.0K 7.9G 1% /dev
<br>
tmpfs 1.6G 724K 1.6G 1% /run
<br>
/dev/mapper/ctrl1--vg-root 396G 6.4G 370G 2% /
<br>
none 4.0K 0 4.0K 0% /sys/fs/cgroup
<br>
none 5.0M 0 5.0M 0% /run/lock
<br>
none 7.9G 0 7.9G 0% /run/shm
<br>
none 100M 0 100M 0% /run/user
<br>
/dev/sdc1 236M 38M 186M 17% /boot
<br>
10.16.16.30:/srv/glance 739G 97G 604G 14% /var/lib/glance/images<br>
<br>
</div>
And to save from massed output from a LS, ever file in /var/lib/glance/images is: -rw-r----- 1 glance glance<br>
<br>
</div>
No apparmour installed or configured </div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, 3 Feb 2016 at 10:17 Abel Lopez <<a href="mailto:alopgeek@gmail.com" target="_blank">alopgeek@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Ok, with file store, some of the silly things that crop up are around directory permissions, disk space, SELinux/apparmour.
<div><br>
</div>
<div>Make sure the glance user and group have ownership (recursively) of the /var/lib/glance directory, make sure you're not low on space, if you have SELinux set to enforcing, test setting it to permissive (if that is the issue, resolve the contexts)</div>
</div>
<div style="word-wrap:break-word">
<div><br>
<div>
<blockquote type="cite">
<div>On Feb 2, 2016, at 3:13 PM, Liam Haworth <<a href="mailto:liam.haworth@bluereef.com.au" target="_blank">liam.haworth@bluereef.com.au</a>> wrote:</div>
<br>
<div>
<div dir="ltr">Glance is configured to use file store to /var/lib/glance/images<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, 3 Feb 2016 at 10:12 Abel Lopez <<a href="mailto:alopgeek@gmail.com" target="_blank">alopgeek@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">I ran into a similar issue in Havana, but that was because we were doing some 'behind-the-scenes' modification of the image (format conversion)
<div>Once we stopped that, the issue went away. </div>
<div><br>
</div>
<div>What is your glance store configured as?</div>
<div><br>
<div>
<blockquote type="cite"></blockquote>
</div>
</div>
</div>
<div style="word-wrap:break-word">
<div>
<div>
<blockquote type="cite">
<div>On Feb 2, 2016, at 3:05 PM, Liam Haworth <<a href="mailto:liam.haworth@bluereef.com.au" target="_blank">liam.haworth@bluereef.com.au</a>> wrote:</div>
<br>
</blockquote>
</div>
</div>
</div>
<div style="word-wrap:break-word">
<div>
<div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>Hey All,<br>
<br>
</div>
This sounds like an old bug after trying to google it but everything I found doesn't really seem to help. I'm trying to upload a 2.5GB QCOW2 image to glance to be used by users, the upload goes fine and in the glance registry logs I can see that it has successfully
saved the image but then it does this<br>
<br>
2016-02-03 09:51:49.607 2826 DEBUG glance.registry.api.v1.images [req-5ba18ea3-5777-4023-9f85-040aca48dfa7 --trunced-- - - -] Updating image 03a920ce-7979-4439-ab71-bc3dd34df3d3 with metadata: {u'status': u'killed'} update /usr/lib/python2.7/dist-packages/glance/registry/api/v1/images.py:470<br>
<br>
</div>
What reasons are their for it to do this to an image that just successfully uploaded?<br>
<br>
</div>
Thanks,<br>
<br>
</div>
Liam Haworth.<br>
</div>
<div dir="ltr">-- <br>
</div>
<div dir="ltr"><span>
<div><b><span lang="EN-US">Liam Haworth</span></b><span lang="EN-US"> | </span><span lang="EN-US"><font color="#3d85c6"><font face="Verdana,sans-serif"><span>Junior</span><span> Software </span><span>Engineer</span></font></font></span><span> </span><span>| </span><span lang="EN-US"><span lang="EN-AU"><a href="http://www.bluereef.com.au/" target="_blank">www.bluereef.com.au</a> </span></span></div>
<div><u><span>_________________________________________________________________</span></u><span></span></div>
<div><b><span lang="EN-US">T</span></b><span>: <a>+61 3 9898 8000</a> </span><span>| </span><b><span lang="EN-US">F</span></b><span lang="EN-US">:<b> </b></span><span><a>+61 3 9898 8055</a></span></div>
<div><span><a><br>
</a></span></div>
<div><br>
</div>
</span></div>
</div>
</blockquote>
</div>
</div>
</div>
<div style="word-wrap:break-word">
<div>
<div>
<blockquote type="cite">
<div>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<div dir="ltr">-- <br>
</div>
<div dir="ltr"><span>
<div><b><span lang="EN-US">Liam Haworth</span></b><span lang="EN-US"> | </span><span lang="EN-US"><font color="#3d85c6"><font face="Verdana,sans-serif"><span>Junior</span><span> Software </span><span>Engineer</span></font></font></span><span> </span><span>| </span><span lang="EN-US"><span lang="EN-AU"><a href="http://www.bluereef.com.au/" target="_blank">www.bluereef.com.au</a> </span></span></div>
<div><u><span>_________________________________________________________________</span></u><span></span></div>
<div><b><span lang="EN-US">T</span></b><span>: <a>+61 3 9898 8000</a> </span><span>| </span><b><span lang="EN-US">F</span></b><span lang="EN-US">:<b> </b></span><span><a>+61 3 9898 8055</a></span></div>
<div><span><a><br>
</a></span></div>
<div><br>
</div>
</span></div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<div dir="ltr">-- <br>
</div>
<div dir="ltr"><span>
<div><b><span lang="EN-US">Liam Haworth</span></b><span lang="EN-US"> | </span><span lang="EN-US"><font color="#3d85c6"><font face="Verdana,sans-serif"><span>Junior</span><span> Software </span><span>Engineer</span></font></font></span><span> </span><span>| </span><span lang="EN-US"><span lang="EN-AU"><a href="http://www.bluereef.com.au/" target="_blank">www.bluereef.com.au</a> </span></span></div>
<div><u><span>_________________________________________________________________</span></u><span></span></div>
<div><b><span lang="EN-US">T</span></b><span>: <a>+61 3 9898 8000</a> </span><span>| </span><b><span lang="EN-US">F</span></b><span lang="EN-US">:<b> </b></span><span><a>+61 3 9898 8055</a></span></div>
<div><span><a><br>
</a></span></div>
<div><br>
</div>
</span></div>
</div>
</div>
</span></div></blockquote></div><div dir="ltr">-- <br></div><div dir="ltr"><span><div><b><span lang="EN-US">Liam Haworth</span></b><span lang="EN-US"> | </span><span lang="EN-US"><font color="#3d85c6"><font face="Verdana, sans-serif"><span>Junior</span><span> Software </span><span>Engineer</span></font></font></span><span> </span><span>| </span><span lang="EN-US"><span lang="EN-AU"><a href="http://www.bluereef.com.au/">www.bluereef.com.au</a> </span></span></div><div><u><span>_________________________________________________________________</span></u><span></span></div><div><b><span lang="EN-US">T</span></b><span>: <a>+61 3 9898 8000</a> </span><span>| </span><b><span lang="EN-US">F</span></b><span lang="EN-US">:<b> </b></span><span><a>+61 3 9898 8055</a></span></div><div><span><a><br></a></span></div><div><br></div></span></div>