<html><body>
<p><font size="2" face="sans-serif">Padraig, Robert, Chenrui,</font><br>
<font size="2" face="sans-serif">It seems to be a problem with nova metadata service.</font><br>
<br>
<tt><font size="2">> Are you using libguestfs to do the injection? </font></tt><br>
<tt><font size="2">Yes - I wasn't originally, but have installed this on my controller and compute nodes.</font></tt><br>
<tt><font size="2"><br>
> What's the value of the following in nova.conf?<br>
> libvirt_inject_key<br>
> libvirt_inject_partition</font></tt><br>
<br>
<font size="2" face="sans-serif">I have the following set - I added inject_password as well to see if any additional information could be seen.</font><br>
<font size="2" face="sans-serif">/etc/nova/nova.conf:libvirt_inject_partition = -1</font><br>
<font size="2" face="sans-serif">/etc/nova/nova.conf:libvirt_inject_key = True</font><br>
<font size="2" face="sans-serif">/etc/nova/nova.conf:libvirt_inject_password = True</font><br>
<br>
<font size="2" face="sans-serif">When I boot a Ubuntu precise image I see the following in the console log:</font>
<ul style="padding-left: 18pt"><font size="2" face="sans-serif">[ 1.573692] EXT4-fs (vda1): re-mounted. Opts: (null)</font><br>
<font size="2" face="sans-serif">cloud-init start-local running: Tue, 29 Oct 2013 00:15:41 +0000. up 5.11 seconds</font><br>
<font size="2" face="sans-serif">no instance data found in start-local</font><br>
<font size="2" face="sans-serif">ci-info: lo : 1 127.0.0.1 255.0.0.0 .</font><br>
<font size="2" face="sans-serif">ci-info: eth0 : 1 10.10.100.2 255.255.255.0 fa:16:3e:a6:5a:98</font><br>
<font size="2" face="sans-serif">ci-info: route-0: 0.0.0.0 10.10.100.3 0.0.0.0 eth0 UG</font><br>
<font size="2" face="sans-serif">ci-info: route-1: 10.10.100.0 0.0.0.0 255.255.255.0 eth0 U</font><br>
<font size="2" face="sans-serif">cloud-init start running: Tue, 29 Oct 2013 00:15:41 +0000. up 5.89 seconds</font><br>
<font size="2" face="sans-serif">2013-10-29 00:16:32,646 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [50/120s]: url error [timed out]</font><br>
<font size="2" face="sans-serif">2013-10-29 00:17:23,698 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [101/120s]: url error [timed out]</font><br>
<font size="2" face="sans-serif">2013-10-29 00:17:41,717 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [119/120s]: url error [timed out]</font><br>
<font size="2" face="sans-serif">2013-10-29 00:17:42,718 - DataSourceEc2.py[CRITICAL]: giving up on md after 120 seconds</font><br>
<br>
<font size="2" face="sans-serif">no instance data found in start</font><br>
<font size="2" face="sans-serif">Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd</font></ul>
<br>
<font size="2" face="sans-serif">Here are the metadata values defined in nova.conf:</font><br>
<font size="2" face="sans-serif"># grep metadata_ /etc/nova/nova.conf</font><br>
<font size="2" face="sans-serif">metadata_host=192.167.11.5</font><br>
<font size="2" face="sans-serif">metadata_port=8775</font><br>
<font size="2" face="sans-serif">metadata_listen=0.0.0.0</font><br>
<font size="2" face="sans-serif">metadata_listen_port=8775</font><br>
<font size="2" face="sans-serif">metadata_manager=nova.api.manager.MetadataManager</font><br>
<br>
<font size="2" face="sans-serif">I am able to query the metadata server from compute / controller nodes:</font><br>
<font size="2" face="sans-serif"># curl <a href="http://192.167.11.5:8775">http://192.167.11.5:8775</a></font><br>
<font size="2" face="sans-serif">1.0</font><br>
<font size="2" face="sans-serif">...</font><br>
<font size="2" face="sans-serif">2008-09-01</font><br>
<font size="2" face="sans-serif">2009-04-04</font><br>
<br>
<font size="2" face="sans-serif">If I boot cirros image and try to query the metadata server using 169.264.169.254, it fails:</font><br>
<font size="2" face="sans-serif">$ curl <a href="http://169.254.169.254/">http://169.254.169.254/</a></font><br>
<font size="2" face="sans-serif">curl: (7) couldn't connect to host</font><br>
<br>
<font size="2" face="sans-serif">I suspect that IP tables rules that are created to redirect from 169.264.169.254 on the guest to $metadata_host:$metadata_port are not being created correctly - suggestions on how to debug this or why this might be the case? </font><br>
<br>
<font size="2" face="sans-serif">Is there documentation that helps with the setup and verification of nova api metadata service?</font><br>
<br>
<font size="2" face="sans-serif">Thanks,<br>
Bill Owen <br>
billowen@us.ibm.com<br>
Strategic Test Methods and Tools <br>
520-799-4829, T/L 321-4829<br>
</font><br>
<br>
<img width="16" height="16" src="cid:1__=08BBF680DF9004F38f9e8a93df938@us.ibm.com" border="0" alt="Inactive hide details for Pádraig Brady ---10/26/2013 02:41:27 PM---On 10/26/2013 01:43 AM, Bill Owen wrote: > I've just update"><font size="2" color="#424282" face="sans-serif">Pádraig Brady ---10/26/2013 02:41:27 PM---On 10/26/2013 01:43 AM, Bill Owen wrote: > I've just updated my test environment to stable-havana.</font><br>
<br>
<font size="1" color="#5F5F5F" face="sans-serif">From: </font><font size="1" face="sans-serif">Pádraig Brady <P@draigbrady.com></font><br>
<font size="1" color="#5F5F5F" face="sans-serif">To: </font><font size="1" face="sans-serif">Bill Owen/Tucson/IBM@IBMUS</font><br>
<font size="1" color="#5F5F5F" face="sans-serif">Cc: </font><font size="1" face="sans-serif">openstack@lists.openstack.org</font><br>
<font size="1" color="#5F5F5F" face="sans-serif">Date: </font><font size="1" face="sans-serif">10/26/2013 02:41 PM</font><br>
<font size="1" color="#5F5F5F" face="sans-serif">Subject: </font><font size="1" face="sans-serif">Re: [Openstack] Key Injection not working after upgrading from Grizzly to Havana</font><br>
<hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br>
<br>
<br>
<tt><font size="2">On 10/26/2013 01:43 AM, Bill Owen wrote:<br>
> I've just updated my test environment to stable-havana.<br>
> <br>
> I have booted vm instances with Fedora and Ubuntu images with a key_name specified:<br>
> $ nova boot --key_name key <vm-name> --image <image-id> --flavor 2 test_vm<br>
> <br>
> After the image becomes active, I try to ssh to the image, but get an error message: <br>
> $ ssh -i key.pem fedora@<vm-ip-addr><br>
> Permission denied (publickey,gssapi-keyex,gssapi-with-mic).<br>
> <br>
> I tried using keys/images that worked in grizzly, as well as newly created keys and new images following the instructions in the install docs:<br>
> </font></tt><tt><font size="2"><a href="http://docs.openstack.org/trunk/install-guide/install/apt/content/nova-boot.html">http://docs.openstack.org/trunk/install-guide/install/apt/content/nova-boot.html</a></font></tt><tt><font size="2"><br>
> <br>
> I don't see anything about changes in this area in release notes. Any suggestions on what I might be missing or how to debug would be appreciated!<br>
> In particular, is there a way to increase debug logging so I can see when it tries to do the key injection on the new vm?<br>
> <br>
> FWIW, cirros image boots and I can ssh/login using cirros user and password.<br>
<br>
Injection is not under active development in Havana,<br>
and so theoretically nothing should have changed here.<br>
<br>
Are you using libguestfs to do the injection?<br>
What's the value of the following in nova.conf?<br>
<br>
libvirt_inject_key<br>
libvirt_inject_partition<br>
<br>
Note failure to inject a key does not cause a guest to error,<br>
only failure to inject a user specified file does at present.<br>
However at debug level, messages are printed as to why there<br>
were errors with injecting the other components. So please<br>
set debug=True in nova.conf, restart the nova-compute service,<br>
and try again, keeping an eye on /var/log/nova/compute.log<br>
<br>
thanks,<br>
Pádraig.<br>
<br>
</font></tt><br>
</body></html>