Open Stack

I've fixed this - the VMs should now be start-able.  They have the free XenServer license applied to them, 

Could you add some earlier warning so that we get a clear message if the VMs fail to start vs fail after they have started?

Having the primary symptom being the xenstore issue is slightly weird :)  I imagine the smokestack tests should fail rather than timeout in this case?

Bob

-----Original Message-----
From: Dan Prince [mailto:dprince at redhat.com] 
Sent: 12 June 2013 15:54
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] XenServer 6 tests failing this morning



----- Original Message -----
> From: "Bob Ball" <bob.ball at citrix.com>
> To: "OpenStack Development Mailing List" 
> <openstack-dev at lists.openstack.org>
> Sent: Wednesday, June 12, 2013 9:33:06 AM
> Subject: Re: [openstack-dev] XenServer 6 tests failing this morning
> 
> Sorry, I should have replied to the list as well.
> 
> We debugged this one off-list as the debugging included IP addresses 
> and some private information.
> 
> The root cause is actually a VM dying and smokestack continually 
> asking for the SSH key, so we had a constant stream of xenstore-read 
> requests for /local/domain/-1/data/ssh_key.

I can add a fix to check for a valid DOM id before asking for the ssh_key.... but the fact that the VMs die is the root cause of the failure so we need to fix that first.

I took a closer look this morning and it looks like the machines simply need licenses. Bob: Can you guys have a look at getting licenses for these machines?

# xe vm-start uuid=d497ad9a-cdb2-333d-fd43-b0d1ab8a0a80
Your license has expired.  Please contact your support representative.

Dan

> 
> This is something that Dan is looking at fixing in smokestack, and 
> hopefully we will get more information on what happened with the test 
> domains separately.






> 
> Bob
> 
> -----Original Message-----
> From: John Garbutt [mailto:john at johngarbutt.com]
> Sent: 12 June 2013 14:20
> To: OpenStack Development Mailing List
> Subject: Re: [openstack-dev] XenServer 6 tests failing this morning
> 
> So apparently the errors are harmless:
> http://forums.citrix.com/message.jspa?messageID=1670827
> 
> It is probably when nova is polling for the agent to respond.
> I assume the image you are using does have the agent present?
> 
> But 1.2G worth seems... excessive.
> Its worth checking that log rotate is working correctly?
> I don't remember what all the defaults are.
> 
> Also, is the timeout due to everything slowing up and you running out 
> of Dom0 disk space? Or was there something else going on? I guess 
> clearing out all the logs should give us a bit of time to find the real problem.
> 
> John
> 
> On 11 June 2013 12:57, Dan Prince <dprince at redhat.com> wrote:
> > Hi all,
> >
> > As of this morning it looks like the / partition on the SmokeStack 
> > xenserver machines is full. The issue seems to be that the 
> > xenstored-access.log has grown to 1.2G in less than 12 hours. Tons 
> > of
> > this:
> >
> > [root at 10-13-39-10 log]# tail xenstored-access.log Jun 11 11:49:12 
> > 10-13-39-10 /opt/xensource/bin/xenstored:  A14411977 newconn Jun 11 
> > 11:49:12 10-13-39-10 /opt/xensource/bin/xenstored:  A14411977
> > error     ENOENT
> > Jun 11 11:49:12 10-13-39-10 /opt/xensource/bin/xenstored:  A14411977 
> > endconn
> >
> > Any idea if this an upstream regression or perhaps a configuration issue?
> >
> > The SmokeStack tests are still running... but they all timeout after 
> > an hour.
> >
> > Dan
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev