Open Stack

All,

Briefly, I wanted to announce the availability of a piece of code that may
have some use for the members of this list.
https://github.com/BMDan/OpenStack-Hypervisor-Balance/ is designed to
capitalize on the live migration feature to allow both draining of
hypervisors (e.g. prior to a kernel upgrade, or a hardware change
necessitating downtime) and ongoing dynamic balancing amongst hypervisors
in a manner similar to the OpenStack Neat project, albeit much
lighter-weight.  This has been in production use in our Ceph-backed clouds
for several months now, with great success.

In addition to the public announcement of sysdrain.py, I wanted to inquire
on two topics relevant to live migration.  First, related to the work
above, I was wondering if anyone had come up with a good way to fence
redundant pairs to avoid them landing on the same HV.  I could use zones
for this, I recognize, but I then have to place non-redundant machines in
an arbitrarily-chosen zone.  My next planned feature for the code above is
to allow plug-ins that control placement, with the pursuit of this goal the
intention.

Second, I've seen some issues with live-migration recently on a Havana
cloud initially built on Ubuntu 12.04, but upgraded to the 3.11 kernel from
Ubuntu 13.10.  Specifically, I'm wondering if I'm running into an issue
related to the HPET changes that recently landed.  What I'm seeing is,
after live-migrating, instances get very, very confused, showing negative
uptime and dates thousands of years into the future.  The clocks on the
compute nodes are synced by NTP and are (visually) within less than a
second of one another.  Any clues on next steps to debug this one?  Nothing
appears in any of the logs on the hypervisors, though the virt is very
chatty in dmesg and elsewhere (as one might expect when you wake up to
discover that you  have a few millennia of cron jobs that you've missed!).
All packages are the very latest from ubuntu-cloud at Canonical.

Regards,
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20140528/92898076/attachment.html>