Open Stack

> "Daniel P. Berrange" <berrange at redhat.com> wrote on 04/15/2015 
11:35:39 
> AM:
>
> > From: "Daniel P. Berrange" <berrange at redhat.com>
> > To: "OpenStack Development Mailing List (not for usage questions)" 
> > <openstack-dev at lists.openstack.org>
> > Date: 04/15/2015 11:42 AM
> > Subject: Re: [openstack-dev] [nova] why is evacuate marked as missing 
> > for libvirt?
> > 
> > On Tue, Apr 14, 2015 at 01:44:45PM -0400, Russell Bryant wrote:
> > > On 04/14/2015 12:22 PM, Matt Riedemann wrote:
> > > > This came up in IRC this morning, but the hypervisor support 
matrix 
> is
> > > > listing evacuate as 'missing' for the libvirt driver:
> > > > 
> > > > http://docs.openstack.org/developer/nova/support-
> > matrix.html#operation_evacuate
> > > > 
> > > > 
> > > > Does anyone know why that is?  The rebuild method in the compute 
> manager
> > > > just re-uses other virt driver operations so by default it's 
> implemented
> > > > by all drivers.  The only one that overrides rebuild for evacuate 
is 
> the
> > > > ironic driver.
> > > 
> > > I think it's a case where there are a couple of different things
> > > referred to with 'evacuate'.  I believe this was originally added to
> > > track something that was effectively XenServer specific and the
> > > description of the feature seems to reflect that.  We've since added 

> the
> > > more generic 'evacuate' API, so it's pretty confusing.  It should
> > > probably be reworked to track which drivers work with the 'evacuate' 

> API
> > > call, and perhaps have a different entry for whatever this different
> > > XenServer thing is (or was).
> > 
> > Yep, if there's any mistakes or bizarre things in the support matrix
> > just remember that the original wiki page had essentially zero 
> information
> > about what each feature item was referring to - just the two/three 
word
> > feature name. When I turned it into formal docs I tried to add better
> > explanations, but it is entirely possible my interpretations were 
wrong
> > in places. So if in doubt assume the support matrix is wrong, and just
> > send a review to update it to some saner state with better description
> > of the feature. Pretty much all the features in the matrix could do
> > with better explanations and/or being broken up into finer grained
> > features - there's plenty of scope for people to submit patches to
> > improve the granularity of items.
>
> I think that the confusion is caused by something called the "host
> maintenance mode" [1]. When this is enabled, an evacuate is triggered
> by the underlying hypervisor. This can mode can be set by the CLI [2] 
> and is not implemented by the libvirt driver.
> The probably intented API for the feature "evacuate" is [3] which can 
> be triggered via CLI with:
> * nova evacuate <server>
> * nova host-evacuate <host>
> * nova host-evacuate-live <host>
>
> The feature "evacuate" has hereby a dependency to "live-migration". As
> the system z platform doesn't yet has [4] merged, "evacuate" is there 
> "partial" [5] (TODO for me) whereas for x86 there should be "complete".
> Please correct me if I'm wrong here.
>
> Unfortunately I couldn't find any tempest tests for the "evacuate" 
> feature, so I tested in manually.
>
> [1] virt.driver.ComputeDriver.host_maintenance_mode(self, host, mode)
> > 
> 
https://github.com/openstack/nova/blob/2015.1.0rc1/nova/virt/driver.py#L1016
> [2] Nova CLI; command "nova host-update"
> > 
> http://docs.openstack.org/cli-reference/content/novaclient_commands.html
> [3] nova.api.openstack.compute.contrib.evacuate
> > 
> 
https://github.com/openstack/nova/blob/2015.1.0rc1/nova/api/openstack/compute/contrib/evacuate.py
> [4] libvirt: handle NotSupportedError in compareCPU
> > https://review.openstack.org/#/c/166130/
> [5] "Update hypervisor support matrix with column for kvm on system z"
> > https://review.openstack.org/#/c/172391/
> Regards,
> Markus Zoeller (markus_z)


Today the topic about "evacuate" and "host maintenance mode" came up
again in the IRC nova channel. I'd like to try again to state how I
understand it. At the end will be suggestions how this could be solved.


CLI -> API -> Driver
====================
The mapping between the python-novaclient, the nova REST API and
the nova.virt.driver looks like this:

CLI                        API                 Driver
-----------------------    ----------------    -------------------------
nova host-update           hosts.py            host_maintenance_mode()
                           update()            set_host_enabled()
-----------------------    ----------------    -------------------------
nova evacuate              evacuate.py         rebuild()
                           _evacuate() 
-----------------------    ----------------    -------------------------   
 
nova host-evacuate         evacuate.py         rebuild()
[calls API in a loop]      _evacuate()
-----------------------    ----------------    -------------------------
nova host-evacuate-live    admin_actions.py    live_migration()
[calls API in a loop]      _migrate_live()
-----------------------    ----------------    -------------------------

The CLI command "nova host-update" [1] calls the "update" REST API[2][3].
The REST API calls the method "host_maintenance_mode" [4] which 
is implemented by XEN in a way that all instances get migrated to
another host. It also sets the host in state where it won't be eligible
for other instances which will be launched after that [5].

For the sake of completeness, there is also:
    CLI:      nova host-servers-migrate 
    REST API: admin_actions.py  -> _migrate()
    Manager:  resize_instance()


Conclusion
==========
As an interim solution until the new "feature classification" system
is established I suggest the following actions:

1) The title "Evacuate instances from host" is misleading. 
    1) change it to "Set host in maintenance mode". 
    2) Clarify that this is the "old/deprecated" way/API.
    3) Only XEN is implementing that (s390 and hyper-v were probably 
       mislead)
2) Add another feature row which is called "Evacuate instances from host"
   1) clarify that this uses the specific "evacuate" API and should be
      prefered over "Set host in maintenance mode". 
   2) The platforms should update their specific entries.

Does this look like a reasonable approach?


References
==========
[1] nova; CLI; "nova host-update";
    
http://docs.openstack.org/cli-reference/content/novaclient_commands.html#novaclient_subcommand_host-update
 
[2] nova-client; "hosts.py", method "update", branch "stable/kilo";
    
https://github.com/openstack/python-novaclient/blob/stable/kilo/novaclient/v2/hosts.py#L31
 
[3] nova; API; "hosts.py"; method "update"; release 2015.1.0; 
    
https://github.com/openstack/nova/blob/2015.1.0/nova/api/openstack/compute/contrib/hosts.py#L97
 
[4] nova; "driver.py"; method "host_maintenance_mode"; release 2015.1.0; 
    
https://github.com/openstack/nova/blob/2015.1.0/nova/virt/driver.py#L1016
 
[5] nova; "driver.py"; method "set_host_enabled"; release 2015.1.0; 
    
https://github.com/openstack/nova/blob/2015.1.0/nova/virt/driver.py#L1022