Open Stack

"Daniel P. Berrange" <berrange at redhat.com> wrote on 08/12/2014 07:57:14 
PM:

> From: "Daniel P. Berrange" <berrange at redhat.com>
> To: Markus Zoeller/Germany/IBM at IBMDE
> Cc: openstack at lists.openstack.org
> Date: 08/12/2014 07:57 PM
> Subject: Re: [Openstack] [nova] Libvirt driver domain metadata - add
> instance metadata dictionary?
> 
> On Tue, Aug 12, 2014 at 04:50:12PM +0200, Markus Zoeller wrote:
> > "Daniel P. Berrange" <berrange at redhat.com> wrote on 08/11/2014 
11:39:25 
> > AM:
> > 
> > > From: "Daniel P. Berrange" <berrange at redhat.com>
> > > To: Matt Riedemann <mriedem at linux.vnet.ibm.com>
> > > Cc: 
> > > Date: 08/11/2014 11:49 AM
> > > Subject: Re: [Openstack] [nova] Libvirt driver domain metadata - add
> > > instance metadata dictionary?
> > > 
> > > On Fri, Aug 01, 2014 at 03:47:48PM -0500, Matt Riedemann wrote:
> > > > 
> > > > 
> > > > On 7/31/2014 6:58 AM, Markus Zoeller wrote:
> > > > >The blueprint "libvirt-driver-domain-metadata" introduces some of 
the
> > > > >istances properties to the `libvirt.xml` file. For example the 
name
> > > > >of the instance, the name of the flavor and the creation date.
> > > > >
> > > > >Would it make sense to add the instance.metadata dictionary also?
> > > > >
> > > > >API: /v2/​{tenant_id}​/servers/​{server_id}​/metadata
> > > > >Code: https://github.com/openstack/nova/blob/master/
> > > > >       nova/objects/instance.py#L148
> > > > >
> > > > 
> > > > You could ask danpb in #openstack-nova IRC about his thoughts, but 
looking
> > > > at the spec and code it looks like a specific metadata schema was 
in mind.
> > > > The metadata that a user can pass in when spawning an instance is 
arbitrary
> > > > so it wouldn't really fit into the schema created unless that was 
modified
> > > > to add some custom values, which would be the user metadata.
> > > > 
> > > > Is there a use case for putting user metadata in there?  Looks 
like the
> > > > blueprint is for adding specific metadata so an admin can 
correlate his
> > > > libvirt domains against nova API calls.
> > > 
> > > The intent was primarily to aid in debugging libvirt by providing 
information
> > > that is/was relevant to the libvirt guest configuration.
> > > 
> > > The instance metadata dict is not something that affects libvirt - 
IIRC it
> > > is only relevant to the guest OS, so i don't think it is relevant to 
include
> > > in the libvirt XML
> > 
> > Maybe the direction I'm heading is wrong. My intention is to enable a 
correlation
> > between multiple libvirt domains independent from their flavor.
> > E.g.
> > --------------     --------------    --------------
> > | Server: A  |     | Server: B  |    | Server: C  |
> > | Flavor: X  |     | Flavor: X  |    | Flavor: Y  |
> > | Group: foo |     | Group: bar |    | Group: foo |
> > --------------     --------------    --------------
> > 
> > I'd like to enable the hypervisor to understand that server A and C 
are correlated
> > because of the same "Group: foo". Is there already another mechanism 
which enables that?
> 
> What is the purpose of the grouping ?  The domain metadata in libvirt is
> explicitly declared to be completely opaque to the hypervisor and/or
> libvirt itself. It holds metadata that is exclusively for use / 
definition
> by the managment application (ie nova itself) and libvirt won't try to
> interpret it at all. 
> 
> If you're looking at grouping for the POV of doing resource management,
> the libvirt has an explicit concept of resource partitioning which 
allows
> you to group VMs together and apply resource constraints to the VMs as
> a whole. I don't know if that's what you're after.
> 
> http://libvirt.org/cgroups.html#customPartiton
> 
> Regards,
> Daniel

The purpose of the grouping is indeed because of doing resource 
management.
There is the idea of a dedicated resource optimization component within 
the 
host and *above* the libvirt layer (I was imprecise with my previous 
statement 
of "enable the hypervisor"). The structure would be like this:
+--------------------------------------+ 
|                 Host                 | 
|                                      | 
|+------------------------------------+| 
|| Resource Optimization Component    || 
|+------------------------------------+|                    
|+------------------------------------+| 
|| libvirt                            || 
|+------------------------------------+| 
|+----------+ +----------+ +----------+| 
||Server: A | |Server: B | |Server: C || 
||Flavor: X | |Flavor: X | |Flavor: Y || 
||Group: foo| |Group: bar| |Group: foo|| 
|+----------+ +----------+ +----------+| 
+--------------------------------------+ 

Pushing the instance_metadata down to the libvirt.xml into the metadata 
tag 
would still be agnostic to libvirt but enables the "resource optimization 
component" to make a correlation. That's the full background, sorry for 
being
imprecisely in the previous descriptions.

The XML I imagined would look like this (see nova:meta tag):
<domain type='kvm'>
    ...rest of domain XML config...
    <metadata>
      <nova:instance xmlns:nova="http://openstack.org/nova/instance/1">
        <nova:package version="2014.2.3"/>
        <nova:flavor name="X">
          <nova:memory>512</nova:memory>
          <nova:disk>10</nova:disk>
          ....
        </nova:flavor>
        <nova:name>A</nova:name>
        <nova:creationTime>2014-12-25 12:03:20</nova:creationTime>
        <nova:owner>
          <nova:user uuid="85bd45c0...213684">joe</nova:user>
          <nova:project uuid="d33b8c0e...342d69">acmecorp</nova:project>
        </nova:owner>
        <nova:root type="image|volume" uuid="69f2991b...f29a8bc"/>
        <nova:meta>
          "Group:foo", 
        </nova:meta>
      </nova:instance>
    </metadata>
  </domain>

That would be a generic way to store the instance_metadata into the 
libvirt.xml file for other applications.

The libvirt cgroup custom partition concept looks interesting, I have
to read more about this. Does OpenStack already use that in some way?