[openstack-dev] [ironic] [nova] traits discussion call
Dmitry Tantsur
dtantsur at redhat.com
Mon Oct 30 14:19:18 UTC 2017
It's a holiday here tomorrow, but I don't have any specific plans, so I think
I'll be able to make it.
On 10/30/2017 03:13 PM, Jay Pipes wrote:
> I'd prefer to have you on the call, Dima. How about we push it back to tomorrow
> at the same time?
>
> Can everyone make it then?
>
> -jay
>
> On 10/30/2017 10:11 AM, Dmitry Tantsur wrote:
>> Aaaand sorry again, but due to sudden errands I won't be able to attend.
>> Please feel free to use my bluejeans room anyway. I think my position on
>> traits is more or less clear from previous discussions with John, Sam and Eric.
>>
>> 2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>>:
>>
>> Sigh, sorry. I forgot that we're moving back to winter time this
>> weekend. I *think* the time is 3pm UTC then. It seems to be 11am
>> eastern US:
>>
>> https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et
>>
>>
>> <https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et>.
>>
>>
>>
>> On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:
>>
>> And the winner is Mon, 30 Oct, 2pm UTC!
>>
>> The bluejeans ID is https://bluejeans.com/757528759
>> <https://bluejeans.com/757528759>
>> (works without plugins in recent FF and Chrome; if it asks to
>> install an app, ignore it and look for a link saying "join with
>> browser")
>>
>> On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:
>>
>> Hi all!
>>
>> I'd like to invite you to the discussion of the way to
>> implement traits in
>> ironic and the ironic virt driver. Please vote for the time at
>> https://doodle.com/poll/ts43k98kkvniv8uz
>> <https://doodle.com/poll/ts43k98kkvniv8uz>. Please vote by
>> EOD tomorrow.
>>
>> Note that it's going to be a technical discussion - please
>> make sure you
>> understand what traits are and why ironic cares about them.
>> See below for more
>> context.
>>
>> We'll probably use my bluejeans account, as it works without
>> plugins in modern
>> browsers. I'll post a meeting ID when we pick the date.
>>
>>
>> On 10/23/2017 04:09 PM, Eric Fried wrote:
>>
>> We discussed this a little bit further in IRC [1].
>> We're all in
>> agreement, but it's worth being precise on a couple of
>> points:
>>
>> * We're distinguishing between a "feature" and the
>> "trait" that
>> represents it in placement. For the sake of this
>> discussion, a
>> "feature" can (maybe) be switched on or off, but a
>> "trait" can either be
>> present or absent on a RP.
>> * It matters *who* can turn a feature on/off.
>> * If it can be done by virt at spawn time, then it
>> makes sense to have
>> the trait on the RP, and you can switch the feature
>> on/off via a
>> separate extra_spec.
>> * But if it's e.g. an admin action, and spawn has
>> no control, then the
>> trait needs to be *added* whenever the feature is *on*,
>> and *removed*
>> whenever the feature is *off*.
>>
>> [1]
>>
>> http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13
>>
>>
>> <http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13>
>>
>>
>>
>> On 10/23/2017 08:15 AM, Sylvain Bauza wrote:
>>
>>
>>
>> On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried
>> <openstack at fried.cc
>> <mailto:openstack at fried.cc
>> <mailto:openstack at fried.cc>>> wrote:
>>
>> I agree with Sean. In general terms:
>>
>> * A resource provider should be marked with a
>> trait if that feature
>> * Can be turned on or off (whether it's
>> currently on or not); or
>> * Is always on and can't ever be turned off.
>>
>>
>> No, traits are not boolean. If a resource provider
>> stops providing a
>> capability, then the existing related trait should
>> just be removed,
>> that's it.
>> If you see a trait, that's just means that the
>> related capability for
>> the Resource Provider is supported, that's it too.
>>
>> MHO.
>>
>> -Sylvain
>>
>>
>>
>> * A consumer wanting that feature present
>> (doesn't matter whether it's
>> on or off) should specify it as a required
>> *trait*.
>> * A consumer wanting that feature present and
>> turned on should
>> * Specify it as a required trait; AND
>> * Indicate that it be turned on via some
>> other mechanism (e.g. a
>> separate extra_spec).
>>
>> I believe this satisfies Dmitry's (Ironic's)
>> needs, but also Jay's drive
>> for placement purity.
>>
>> Please invite me to the hangout or whatever.
>>
>> Thanks,
>> Eric
>>
>> On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
>> >
>> >
>> >
>> >
>> > *From:*Jay Pipes [mailto:jaypipes at gmail.com
>> <mailto:jaypipes at gmail.com>
>> <mailto:jaypipes at gmail.com
>> <mailto:jaypipes at gmail.com>>]
>> > *Sent:* Monday, October 23, 2017 12:20 PM
>> > *To:* OpenStack Development Mailing List
>> <openstack-dev at lists.openstack.org
>> <mailto:openstack-dev at lists.openstack.org>
>> <mailto:openstack-dev at lists.openstack.org
>> <mailto:openstack-dev at lists.openstack.org>>>
>> > *Subject:* Re: [openstack-dev] [ironic]
>> ironic and traits
>> >
>> >
>> >
>> > Writing from my phone... May I ask that
>> before you proceed with any plan
>> > that uses traits for state information that
>> we have a hangout or
>> > videoconference to discuss this?
>> Unfortunately today and tomorrow I'm
>> > not able to do a hangout but I can do one
>> on Wednesday any time of the day.
>> >
>> >
>> >
>> > */[Mooney, Sean K] on the uefi boot topic I
>> did bring up at the
>> ptg that
>> > we wanted to standardizes tratis for
>> “verified boot” /*
>> >
>> > */that included a trait for uefi secure
>> boot enabled and to
>> indicated a
>> > hardware root of trust, e.g. intel boot
>> guard or similar/*
>> >
>> > */we distinctly wanted to be able to tag
>> nova compute hosts with those
>> > new traits so we could require that vms
>> that request/*
>> >
>> > */a host with uefi secure boot enabled and
>> a hardware root of
>> trust are
>> > scheduled only to those nodes. /*
>> >
>> > */ /*
>> >
>> > */There are many other examples that effect
>> both vms and bare
>> metal such
>> > as, ecc/interleaved memory, cluster on die, /*
>> >
>> > */l3 cache code and data prioritization,
>> vt-d/vt-c, HPET, Hyper
>> > threading, power states … all of these
>> feature may be present on the
>> > platform/*
>> >
>> > */but I also need to know if they are
>> turned on. Ruling out state in
>> > traits means all of this logic will
>> eventually get pushed to scheduler
>> > filters/*
>> >
>> > */which will be suboptimal long term as
>> more state is tracked.
>> Software
>> > defined infrastructure may be the future
>> but hardware defined
>> software/*
>> >
>> > */is sadly the present…/*
>> >
>> > */ /*
>> >
>> > */I do however think there should be a
>> sperateion between asking for a
>> > host that provides x with a trait and
>> asking for x to be
>> configure via/*
>> >
>> > */A trait. The trait secure_boot_enabled
>> should never result in the
>> > feature being enabled It should just find a
>> host with it on. If
>> you want/*
>> >
>> > */To request it to be turned on you would
>> request a host with
>> > secure_boot_capable as a trait and have a
>> flavor extra spec or image
>> > property to request/*
>> >
>> > */Ironic to enabled it. these are two very
>> different request and
>> should
>> > not be treated the same. /*
>> >
>> >
>> >
>> >
>> >
>> > Lemme know!
>> >
>> > -jay
>> >
>> >
>> >
>> > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur"
>> <dtantsur at redhat.com <mailto:dtantsur at redhat.com>
>> <mailto:dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>>
>> > <mailto:dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>
>> <mailto:dtantsur at redhat.com
>> <mailto:dtantsur at redhat.com>>>> wrote:
>> >
>> > Hi Jay!
>> >
>> > I appreciate your comments, but I think
>> you're approaching the
>> > problem from purely VM point of view.
>> Things simply don't work the
>> > same way in bare metal, at least not if
>> we want to provide the same
>> > user experience.
>> >
>> >
>> >
>> > On Sun, Oct 22, 2017 at 2:25 PM, Jay
>> Pipes <jaypipes at gmail.com
>> <mailto:jaypipes at gmail.com>
>> <mailto:jaypipes at gmail.com <mailto:jaypipes at gmail.com>>
>> > <mailto:jaypipes at gmail.com
>> <mailto:jaypipes at gmail.com>
>> <mailto:jaypipes at gmail.com
>> <mailto:jaypipes at gmail.com>>>> wrote:
>> >
>> > Sorry for delay, took a week off
>> before starting a new job.
>> > Comments inline.
>> >
>> > On 10/16/2017 12:24 PM, Dmitry
>> Tantsur wrote:
>> >
>> > Hi all,
>> >
>> > I promised John to dump my
>> thoughts on traits to the
>> ML, so
>> > here we go :)
>> >
>> > I see two roles of traits (or
>> kinds of traits) for
>> bare metal:
>> > 1. traits that say what the
>> node can do already (e.g. "the
>> > node is
>> > doing UEFI boot")
>> > 2. traits that say what the
>> node can be *configured* to do
>> > (e.g. "the node can
>> > boot in UEFI mode")
>> >
>> >
>> > There's only one role for traits.
>> #2 above. #1 is state
>> > information. Traits are not for
>> state information. Traits are
>> > only for communicating capabilities
>> of a resource provider
>> > (baremetal node).
>> >
>> >
>> >
>> > These are not different, that's what
>> I'm talking about here. No
>> > users care about the difference between
>> "this node was put in UEFI
>> > mode by an operator in advance", "this
>> node was put in UEFI
>> mode by
>> > an ironic driver on demand" and "this
>> node is always in UEFI mode,
>> > because it's AARCH64 and it does not
>> have BIOS". These situation
>> > produce the same result (the node is
>> booted in UEFI mode), and
>> thus
>> > it's up to ironic to hide this difference.
>> >
>> >
>> >
>> > My suggestion with traits is one way to
>> do it, I'm not sure
>> what you
>> > suggest though.
>> >
>> >
>> >
>> >
>> > For example, let's say we add the
>> following to the os-traits
>> > library [1]
>> >
>> > * STORAGE_RAID_0
>> > * STORAGE_RAID_1
>> > * STORAGE_RAID_5
>> > * STORAGE_RAID_6
>> > * STORAGE_RAID_10
>> >
>> > The Ironic administrator would add
>> all RAID-related traits to
>> > the baremetal nodes that had the
>> *capability* of
>> supporting that
>> > particular RAID setup [2]
>> >
>> > When provisioned, the baremetal
>> node would either have RAID
>> > configured in a certain level or
>> not configured at all.
>> >
>> >
>> > A very important note: the
>> Placement API and Nova
>> scheduler (or
>> > future Ironic scheduler) doesn't
>> care about this. At all.
>> I know
>> > it sounds like I'm being callous,
>> but I'm not. Placement and
>> > scheduling doesn't care about the
>> state of things. It only
>> cares
>> > about the capabilities of target
>> destinations. That's it.
>> >
>> >
>> >
>> > Yes, because VMs always start with a
>> clean state, and
>> hypervisor is
>> > there to ensure that. We don't have
>> this luxury in ironic :) E.g.
>> > our SNMP driver is not even aware of
>> boot modes (or RAID, or BIOS
>> > configuration), which does not mean
>> that a node using it cannot be
>> > in UEFI mode (have a RAID or BIOS
>> pre-configured, etc, etc).
>> >
>> >
>> >
>> >
>> >
>> > This seems confusing, but it's
>> actually very useful.
>> Say, I
>> > have a flavor that
>> > requests UEFI boot via a trait.
>> It will match both the
>> nodes
>> > that are already in
>> > UEFI mode, as well as nodes
>> that can be put in UEFI mode.
>> >
>> >
>> > No :) It will only match nodes that
>> have the UEFI capability.
>> > The set of providers that have the
>> ability to be booted
>> via UEFI
>> > is *always* a superset of the set
>> of providers that *have been
>> > booted via UEFI*. Placement and
>> scheduling decisions only care
>> > about that superset -- the
>> providers with a particular
>> capability.
>> >
>> >
>> >
>> > Well, no, it will. Again, you're purely
>> basing on the VM idea,
>> where
>> > a VM is always *put* in UEFI mode, no
>> matter how the hypervisor
>> > looks like. It is simply not the case
>> for us. You have to care
>> what
>> > state the node is, because many drivers
>> cannot change this state.
>> >
>> >
>> >
>> >
>> >
>> > This idea goes further with
>> deploy templates (new concept
>> > we've been thinking
>> > about). A flavor can request
>> something like CUSTOM_RAID_5,
>> > and it will match the
>> > nodes that already have RAID 5,
>> or, more
>> interestingly, the
>> > nodes on which we
>> > can build RAID 5 before
>> deployment. The UEFI example above
>> > can be treated in a
>> > similar way.
>> >
>> > This ends up with two sources
>> of knowledge about traits in
>> > ironic:
>> > 1. Operators setting something
>> they know about hardware
>> > ("this node is in UEFI
>> > mode"),
>> > 2. Ironic drivers reporting
>> something they
>> > 2.1. know about hardware
>> ("this node is in UEFI mode" -
>> > again)
>> > 2.2. can do about hardware
>> ("I can put this node in
>> UEFI
>> > mode")
>> >
>> >
>> > You're correct that both pieces of
>> information are important.
>> > However, only the "can do about
>> hardware" part is relevant to
>> > Placement and Nova.
>> >
>> > For case #1 we are planning on
>> a new CRUD API to set/unset
>> > traits for a node.
>> >
>> >
>> > I would *strongly* advise against
>> this. Traits are not for
>> state
>> > information.
>> >
>> > Instead, consider having a DB (or
>> JSON) schema that lists
>> state
>> > information in fields that are
>> explicitly for that state
>> > information.
>> >
>> > For example, a schema that looks
>> like this:
>> >
>> > {
>> > "boot": {
>> > "mode": <one of 'bios' or 'uefi'>,
>> > "params": <dict>
>> > },
>> > "disk": {
>> > "raid": {
>> > "level": <int>,
>> > "controller": <one of 'sw' or
>> 'hw'>,
>> > "driver": <string>,
>> > "params": <dict>
>> > }, ...
>> > },
>> > "network": {
>> > ...
>> > }
>> > }
>> >
>> > etc, etc.
>> >
>> > Don't use trait strings to
>> represent state information.
>> >
>> >
>> >
>> > I don't see an alternative proposal
>> that will satisfy what we have
>> > to solve.
>> >
>> >
>> >
>> >
>> > Best,
>> > -jay
>> >
>> > Case #2 is more interesting. We
>> have two options, I think:
>> >
>> > a) Operators still set traits
>> on nodes, drivers are simply
>> > validating them. E.g.
>> > an operators sets
>> CUSTOM_RAID_5, and the node's RAID
>> > interface checks if it is
>> > possible to do. The downside is
>> obvious - with a lot of
>> > deploy templates
>> > available it can be a lot of
>> manual work.
>> >
>> > b) Drivers report the traits,
>> and they get somehow
>> added to
>> > the traits provided
>> > by an operator. Technically,
>> there are sub-cases again:
>> > b.1) The new traits API
>> returns a union of
>> > operator-provided and
>> > driver-provided traits
>> > b.2) The new traits API
>> returns only operator-provided
>> > traits; driver-provided
>> > traits are returned e.g. via a
>> new field
>> > (node.driver_traits). Then nova
>> will
>> > have to merge the lists itself.
>> >
>> > My personal favorite is the
>> last option: I'd like a clear
>> > distinction between
>> > different "sources" of traits,
>> but I'd also like to reduce
>> > manual work for
>> > operators.
>> >
>> > A valid counter-argument is:
>> what if an operator wants to
>> > override a
>> > driver-provided trait? E.g. a
>> node can do RAID 5, but I
>> > don't want this
>> > particular node to do it for
>> any reason. I'm not sure if
>> > it's a valid case, and
>> > what to do about it.
>> >
>> > Let me know what you think.
>> >
>> > Dmitry
>> >
>> >
>> > [1]
>> http://git.openstack.org/cgit/openstack/os-traits/tree/
>> <http://git.openstack.org/cgit/openstack/os-traits/tree/>
>> <http://git.openstack.org/cgit/openstack/os-traits/tree/
>> <http://git.openstack.org/cgit/openstack/os-traits/tree/>>
>> > [2] Based on how many attached
>> disks the node had, the
>> presence
>> > and abilities of a hardware RAID
>> controller, etc
>> >
>> >
>> >
>> >
>>
>> __________________________________________________________________________
>> > OpenStack Development Mailing List
>> (not for usage questions)
>> > Unsubscribe:
>> >
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
>> >
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>>
>> >
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
>> >
>> >
>> >
>> >
>> >
>> __________________________________________________________________________
>> > OpenStack Development Mailing List (not
>> for usage questions)
>> > Unsubscribe:
>> >
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
>> >
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>>
>> >
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
>> >
>> >
>> >
>> >
>>
>> __________________________________________________________________________
>> > OpenStack Development Mailing List (not for
>> usage questions)
>> > Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
>> >
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
>> >
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for
>> usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
>>
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage
>> questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>>
>>
>> --
>> --
>> -- Dmitry Tantsur
>> --
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list