[openstack-dev] [ironic] [nova] traits discussion call
Jay Pipes
jaypipes at gmail.com
Mon Oct 30 14:13:46 UTC 2017
I'd prefer to have you on the call, Dima. How about we push it back to
tomorrow at the same time?
Can everyone make it then?
-jay
On 10/30/2017 10:11 AM, Dmitry Tantsur wrote:
> Aaaand sorry again, but due to sudden errands I won't be able to attend.
> Please feel free to use my bluejeans room anyway. I think my position on
> traits is more or less clear from previous discussions with John, Sam
> and Eric.
>
> 2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>>:
>
> Sigh, sorry. I forgot that we're moving back to winter time this
> weekend. I *think* the time is 3pm UTC then. It seems to be 11am
> eastern US:
> https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et
> <https://www.timeanddate.com/worldclock/converter.html?iso=20171030T150000&p1=37&p2=tz_et>.
>
>
> On 10/24/2017 06:00 PM, Dmitry Tantsur wrote:
>
> And the winner is Mon, 30 Oct, 2pm UTC!
>
> The bluejeans ID is https://bluejeans.com/757528759
> <https://bluejeans.com/757528759>
> (works without plugins in recent FF and Chrome; if it asks to
> install an app, ignore it and look for a link saying "join with
> browser")
>
> On 10/23/2017 05:02 PM, Dmitry Tantsur wrote:
>
> Hi all!
>
> I'd like to invite you to the discussion of the way to
> implement traits in
> ironic and the ironic virt driver. Please vote for the time at
> https://doodle.com/poll/ts43k98kkvniv8uz
> <https://doodle.com/poll/ts43k98kkvniv8uz>. Please vote by
> EOD tomorrow.
>
> Note that it's going to be a technical discussion - please
> make sure you
> understand what traits are and why ironic cares about them.
> See below for more
> context.
>
> We'll probably use my bluejeans account, as it works without
> plugins in modern
> browsers. I'll post a meeting ID when we pick the date.
>
>
> On 10/23/2017 04:09 PM, Eric Fried wrote:
>
> We discussed this a little bit further in IRC [1].
> We're all in
> agreement, but it's worth being precise on a couple of
> points:
>
> * We're distinguishing between a "feature" and the
> "trait" that
> represents it in placement. For the sake of this
> discussion, a
> "feature" can (maybe) be switched on or off, but a
> "trait" can either be
> present or absent on a RP.
> * It matters *who* can turn a feature on/off.
> * If it can be done by virt at spawn time, then it
> makes sense to have
> the trait on the RP, and you can switch the feature
> on/off via a
> separate extra_spec.
> * But if it's e.g. an admin action, and spawn has
> no control, then the
> trait needs to be *added* whenever the feature is *on*,
> and *removed*
> whenever the feature is *off*.
>
> [1]
> http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13
> <http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13>
>
>
> On 10/23/2017 08:15 AM, Sylvain Bauza wrote:
>
>
>
> On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried
> <openstack at fried.cc
> <mailto:openstack at fried.cc
> <mailto:openstack at fried.cc>>> wrote:
>
> I agree with Sean. In general terms:
>
> * A resource provider should be marked with a
> trait if that feature
> * Can be turned on or off (whether it's
> currently on or not); or
> * Is always on and can't ever be turned off.
>
>
> No, traits are not boolean. If a resource provider
> stops providing a
> capability, then the existing related trait should
> just be removed,
> that's it.
> If you see a trait, that's just means that the
> related capability for
> the Resource Provider is supported, that's it too.
>
> MHO.
>
> -Sylvain
>
>
>
> * A consumer wanting that feature present
> (doesn't matter whether it's
> on or off) should specify it as a required
> *trait*.
> * A consumer wanting that feature present and
> turned on should
> * Specify it as a required trait; AND
> * Indicate that it be turned on via some
> other mechanism (e.g. a
> separate extra_spec).
>
> I believe this satisfies Dmitry's (Ironic's)
> needs, but also Jay's drive
> for placement purity.
>
> Please invite me to the hangout or whatever.
>
> Thanks,
> Eric
>
> On 10/23/2017 07:22 AM, Mooney, Sean K wrote:
> >
> >
> >
> >
> > *From:*Jay Pipes [mailto:jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>
> <mailto:jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>>]
> > *Sent:* Monday, October 23, 2017 12:20 PM
> > *To:* OpenStack Development Mailing List
> <openstack-dev at lists.openstack.org
> <mailto:openstack-dev at lists.openstack.org>
> <mailto:openstack-dev at lists.openstack.org
> <mailto:openstack-dev at lists.openstack.org>>>
> > *Subject:* Re: [openstack-dev] [ironic]
> ironic and traits
> >
> >
> >
> > Writing from my phone... May I ask that
> before you proceed with any plan
> > that uses traits for state information that
> we have a hangout or
> > videoconference to discuss this?
> Unfortunately today and tomorrow I'm
> > not able to do a hangout but I can do one
> on Wednesday any time of the day.
> >
> >
> >
> > */[Mooney, Sean K] on the uefi boot topic I
> did bring up at the
> ptg that
> > we wanted to standardizes tratis for
> “verified boot” /*
> >
> > */that included a trait for uefi secure
> boot enabled and to
> indicated a
> > hardware root of trust, e.g. intel boot
> guard or similar/*
> >
> > */we distinctly wanted to be able to tag
> nova compute hosts with those
> > new traits so we could require that vms
> that request/*
> >
> > */a host with uefi secure boot enabled and
> a hardware root of
> trust are
> > scheduled only to those nodes. /*
> >
> > */ /*
> >
> > */There are many other examples that effect
> both vms and bare
> metal such
> > as, ecc/interleaved memory, cluster on die, /*
> >
> > */l3 cache code and data prioritization,
> vt-d/vt-c, HPET, Hyper
> > threading, power states … all of these
> feature may be present on the
> > platform/*
> >
> > */but I also need to know if they are
> turned on. Ruling out state in
> > traits means all of this logic will
> eventually get pushed to scheduler
> > filters/*
> >
> > */which will be suboptimal long term as
> more state is tracked.
> Software
> > defined infrastructure may be the future
> but hardware defined
> software/*
> >
> > */is sadly the present…/*
> >
> > */ /*
> >
> > */I do however think there should be a
> sperateion between asking for a
> > host that provides x with a trait and
> asking for x to be
> configure via/*
> >
> > */A trait. The trait secure_boot_enabled
> should never result in the
> > feature being enabled It should just find a
> host with it on. If
> you want/*
> >
> > */To request it to be turned on you would
> request a host with
> > secure_boot_capable as a trait and have a
> flavor extra spec or image
> > property to request/*
> >
> > */Ironic to enabled it. these are two very
> different request and
> should
> > not be treated the same. /*
> >
> >
> >
> >
> >
> > Lemme know!
> >
> > -jay
> >
> >
> >
> > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur"
> <dtantsur at redhat.com <mailto:dtantsur at redhat.com>
> <mailto:dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>>
> > <mailto:dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>
> <mailto:dtantsur at redhat.com
> <mailto:dtantsur at redhat.com>>>> wrote:
> >
> > Hi Jay!
> >
> > I appreciate your comments, but I think
> you're approaching the
> > problem from purely VM point of view.
> Things simply don't work the
> > same way in bare metal, at least not if
> we want to provide the same
> > user experience.
> >
> >
> >
> > On Sun, Oct 22, 2017 at 2:25 PM, Jay
> Pipes <jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>
> <mailto:jaypipes at gmail.com <mailto:jaypipes at gmail.com>>
> > <mailto:jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>
> <mailto:jaypipes at gmail.com
> <mailto:jaypipes at gmail.com>>>> wrote:
> >
> > Sorry for delay, took a week off
> before starting a new job.
> > Comments inline.
> >
> > On 10/16/2017 12:24 PM, Dmitry
> Tantsur wrote:
> >
> > Hi all,
> >
> > I promised John to dump my
> thoughts on traits to the
> ML, so
> > here we go :)
> >
> > I see two roles of traits (or
> kinds of traits) for
> bare metal:
> > 1. traits that say what the
> node can do already (e.g. "the
> > node is
> > doing UEFI boot")
> > 2. traits that say what the
> node can be *configured* to do
> > (e.g. "the node can
> > boot in UEFI mode")
> >
> >
> > There's only one role for traits.
> #2 above. #1 is state
> > information. Traits are not for
> state information. Traits are
> > only for communicating capabilities
> of a resource provider
> > (baremetal node).
> >
> >
> >
> > These are not different, that's what
> I'm talking about here. No
> > users care about the difference between
> "this node was put in UEFI
> > mode by an operator in advance", "this
> node was put in UEFI
> mode by
> > an ironic driver on demand" and "this
> node is always in UEFI mode,
> > because it's AARCH64 and it does not
> have BIOS". These situation
> > produce the same result (the node is
> booted in UEFI mode), and
> thus
> > it's up to ironic to hide this difference.
> >
> >
> >
> > My suggestion with traits is one way to
> do it, I'm not sure
> what you
> > suggest though.
> >
> >
> >
> >
> > For example, let's say we add the
> following to the os-traits
> > library [1]
> >
> > * STORAGE_RAID_0
> > * STORAGE_RAID_1
> > * STORAGE_RAID_5
> > * STORAGE_RAID_6
> > * STORAGE_RAID_10
> >
> > The Ironic administrator would add
> all RAID-related traits to
> > the baremetal nodes that had the
> *capability* of
> supporting that
> > particular RAID setup [2]
> >
> > When provisioned, the baremetal
> node would either have RAID
> > configured in a certain level or
> not configured at all.
> >
> >
> > A very important note: the
> Placement API and Nova
> scheduler (or
> > future Ironic scheduler) doesn't
> care about this. At all.
> I know
> > it sounds like I'm being callous,
> but I'm not. Placement and
> > scheduling doesn't care about the
> state of things. It only
> cares
> > about the capabilities of target
> destinations. That's it.
> >
> >
> >
> > Yes, because VMs always start with a
> clean state, and
> hypervisor is
> > there to ensure that. We don't have
> this luxury in ironic :) E.g.
> > our SNMP driver is not even aware of
> boot modes (or RAID, or BIOS
> > configuration), which does not mean
> that a node using it cannot be
> > in UEFI mode (have a RAID or BIOS
> pre-configured, etc, etc).
> >
> >
> >
> >
> >
> > This seems confusing, but it's
> actually very useful.
> Say, I
> > have a flavor that
> > requests UEFI boot via a trait.
> It will match both the
> nodes
> > that are already in
> > UEFI mode, as well as nodes
> that can be put in UEFI mode.
> >
> >
> > No :) It will only match nodes that
> have the UEFI capability.
> > The set of providers that have the
> ability to be booted
> via UEFI
> > is *always* a superset of the set
> of providers that *have been
> > booted via UEFI*. Placement and
> scheduling decisions only care
> > about that superset -- the
> providers with a particular
> capability.
> >
> >
> >
> > Well, no, it will. Again, you're purely
> basing on the VM idea,
> where
> > a VM is always *put* in UEFI mode, no
> matter how the hypervisor
> > looks like. It is simply not the case
> for us. You have to care
> what
> > state the node is, because many drivers
> cannot change this state.
> >
> >
> >
> >
> >
> > This idea goes further with
> deploy templates (new concept
> > we've been thinking
> > about). A flavor can request
> something like CUSTOM_RAID_5,
> > and it will match the
> > nodes that already have RAID 5,
> or, more
> interestingly, the
> > nodes on which we
> > can build RAID 5 before
> deployment. The UEFI example above
> > can be treated in a
> > similar way.
> >
> > This ends up with two sources
> of knowledge about traits in
> > ironic:
> > 1. Operators setting something
> they know about hardware
> > ("this node is in UEFI
> > mode"),
> > 2. Ironic drivers reporting
> something they
> > 2.1. know about hardware
> ("this node is in UEFI mode" -
> > again)
> > 2.2. can do about hardware
> ("I can put this node in
> UEFI
> > mode")
> >
> >
> > You're correct that both pieces of
> information are important.
> > However, only the "can do about
> hardware" part is relevant to
> > Placement and Nova.
> >
> > For case #1 we are planning on
> a new CRUD API to set/unset
> > traits for a node.
> >
> >
> > I would *strongly* advise against
> this. Traits are not for
> state
> > information.
> >
> > Instead, consider having a DB (or
> JSON) schema that lists
> state
> > information in fields that are
> explicitly for that state
> > information.
> >
> > For example, a schema that looks
> like this:
> >
> > {
> > "boot": {
> > "mode": <one of 'bios' or 'uefi'>,
> > "params": <dict>
> > },
> > "disk": {
> > "raid": {
> > "level": <int>,
> > "controller": <one of 'sw' or
> 'hw'>,
> > "driver": <string>,
> > "params": <dict>
> > }, ...
> > },
> > "network": {
> > ...
> > }
> > }
> >
> > etc, etc.
> >
> > Don't use trait strings to
> represent state information.
> >
> >
> >
> > I don't see an alternative proposal
> that will satisfy what we have
> > to solve.
> >
> >
> >
> >
> > Best,
> > -jay
> >
> > Case #2 is more interesting. We
> have two options, I think:
> >
> > a) Operators still set traits
> on nodes, drivers are simply
> > validating them. E.g.
> > an operators sets
> CUSTOM_RAID_5, and the node's RAID
> > interface checks if it is
> > possible to do. The downside is
> obvious - with a lot of
> > deploy templates
> > available it can be a lot of
> manual work.
> >
> > b) Drivers report the traits,
> and they get somehow
> added to
> > the traits provided
> > by an operator. Technically,
> there are sub-cases again:
> > b.1) The new traits API
> returns a union of
> > operator-provided and
> > driver-provided traits
> > b.2) The new traits API
> returns only operator-provided
> > traits; driver-provided
> > traits are returned e.g. via a
> new field
> > (node.driver_traits). Then nova
> will
> > have to merge the lists itself.
> >
> > My personal favorite is the
> last option: I'd like a clear
> > distinction between
> > different "sources" of traits,
> but I'd also like to reduce
> > manual work for
> > operators.
> >
> > A valid counter-argument is:
> what if an operator wants to
> > override a
> > driver-provided trait? E.g. a
> node can do RAID 5, but I
> > don't want this
> > particular node to do it for
> any reason. I'm not sure if
> > it's a valid case, and
> > what to do about it.
> >
> > Let me know what you think.
> >
> > Dmitry
> >
> >
> > [1]
> http://git.openstack.org/cgit/openstack/os-traits/tree/
> <http://git.openstack.org/cgit/openstack/os-traits/tree/>
>
> <http://git.openstack.org/cgit/openstack/os-traits/tree/
> <http://git.openstack.org/cgit/openstack/os-traits/tree/>>
> > [2] Based on how many attached
> disks the node had, the
> presence
> > and abilities of a hardware RAID
> controller, etc
> >
> >
> >
> >
>
> __________________________________________________________________________
> > OpenStack Development Mailing List
> (not for usage questions)
> > Unsubscribe:
> >
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
> >
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>>
> >
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
> >
> >
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not
> for usage questions)
> > Unsubscribe:
> >
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
> >
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>>
> >
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
> >
> >
> >
> >
>
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for
> usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
> >
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
> >
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for
> usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage
> questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> --
> --
> -- Dmitry Tantsur
> --
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
More information about the OpenStack-dev
mailing list