[nova] SCS standardized flavor naming
Hi, we (SCS) are working on defining a fully open source cloud and container stack as part of the Gaia-X[1] project. The intention is to provide a common well-standardized way to deploy, manage, configure, and operate the needed software. The vision is to have a network of federated clouds that can be used as one, which requires IAM federation and a high level of compatibility and uniformity. Our project is called Sovereign Cloud Stack (SCS)[2]. Obviously, we are using existing open source projects from the OIF, the CNCF and others and are seeking alignment with these communities. Some experts well-known in the OpenStack universe are participating in our effort. On the OpenStack side, we are using OSISM[3] which leverages kolla-ansible. We would like to seek your input and feedback into our attempt of defining a standardized naming scheme for flavors and a list of standard flavors available in all clouds that deliver SCS-compliant IaaS. Find the draft proposal at https://github.com/SovereignCloudStack/Operational-Docs/blob/main/flavor-nam... We prefer feedback as github issues and/or PRs. Knowing that the OpenStack community prefers gerrit, we'll of course also incorporate any comment we get via this mailing list into our thinking. We hope you can accept us pasting content from mails into github issues, so we create a track record of the taken decisions. (Please indicate if this is not OK for you and we'll refrain from doing so.) Before you ask why we believe the proposal is useful: We are perfectly aware that it is possible to discover flavor properties; however, in many automation recipes (playbooks, terraform vars etc) we find flavor names encoded and it is one pain point having to adapt them on every cloud that you use. So we want to have some uniformity on this across SCS clouds. (With similar reasoning, expect a image naming and image metadata proposal next.) We are looking for feedback in two directions: (1) If you are aware of similar efforts to standardize flavor naming, please point us to it, so we can seek contact and align. (2) Please look at the proposal itself. When looking into the details how we specify how to optionally(!) encode a number of details into flavor names, please keep in mind that this is indeed optional. We expect most flavor names to be as simple as SCS-4V:8:20 or even SCS-4V:8, even though complicated SCS-8C:32:2x200S-bms-i2-GNa:64-ib [4] is possible for clouds that provide that level of differentiation and want/need to expose this via the flavor name. Of course, input from existing providers of OpenStack infrastructure is particularly valuable. Feedback welcome! [1] https://gaia-x.eu/ [2] https://scs.community/ [3] https://osism.de/ [4] In case you wonder: 8 dedicated cores, 32GiB RAM, 2x200GB SSD disks on bare metal sys, intel Cascade Lake, nVidia GPU with 64 Ampere SMs and InfiniBand. PS: Cc'ing some folks who have contributed to this. -- Kurt Garloff CTO Sovereign Cloud Stack OSB Alliance e.V.
On 6/17/21 7:54 PM, Kurt Garloff wrote:
Hi,
we (SCS) are working on defining a fully open source cloud and container stack as part of the Gaia-X[1] project. The intention is to provide a common well-standardized way to deploy, manage, configure, and operate the needed software. The vision is to have a network of federated clouds that can be used as one, which requires IAM federation and a high level of compatibility and uniformity. Our project is called Sovereign Cloud Stack (SCS)[2]. Obviously, we are using existing open source projects from the OIF, the CNCF and others and are seeking alignment with these communities. Some experts well-known in the OpenStack universe are participating in our effort. On the OpenStack side, we are using OSISM[3] which leverages kolla-ansible.
We would like to seek your input and feedback into our attempt of defining a standardized naming scheme for flavors and a list of standard flavors available in all clouds that deliver SCS-compliant IaaS.
Find the draft proposal at https://github.com/SovereignCloudStack/Operational-Docs/blob/main/flavor-nam...
We prefer feedback as github issues and/or PRs. Knowing that the OpenStack community prefers gerrit, we'll of course also incorporate any comment we get via this mailing list into our thinking. We hope you can accept us pasting content from mails into github issues, so we create a track record of the taken decisions. (Please indicate if this is not OK for you and we'll refrain from doing so.)
Before you ask why we believe the proposal is useful: We are perfectly aware that it is possible to discover flavor properties; however, in many automation recipes (playbooks, terraform vars etc) we find flavor names encoded and it is one pain point having to adapt them on every cloud that you use. So we want to have some uniformity on this across SCS clouds. (With similar reasoning, expect a image naming and image metadata proposal next.)
We are looking for feedback in two directions:
(1) If you are aware of similar efforts to standardize flavor naming, please point us to it, so we can seek contact and align.
(2) Please look at the proposal itself. When looking into the details how we specify how to optionally(!) encode a number of details into flavor names, please keep in mind that this is indeed optional. We expect most flavor names to be as simple as SCS-4V:8:20 or even SCS-4V:8, even though complicated SCS-8C:32:2x200S-bms-i2-GNa:64-ib [4] is possible for clouds that provide that level of differentiation and want/need to expose this via the flavor name.
Of course, input from existing providers of OpenStack infrastructure is particularly valuable.
Feedback welcome!
[1] https://gaia-x.eu/ [2] https://scs.community/ [3] https://osism.de/ [4] In case you wonder: 8 dedicated cores, 32GiB RAM, 2x200GB SSD disks on bare metal sys, intel Cascade Lake, nVidia GPU with 64 Ampere SMs and InfiniBand.
PS: Cc'ing some folks who have contributed to this.
Hi, While I do like the idea of a standard for flavor naming, I don't like at all what I've seen as your example. IMO, it's best if more explicit. I wouldn't be able to read one of your flavor names, and immediately know what it is made of. To the contrary... We're using naming scheme like this: nvt4-a8-ram24-disk50-perf2 This means: - nvt4: nvidia T4 GPU - a8: AMD VCPU 8 (we also have i4 for example, for Intel) - ram24: 24 GB of RAM - disk50: 50 GB of local system disk - perf2: level 2 of IOps / IO bandwidth Having explicit and full "ram", "disk" and "perf" in the name helps a lot to understand. I think it's much nicer than then cryptic: "SCS-16T:64:200s-GNa:64-ib" which I would never be able to decode without a document on the side. I understand that you're attempting to make the flavor name smaller, but IMO that's a bad idea. I don't see any problem with an explicit and longer flavor name. Cheers, Thomas Goirand (zigo)
Hi Thomas,
On 6/17/21 7:54 PM, Kurt Garloff wrote: [...]
We would like to seek your input and feedback into our attempt of defining a standardized naming scheme for flavors and a list of standard flavors available in all clouds that deliver SCS-compliant IaaS.
Find the draft proposal at https://github.com/SovereignCloudStack/Operational-Docs/blob/main/flavor-nam... [...] (2) Please look at the proposal itself. When looking into the details how we specify how to optionally(!) encode a number of details into flavor names, please keep in mind that this is indeed optional. We expect most flavor names to be as simple as SCS-4V:8:20 or even SCS-4V:8, even though complicated SCS-8C:32:2x200S-bms-i2-GNa:64-ib [4] is possible for clouds that provide that level of differentiation and want/need to expose this via the flavor name.
Of course, input from existing providers of OpenStack infrastructure is particularly valuable.
Feedback welcome!
[1] https://gaia-x.eu/ [2] https://scs.community/ [3] https://osism.de/ [4] In case you wonder: 8 dedicated cores, 32GiB RAM, 2x200GB SSD disks on bare metal sys, intel Cascade Lake, nVidia GPU with 64 Ampere SMs and InfiniBand.
PS: Cc'ing some folks who have contributed to this. On 18/06/2021 00:40, Thomas Goirand wrote: Hi,
While I do like the idea of a standard for flavor naming, I don't like at all what I've seen as your example. IMO, it's best if more explicit. I wouldn't be able to read one of your flavor names, and immediately know what it is made of. To the contrary...
We're using naming scheme like this: nvt4-a8-ram24-disk50-perf2
This means: - nvt4: nvidia T4 GPU - a8: AMD VCPU 8 (we also have i4 for example, for Intel) - ram24: 24 GB of RAM - disk50: 50 GB of local system disk - perf2: level 2 of IOps / IO bandwidth We could argue whether the ram and disk prefixes should be used to improve human parsing. I have seen many naming schemes that always had CPU:RAM[:DISK] in this order with both RAM and DISK in GB. This is contained at the beginning of every single name, so you get used to very very quickly. So we compressed this piece more strongly than the optional pieces which you don't see so often, such as -xen/kvm/bms/vmw/hyv or -ib.
The rest gets a tiny bit of time to get used to, agreed. Most SCS flavors in our scheme would just be SCS-8V:24:50, your flavor could be SCS-8C:24:50S-a-GNt:40 or so. (We have not yet written down a way to count Tensor Cores, something that we shouldadd; reporting the 40SMs here is not so relevant as the 320tensor cores.) This assumes that the 8 vCPUs are real cores (otherwise 8Vinstead of 8C), that your perf2 is a SSD type of performance and that the GPU is a pass-through device (and not a virtualized vGPU). What I dislike about your scheme is that you don't put the nonstandard-pieces (the GPU) to the end -- I think it's easier to keep an overview over your flavors if they always start the same way. Also, you can't see whether the 8 vCPUs are dedicated cores, HTs or oversubscribed things; you have not specified the CPU generation (which in my opinion makes the amd vs intel spec somewhat useless -- somecustomers really want to know whether they have Cascade Lake or Ice Lake or Zen1 vs Zen2) nor a way to express generic x86 (x?). Is your scheme used by a lot of providers?
Having explicit and full "ram", "disk" and "perf" in the name helps a lot to understand. I think it's much nicer than then cryptic:
"SCS-16T:64:200s-GNa:64-ib"
which I would never be able to decode without a document on the side. I understand that you're attempting to make the flavor name smaller, but IMO that's a bad idea. I don't see any problem with an explicit and longer flavor name.
If I take your scheme, the flavor would become "scs-x16thr-ram64-disk200-perf2-nva30-ib" or so. This assumes I stick to my ordering, use x for generic x86, thr for dedicated hyperthreads, perf2=ssd and we're using an Nvidia A30 (which has 56SMs not 64 so the match is not perfect) and -ib is verbose enough for infiniband ... It's a bit longish, but no real roadblock. I would argue that readibility is not better once you have spent some minutes with the spec. Again, most flavors would be just SCS-16T:64:200, which does not take a lot of training. Thanks for your feedback -- let's have this discussion! Happy to see others weighing in as well. -- Kurt Garloff CTO Sovereign Cloud Stack OSB Alliance e.V.
Hi Kurt, Thanks for this discussion, this is interesting. On 6/18/21 12:05 PM, Kurt Garloff wrote: ]> Also, you can't see whether the 8 vCPUs are dedicated cores, HTs
or oversubscribed things; you have not specified the CPU generation (which in my opinion makes the amd vs intel spec somewhat useless -- somecustomers really want to know whether they have Cascade Lake or Ice Lake or Zen1 vs Zen2) nor a way to express generic x86 (x?).
This probably makes sense if a cluster has many types of CPU, though you wont find that often. In our case, we only have one type of CPU per cluster: 2x AMD EPYC 7452 32-Cores on every compute for that brand new public cloud we're about to release (so 128 threads on each compute): so we use the EPYC-v2 CPU model of Qemu everywhere. IMO, there's no need to express what type of CPU if there's only one available anyways...
Is your scheme used by a lot of providers?
I don't know, but it's used by us! :) By the way, in our case, perf1 vs perf2 doesn't express a change of backend. Both are using NVMe (4th gen, so REALLY fast...) on a Ceph cluster, but it express a different IOps and bandwidth limiting. So maybe we should find a better way to express I/O perfs than just a lasting number? Maybe nvmeperf1 vs ssdperf1?
If I take your scheme, the flavor would become "scs-x16thr-ram64-disk200-perf2-nva30-ib" or so. This assumes I stick to my ordering, use x for generic x86, thr for dedicated hyperthreads, perf2=ssd and we're using an Nvidia A30 (which has 56SMs not 64 so the match is not perfect) and -ib is verbose enough for infiniband ...
Why would someone want to use infiniband these days? We get Mellanox cards at 2x 25Gbits/s for a very cheap price these days... And then, same thing: why would you express the speed of your network in the flavor, when most likely, it's going to be the same on all flavors (and you cluster will most likely have a single type of NIC speed...)?
It's a bit longish, but no real roadblock.
IMO, it's a way more readable this way. Long isn't a problem, really, if it adds redability.
I would argue that readibility is not better once you have spent some minutes with the spec.
The point is, a new customer will *not* spend time reading the spec. Typically, they will want to just fire-up a VM quickly without reading too much docs... Cheers, Thomas Goirand (zigo)
participants (2)
-
Kurt Garloff
-
Thomas Goirand