May 2021 - openstack-discuss - lists.openstack.org

[ops][victoria][cinder] Import volume?
by DHilsbos＠performair.com 11 May '21

11 May '21

All; I'm looking to transfer several VMs from XenServer to an OpenStack Victoria cloud. Finding explanations for importing Glance images is easy, but I haven't been able to find a tutorial on importing Cinder volumes. Since they are currently independent servers / volumes it seems somewhat wasteful and messy to import each VMs disk as an image just to spawn a volume from it. We're using Ceph as the storage provider for Glance and Cinder. Thank you, Dominic L. Hilsbos, MBA Director - Information Technology Perform Air International Inc. DHilsbos(a)PerformAir.com www.PerformAir.com

3 5

[train][nova][neutron] gratiutous arp on centos 7
by Ignazio Cassano 11 May '21

11 May '21

Hello All, We've just updated openstack centos7 fron stein to train. Unfortunately live migrations works but vm lost a lot of pings after migration. We also applied the following patches but the do not solve: https://review.opendev.org/c/openstack/neutron/+/640258/ https://review.opendev.org/c/openstack/neutron/+/753314/ https://review.opendev.org/c/openstack/neutron/+/766277/ https://review.opendev.org/c/openstack/nova/+/742180/12 https://review.opendev.org/c/openstack/nova/+/747454/4 After that we also tried to apply https://review.opendev.org/c/openstack/nova/+/602432 but the issue remains the same. Please, anyone can help us ? Ignazio

1 0

[zun][horizon] How to display containers among VMs on network topology ?
by Maxime d'Estienne 11 May '21

11 May '21

Hello, I installed Zun on my stack and it works very well, I have a few containers up and running. I also added the zun-ui plugin for Horizon wich allows me to see the list of containers and to create them. But I often use the network topology view, wich I found very clean sometimes. I wondered if there is a solution to display the containers as are Vm's ? Thank you ! Maxime

2 2

[sdk]: identity service if get_application_credential method could use user name
by dmeng 11 May '21

11 May '21

Hello, Hope this email finds you well. Shall I please ask a question about the openstacksdk identity service, the application credential? We would like to get the expiration date of an application credential using the Identity v3 method get_application_credential or find_appplication_credential. We found that they require both the user id and the application credential id in order to get it. The code I'm using is as the following, and this works for us. conn = connection.Connection( session=sess, region_name='Victoria, identity_api_version='3') keystone = conn.identity find = keystone.get_application_credential(user='my_user_id', application_credential = 'app_cred_id') expire_date = find['expires_at'] We are wondering if we could use the user name to get it instead of the user id? If I do get_application_credential(user='catherine', application_credential = 'app_cred_id'), then it will show me an error that "You are not authorized to perform the requested action: identity:get_application_credential". Is there any method that no need user info, can just use the application credential id to get the expiration date? We also didn't find any documentation about the application credential in openstacksdk identity service docs. Thanks and have a great day! Catherine

2 5

Re: [cinder] Multiattach volume mount/attach limit
by Adam Tomas 11 May '21

11 May '21

Hi, thank you for the answer. Regards Adam > > Hi, > > This is currently not possible, and I believe there is no ongoing effort > to add this feature either. > > Cheers, > Gorka. > > >

1 0

[placement] clouds.yaml does not use placement API version variable
by jayaditya gupta 11 May '21

11 May '21

Even after specifying the variable in clouds.yaml placement still uses version 1.0 Story : https://storyboard.openstack.org/#!/story/2008553 Hello can someone help me on this one. I'm not sure where exactly is the issue in placement or in osc-placement . I would really appreciate the insight on this one. Thanks

1 0

[dev][tc]nova-scheduler: cpu-topology: sub-numa(numa nodes within one CPU socket) aware scheduler
by Song Bao Hua (Barry Song) 11 May '21

11 May '21

Dears, Historically, both AMD EPYC and Intel Xeon had NUMA nodes within one CPU socket. For Intel haswell/broadwell, they could have two rings to connect multiple CPUs: +----------------------------+ +-------------------------+ +---+ +-----+ +----+ +---+ +---+ +-----+ +----+ +---+ | | | | +---+ +-----+ +----+ ring +---+ +---+ ring +-----+ +----+ +---+ | | | | +---+ +-----+ +----+ +---+ +---+ +-----+ +----+ +---+ | | | | | | | | +----------------------------+ +-------------------------+ Due to the latency to access the memory located in different rings, cluster-on-die(COD) NUMA nodes did exist in haswell/ broadwell. For AMD EPYC, they used to have 4 DIEs in one CPU socket: +-------------------------------------------------+ | AMD EPYC | | +-------------+ +-------------+ | | | | | | | | | die | | die | | | | |-------- | | | | |\ /| | | | +-------------+ -\ /- +-------------+ | | | /-\ | | | +-------|------+/- -+-------|------+ | | | - | | | | | die | | die | | | | |------| | | | | | | | | | +--------------+ +--------------+ | | | +-------------------------------------------------+ These 4 different DIEs could be configured as 4 different NUMA. However, with the development of hardware, Intel and AMD no longer need to be aware of NUMA nodes within single one CPU. Intel has moved to mesh since skylake(server): +-------|-----------------------+ | | | | | | | | | | |-------|-------|-------|-------| | | | | | | | | | | |-------|-------|-------|-------| | | | | | | | | | | |-------|-------|-------|-------| | | | | | | | | | | +-------------------------------+ AMD moved memory and I/O to a separate DIE, thus made the whole CPU become UMA in EPYC II: +---------+ +---------+ +---------+ +---------+ | cpu die | |cpu die | |cpu die | |cpu die | | | | | | | | | +---------+ +---------+ +---------+ +---------+ +------------------------------------------------------+ | | | memory and I/O DIE | | | +------------------------------------------------------+ +---------+ +---------+ +---------+ +---------+ |cpu die | |cpu die | |cpu die | |cpu die | | | | | | | | | +---------+ +---------+ +---------+ +---------+ Skylake, while still having SNC(sub-numa-cluster) within a DIE for a huge mesh network, doesn't really suffer from this kind of internal sub-numa since the latency difference is really minor to enable SNC or not. According to "64-ia-32-architectures-optimization-manual"[1], for a typical 2P system, disabling and enabling SNC will bring minor latency difference for memory access within one CPU: SNC off: Using buffer size of 2000.000MB Measuring idle latencies (in ns)... Numa node Numa node 0 1 0 81.9 153.1 1 153.7 82.0 SNC on: Using buffer size of 2000.000MB Measuring idle latencies (in ns)... Numa node Numa node 0 1 2 3 0 81.6 89.4 140.4 153.6 1 86.5 78.5 144.3 162.8 2 142.3 153.0 81.6 89.3 3 144.5 162.8 85.5 77.4 As above, if we disable sub-numa in one CPU, the CPU has memory latency as 81.9, but if sub-numa is enabled, the advantage is really minor, SNC can access its own memory with the latency of 81.6 with 0.3 gap only. So SNC doesn't really matter on Xeon. However, the development of kunpeng920's topology is different from Intel and AMD. Similar with AMD EPYC, kunpeng920 has two DIEs in one CPU. Unlike EPYC which has only 8cores in each DIE, each DIE of kunpeng920 has 24 or 32 cores. For a typical 2P system, we can set it to 2NUMA or 4NUMA. +------------------------------+ +------------------------------+ | CPU | | CPU | | +----------+ +----------+ | | +----------+ +----------+ | | | | | | | | | | | | | | | DIE | | DIE | | | | DIE | | DIE | | | | | | | | | | | | | | | | | | | | | | | | | | | +----------+ +----------+ | | +----------+ +----------+ | | | | | +------------------------------+ +------------------------------+ * 2NUMA - DIE interleave In this case, two DIEs become one NUMA. The advantage is that we are getting more CPUs in one NUMA, so this decreases the fragment of CPUs and help deploy more virtual machines on the same host when we apply the rule of pinning VM within one NUMA, compared with disabling DIE interleave. But, this has some obvious disadvantage. Since we need to run DIE interleave, the average memory access latency could increase much. Enabling DIE interleave: Numa node Numa node 0 1 0 95.68 199.21 1 199.21 95.68 Disabling DIE interleave Numa node Numa node 0 1 2 3 0 85.79 104.33 189.95 209.00 1 104.33 85.79 209.00 229.60 2 189.95 209.00 85.79 104.33 3 209.00 229.60 104.33 85.79 As above, one DIE can access its local memory with latency of 85.79, but when die-interleave is enabled, the latency becomes 95.68. The gap 95.68-85.79 isn't minor. * 4NUMA - each DIE becomes one NUMA In this way, NUMA-aware software can access its local memory with much lower latency. Thus, we can gain some performance improvement in openstack if one VM is deployed in one DIE and only access its local memory. Testing has shown 4%+ performance improvement. However, under the rule that VM does not cross NUMA, less VMs could be deployed on the same host. In order to achieve both goals of performance improvement and anti-fragmentation, we are looking for some way of sub-numa awareness in openstack scheduler. That means, 1. we will disable DIE interleave, thus, for 2P system, we are going to have 4 NUMA; 2. we use the rule VM shouldn't cross CPU, thinking one CPU as a big NUMA and each DIE as a smaller sub-numa. 3. For guests, we can define different priorities, for high priority VMs whose performance is a great concern, openstack will try to deploy them in same sub-numa which is a DIE for kunpeng920; for low priority VMs who can endure across-die latency, they could be placed on different sub-numa, but still within same CPU. So basically, we'd like to make openstack aware of the topology of sub-numa within one cpu and support a rule which prevents cross-CPU but allows cross-DIE. At the same time, this scheduler should try its best to deploy important VMs in same sub-numa and allow relatively unimportant VMs to cross two sub-numa. This might be done by customized openstack flavor according to [2], but we are really looking for some flexible and automatic way. If openstack scheduler can recognize this kind of CPU topology and make smart decision accordingly, it would be nice. Please let me know how you think about it. And alternative ways also welcome. [1] https://software.intel.com/content/www/us/en/develop/download/intel-64-and-… [2] https://docs.openstack.org/nova/pike/admin/cpu-topologies.html Thanks Barry

1 0

Re: [ironic] IPA image does not want to boot with UEFI
by Julia Kreger 10 May '21

10 May '21

Adding the list back and trimming the message. Replies in-band. Well, that is good that the server is not signed, nor other esp images are not working. On Thu, Apr 1, 2021 at 11:20 AM Vuk Gojnic <vuk.gojnic(a)gmail.com> wrote: > > Hey Julia, > > Thanks for asking. I have tried with several ESP image options with same effect (one taken from Ubuntu Live ISO that boots on that node, another downloaded and third made with grub tools). None of them was signed. Interesting. At least it is consistent! Have you tried to pull down the iso image and take it apart to verify it is UEFI bootable against a VM or another physical machine? I'm wondering if you need both uefi parameters set. You definitely don't have properties['capabilities']['boot_mode'] set which is used or... maybe a better word to use is drawn in for asserting defaults, but you do have the deploy_boot_mode setting set. I guess a quick manual sanity check of the actual resulting iso image is going to be critical. Debug logging may also be useful, and I'm only thinking that because there is no logging from the generation of the image. > > The server is not in UEFI secure boot mode. Interesting, sure sounds like it is based on your original message. :( > Btw. I will be on holidays for next week so I might not be able to follow up on this discussion before Apr 12th. No worries, just ping us on irc.freenode.net in #openstack-ironic if a reply on the mailing list doesn't grab our attention. > > Bests, > Vuk > > On Thu, Apr 1, 2021 at 4:20 PM Julia Kreger <juliaashleykreger(a)gmail.com> wrote: >> >> Greetings, >> >> Two questions: >> 1) Are the ESP image contents signed, or are they built using one of >> the grub commands? >> 2) Is the machine set to enforce secure boot at this time? >> >> [trim]

2 3

[cinder] Multiattach volume mount/attach limit
by Adam Tomas 10 May '21

10 May '21

Hi, Is there any way to limit the number of possible mounts of multiattach volume? Best regards Adam Tomas

2 1

[tripleo][ci] all c8 master / wallaby blocked
by Wesley Hayutin 10 May '21

10 May '21

https://bugs.launchpad.net/tripleo/+bug/1927952 https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-content-prov… Issues atm w/ quay.io, we are making sure our failover is working properly atm. RDO was missing the latest ceph containers. Thanks

1 1