November 2020 - openstack-discuss

[tripleo][ci] cm2 pip failures
by Wesley Hayutin 30 Nov '20

30 Nov '20

Greetings, FYI.. https://bugs.launchpad.net/tripleo/+bug/1906265 0/

2 2

[qa][all] pip 20.3 breaks jobs
by Balázs Gibizer 30 Nov '20

30 Nov '20

Hi, Today pip 20.3 is released[1] and jobs started failing during keystone install in devstack with: 2020-11-30 15:14:31.117 | + inc/python:pip_install:193 : sudo -H LC_ALL=en_US.UTF-8 SETUPTOOLS_USE_DISTUTILS=stdlib http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite python3.6 -m pip install -c /opt/stack/old/requirements/upper-constraints.txt -e /opt/stack/old/neutron 2020-11-30 15:14:32.271 | Looking in indexes: https://mirror.gra1.ovh.opendev.org/pypi/simple, https://mirror.gra1.ovh.opendev.org/wheel/ubuntu-18.04-x86_64 2020-11-30 15:14:32.272 | DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210. 2020-11-30 15:14:32.272 | ERROR: Links are not allowed as constraints Logstash signature: [2] It seems not all the jobs are effected but at least grenade on master is hit [3]. It seems that passing jobs are using pip from the base image instead of installing a fresh one from pypi. As a first glance the change "Flip the switch on new resolver"[4] seem to be related. Cheers, gibi [1] https://pip.pypa.io/en/stable/news/#id1 [2] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3… [3] https://zuul.opendev.org/t/openstack/builds?job_name=grenade&branch=master [4] https://github.com/pypa/pip/pull/9019

3 2

senlin SR-IOV instance support
by Satish Patel 30 Nov '20

30 Nov '20

Folks, Does senlin support instance type SRIOV, I am successfully able to deploy using standard vm using virtio but for sriov instance we need to create a neutron port and then attach that port to VM. How does that process work with senlin? Currently I have the following profile: (if i want sriov instance then what i need to do?) type: os.nova.server version: 1.0 properties: name: cirros_server flavor: m1.small image: "cirros" key_name: ssh-key networks: - network: net1 metadata: test_key: test_value user_data: | #!/bin/sh echo 'hello, world' > /tmp/test_file ~S

2 2

[devstack][ussuri][keystone] ERROR: Links are not allowed as constraints
by Neil Jerram 30 Nov '20

30 Nov '20

With devstack pinned to stable/ussuri, on Ubuntu Bionic, stack.sh fails for me when it gets to the step of installing keystone: ... + lib/keystone:install_keystone:500 : setup_develop /opt/stack/keystone + inc/python:setup_develop:351 : local bindep + inc/python:setup_develop:352 : [[ /opt/stack/keystone == -bindep* ]] + inc/python:setup_develop:356 : local project_dir=/opt/stack/keystone + inc/python:setup_develop:357 : local extras= + inc/python:setup_develop:358 : _setup_package_with_constraints_edit /opt/stack/keystone -e + inc/python:_setup_package_with_constraints_edit:377 : local bindep + inc/python:_setup_package_with_constraints_edit:378 : [[ /opt/stack/keystone == -bindep* ]] + inc/python:_setup_package_with_constraints_edit:382 : local project_dir=/opt/stack/keystone + inc/python:_setup_package_with_constraints_edit:383 : local flags=-e + inc/python:_setup_package_with_constraints_edit:384 : local extras= ++ inc/python:_setup_package_with_constraints_edit:391 : cd /opt/stack/keystone ++ inc/python:_setup_package_with_constraints_edit:391 : pwd + inc/python:_setup_package_with_constraints_edit:391 : project_dir=/opt/stack/keystone + inc/python:_setup_package_with_constraints_edit:393 : '[' -n /opt/stack/requirements ']' + inc/python:_setup_package_with_constraints_edit:395 : local name ++ inc/python:_setup_package_with_constraints_edit:396 : awk '/^name.*=/ {print $3}' /opt/stack/keystone/setup.cfg + inc/python:_setup_package_with_constraints_edit:396 : name=keystone + inc/python:_setup_package_with_constraints_edit:398 : /opt/stack/requirements/.venv/bin/edit-constraints /opt/stack/requirements/upper-constraints.txt -- keystone '-e file:///opt/stack/keystone#egg=keystone' + inc/python:_setup_package_with_constraints_edit:402 : setup_package /opt/stack/keystone -e + inc/python:setup_package:430 : local bindep=0 + inc/python:setup_package:431 : local bindep_flag= + inc/python:setup_package:432 : local bindep_profiles= + inc/python:setup_package:433 : [[ /opt/stack/keystone == -bindep* ]] + inc/python:setup_package:438 : local project_dir=/opt/stack/keystone + inc/python:setup_package:439 : local flags=-e + inc/python:setup_package:440 : local extras= + inc/python:setup_package:444 : [[ -n -e ]] + inc/python:setup_package:444 : [[ -z '' ]] + inc/python:setup_package:444 : [[ ! -e =~ ^-.* ]] + inc/python:setup_package:449 : [[ ! -z '' ]] + inc/python:setup_package:454 : [[ 0 == 1 ]] + inc/python:setup_package:458 : pip_install -e /opt/stack/keystone Using python 3.6 to install /opt/stack/keystone because python3_enabled=True + inc/python:pip_install:200 : sudo -H LC_ALL=en_US.UTF-8 SETUPTOOLS_USE_DISTUTILS=stdlib http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite /usr/local/bin/pip3.6 install -c /opt/stack/requirements/upper-constraints.txt -e /opt/stack/keystone WARNING: The directory '/home/semaphore/.pip_download_cache' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement.. You can find discussion regarding this at https://github.com/pypa/pip/issues/8210. ERROR: Links are not allowed as constraints As far as I can work out, it's because install_keystone edits /opt/stack/requirements/upper-constraints.txt so that the line for keystone has "-e file:///opt/stack/keystone#egg=keystone", and pip then complains that file:// constraints are not supported. I'm sure there's a good reason but - as is often the case - I'm left wondering why this isn't broken for everyone! Any clues or suggestions gratefully received. Neil

1 0

OVS-DPDK poor performance with Intel 82599
by Satish Patel 30 Nov '20

30 Nov '20

Folks, I am playing with DPDK on my openstack with NIC model 82599 and seeing poor performance, i may be wrong with my numbers so want to see what the community thinks about these results. Compute node hardware: CPU: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz Memory: 64G NIC: Intel 82599 (dual 10G port) [root@compute-lxb-3 ~]# ovs-vswitchd --version ovs-vswitchd (Open vSwitch) 2.13.2 DPDK 19.11.3 VM dpdk (DUT): 8vCPU / 8GB memory I have configured my computer node for all best practice available on the internet to get more performance out. 1. Used isolcpus to islocate CPUs 2. 4 dedicated core for PMD 3. echo isolated_cores=1,9,25,33 >> /etc/tuned/cpu-partitioning-variables.conf 4. Huge pages 5. CPU pinning for VM 6. increase ( ovs-vsctl set interface dpdk-1 options:n_rxq=4 ) 7. VM virtio_ring = 1024 After doing all above I am getting the following result using the Trex packet generator using 64B UDP stream (Total-PPS : 391.93 Kpps) Do you think it's an acceptable result or should it be higher on these NIC models? On the internet folks say it should be a million packets per second so not sure what and how those people reached there or i am missing something in my load test profile. Notes: I am using 8vCPU core on VM do you think adding more cores will help? OR should i add more PMD? Cpu Utilization : 2.2 % 1.8 Gb/core Platform_factor : 1.0 Total-Tx : 200.67 Mbps Total-Rx : 200.67 Mbps Total-PPS : 391.93 Kpps Total-CPS : 391.89 Kcps Expected-PPS : 700.00 Kpps Expected-CPS : 700.00 Kcps Expected-BPS : 358.40 Mbps This is my all configuration: grub.conf: GRUB_CMDLINE_LINUX="vmalloc=384M crashkernel=auto rd.lvm.lv=rootvg01/lv01 console=ttyS1,118200 rhgb quiet intel_iommu=on iommu=pt spectre_v2=off nopti pti=off nospec_store_bypass_disable spec_store_bypass_disable=off l1tf=off default_hugepagesz=1GB hugepagesz=1G hugepages=60 transparent_hugepage=never selinux=0 isolcpus=2,3,4,5,6,7,10,11,12,13,14,15,26,27,28,29,30,31,34,35,36,37,38,39" [root@compute-lxb-3 ~]# ovs-appctl dpif/show netdev@ovs-netdev: hit:605860720 missed:2129 br-int: br-int 65534/3: (tap) int-br-vlan 1/none: (patch: peer=phy-br-vlan) patch-tun 2/none: (patch: peer=patch-int) vhu1d64ea7d-d9 5/6: (dpdkvhostuserclient: configured_rx_queues=8, configured_tx_queues=8, mtu=1500, requested_rx_queues=8, requested_tx_queues=8) vhu9c32faf6-ac 6/7: (dpdkvhostuserclient: configured_rx_queues=8, configured_tx_queues=8, mtu=1500, requested_rx_queues=8, requested_tx_queues=8) br-tun: br-tun 65534/4: (tap) patch-int 1/none: (patch: peer=patch-tun) vxlan-0a410071 2/5: (vxlan: egress_pkt_mark=0, key=flow, local_ip=10.65.0.114, remote_ip=10.65.0.113) br-vlan: br-vlan 65534/1: (tap) dpdk-1 2/2: (dpdk: configured_rx_queues=4, configured_rxq_descriptors=2048, configured_tx_queues=5, configured_txq_descriptors=2048, lsc_interrupt_mode=false, mtu=1500, requested_rx_queues=4, requested_rxq_descriptors=2048, requested_tx_queues=5, requested_txq_descriptors=2048, rx_csum_offload=true, tx_tso_offload=false) phy-br-vlan 1/none: (patch: peer=int-br-vlan) [root@compute-lxb-3 ~]# ovs-appctl dpif-netdev/pmd-rxq-show pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-1 queue-id: 0 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 3 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 4 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 3 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 4 (enabled) pmd usage: 0 % pmd thread numa_id 0 core_id 9: isolated : false port: dpdk-1 queue-id: 1 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 2 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 5 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 2 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 5 (enabled) pmd usage: 0 % pmd thread numa_id 0 core_id 25: isolated : false port: dpdk-1 queue-id: 3 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 0 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 7 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 0 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 7 (enabled) pmd usage: 0 % pmd thread numa_id 0 core_id 33: isolated : false port: dpdk-1 queue-id: 2 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 1 (enabled) pmd usage: 0 % port: vhu1d64ea7d-d9 queue-id: 6 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 1 (enabled) pmd usage: 0 % port: vhu9c32faf6-ac queue-id: 6 (enabled) pmd usage: 0 %

2 6

Re :[all][infra] CI test result table in the new gerrit review UI
by Bogdan Dobrelya 30 Nov '20

30 Nov '20

Hi stackers, Until we have that pretty formatter for CI results, here's [0] a quick script instead. It queries Zuul CI comments by a given Gerrit SSH login and a patch set number, and colorizes it in your console outputs. PS. I'm sorry for top posting. [0] https://gist.github.com/bogdando/ed345b1f974c8e97edd33b61e7bf8158 > On Thu, 2020-11-26 at 17:30 +0000, Tristan Cacqueray wrote: >> On Thu, Nov 26, 2020 at 15:59 Sean Mooney wrote: >> > On Thu, 2020-11-26 at 15:19 +0000, Jeremy Stanley wrote: >> > > On 2020-11-26 15:10:09 +0000 (+0000), Tristan Cacqueray wrote: >> > > [...] >> > > > Here is a CR to add the test result table back in the new gerrit UI: >> > > > >> > > > https://review.opendev.org/c/opendev/system-config/+/763891 >> > > > >> > > > This new implementation uses the javascript plugin extension API. >> > > >> > > Thanks--that was quick work! >> >> You're welcome :) >> >> > is that going to be moved to the zuul org in opendev as a part of the zuul projects >> > deliverables. >> > >> > i kind of think it makes sne to develop it there rather then on the rdo gerrit instance. >> > >> >> I'd be happy to move the project to the opendev gerrit. >> >> > looking at the impleation it does not look like this owrks teh ssame way by parsing the comment >> > https://softwarefactory-project.io/cgit/software-factory/zuul-results-gerri… >> > >> >> That is correct, this is using the Gerrit Js plugin api, which let you >> register a `showChange` callback, which is called with the change info >> https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#… >> >> Thus no need to hack through the dom to get the comments. >> >> Note that the gerrit related code is in another project here: >> https://softwarefactory-project.io/cgit/software-factory/re-gerrit/ >> >> > will this just show the results form zuul. it need to pars all the comments form the third party cis too. >> > basicaly you need to parse all coment form a author that end isn CI and comments form zuul >> >> It should do that already, though I haven't tested it with real data. I >> guess the next step would be to enable such plugin on review-dev. > so the author regular expression is only looking for zuul > https://softwarefactory-project.io/cgit/software-factory/re-gerrit/tree/src… > > so it lookes liek you would need to refactor that slightly to match comments form autors that end in ci too. > > each CI systems comemtn then need have there won tabel section with the job results for that ci to mimic the same behavior > we had in the old ui. > > >> >> Cheers, >> -Tristan -- Best regards, Bogdan Dobrelya, Irc #bogdando

1 0

XDG_SESSION_TYPE error on devstack installation
by Nitish Goel 30 Nov '20

30 Nov '20

Hi Team, I'm trying to install a devstack on a ubuntu 18.04 VM having python 3.6.9 but getting below error. Any suggestions? python - 3.6.9 stack user with sudo access. pip version - pip 20.2.4 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6) https://stackoverflow.com/questions/64989221/xdg-session-type-error-on-devs… NFO keystone.cmd.bootstrap [None req-f46e9c41-1b14-4e5b-82b2-ada81e8b0dcd None None] Created region RegionOne INFO keystone.cmd.bootstrap [None req-f46e9c41-1b14-4e5b-82b2-ada81e8b0dcd None None] Created public endpoint http://10.61.62.241/identity INFO keystone.cmd.bootstrap [None req-f46e9c41-1b14-4e5b-82b2-ada81e8b0dcd None None] Created admin endpoint http://10.61.62.241/identity +./stack.sh:main:1084 create_keystone_accounts +lib/keystone:create_keystone_accounts:314 local admin_project ++lib/keystone:create_keystone_accounts:315 oscwrap project show admin -f value -c id Traceback (most recent call last): File "/usr/local/bin/openstack", line 5, in <module> from openstackclient.shell import main File "/usr/local/lib/python3.6/dist-packages/openstackclient/shell.py", line 24, in <module> from osc_lib import shell File "/usr/local/lib/python3.6/dist-packages/osc_lib/shell.py", line 24, in <module> from cliff import app File "/usr/local/lib/python3.6/dist-packages/cliff/app.py", line 24, in <module> import cmd2 File "/usr/local/lib/python3.6/dist-packages/cmd2/__init__.py", line 30, in <module> from .cmd2 import Cmd File "/usr/local/lib/python3.6/dist-packages/cmd2/cmd2.py", line 48, in <module> from .clipboard import can_clip, get_paste_buffer, write_to_paste_buffer File "/usr/local/lib/python3.6/dist-packages/cmd2/clipboard.py", line 12, in <module> _ = pyperclip.paste() File "/usr/local/lib/python3.6/dist-packages/pyperclip/__init__.py", line 680, in lazy_load_stub_paste copy, paste = determine_clipboard() File "/usr/local/lib/python3.6/dist-packages/pyperclip/__init__.py", line 568, in determine_clipboard os.environ["XDG_SESSION_TYPE"] == "wayland" and File "/usr/lib/python3.6/os.py", line 669, in __getitem__ raise KeyError(key) from None KeyError: 'XDG_SESSION_TYPE' ++functions-common:oscwrap:2346 return 1 +lib/keystone:create_keystone_accounts:315 admin_project= +lib/keystone:create_keystone_accounts:1 exit_trap +./stack.sh:exit_trap:491 local r=1 ++./stack.sh:exit_trap:492 jobs -p +./stack.sh:exit_trap:492 jobs= +./stack.sh:exit_trap:495 [[ -n '' ]] +./stack.sh:exit_trap:501 '[' -f /tmp/tmp.LRWsRkTTkV ']' +./stack.sh:exit_trap:502 rm /tmp/tmp.LRWsRkTTkV +./stack.sh:exit_trap:506 kill_spinner +./stack.sh:kill_spinner:401 '[' '!' -z '' ']' +./stack.sh:exit_trap:508 [[ 1 -ne 0 ]] +./stack.sh:exit_trap:509 echo 'Error on exit' Error on exit +./stack.sh:exit_trap:511 type -p generate-subunit +./stack.sh:exit_trap:512 generate-subunit 1606228299 592 fail +./stack.sh:exit_trap:514 [[ -z /opt/stack/logs ]] +./stack.sh:exit_trap:517 /usr/bin/python3.6 /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs +./stack.sh:exit_trap:526 exit 1

3 6

Re: [designate] DKIM TXT record problem
by pawel＠dpnet.pl 30 Nov '20

30 Nov '20

I have the same problem, would you like to share a command line example, how did you manage to add such a long txt record in the designate (more than 255 characters in one record)? -- Pawel D

1 0

[ironic] [tripleo] RFC: lzma vs gzip for compressing IPA initramfs
by Dmitry Tantsur 30 Nov '20

30 Nov '20

Hi folks, I've been playing with ways to reduce the size of our IPA images. While package removals can only save us tens of megabytes, switching from gzip to lzma reduces the size by around a third (from 373M to 217M in my testing). What's the caveat? The unpacking time increases VERY substantially. On my nested virt lab the 217M image took around 5 minutes to unpack. I'm not sure how much it will impact real bare metal, please feel free to test https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/764371 and tell me. So, what do you think? Switching to lzma by default will likely affect CI run time (assuming we still have DIB jobs somewhere...) and development environments, but it will also provide a visible reduction in the image size (which benefit all environments). Large TripleO images may particularly benefit from this (but also particularly affected by the unpacking time). Feedback is very welcome. Dmitry -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

7 9

[openstack][neutron]How to establish existing Openstack in a single server to other multiple physical NIC port
by dangerzone ar 29 Nov '20

29 Nov '20

Hi, I have deployed packstack (All-In-One) openstack over single server and that openstack traffic is IN/Out via eth1 of the server and that's the network port configured during the packstack deployment. The server has 4 nic ports and only one (nic1) is used during openstack deployment. The openstack running good and my vm/vnf traffic can in/out via eth1. I would like to know how I can utilize other nic port (2,3,4) in my server so that openstack can leverage that port for my other VM/VNF that instantiate in my openstack. For example from the image attached, I want VM/VNF1 connect to nic2 and VM/VNF2 connect to nic3. Please advise how it can be done. I really hope someone could advise and help me further. Thank you. **This is standard openstack (deployed via packstack all-in-one) over single centos7 server that have 4 nic ports**

2 1