December 2020 - openstack-discuss

[tripleo] CI jobs failures on RDO project
by Daniel Pawlik 07 Dec '20

07 Dec '20

Hello, We are facing an outage in our cloud providers. That's why from time to time you can have problems validating CI jobs in RDO project. We are doing our best to fix the issues as soon as possible. Kindly regards, Dan

1 0

[networking-ovs-dpdk]
by Elanchezian Settu 07 Dec '20

07 Dec '20

Hi, Tried ovs-dpdk installation in below setup and noticing it fails with OVS-DPDK initialization timeout problem. Herewith I had attached the local.conf file, which I had used for my setup bring up. Server: Ubuntu 18.04 latest OpenStack: Stein OVS-DPDK: Master byte-compiling /usr/local/lib/python3.6/dist-packages/networking_ovs_dpdk/agent/ovs_dpdk_firewall.py to ovs_dpdk_firewall.cpython-36.pyc running install_egg_info Copying networking_ovs_dpdk.egg-info to /usr/local/lib/python3.6/dist-packages/networking_ovs_dpdk-2015.1.1.dev275-py3.6.egg-info running install_scripts /usr/local/lib/python3.6/dist-packages/pbr/packaging.py:410: EasyInstallDeprecationWarning: Use get_header header = easy_install.get_script_header("", executable, is_wininst) ++/opt/stack/networking-ovs-dpdk/devstack/plugin.sh:source:35 popd /opt/stack/ovs ++/opt/stack/networking-ovs-dpdk/devstack/plugin.sh:source:36 start_ovs_dpdk ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:start_ovs_dpdk:19 '[' -e /etc/init.d/ovs-dpdk ']' ++/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:start_ovs_dpdk:20 sudo service ovs-dpdk start Job for ovs-dpdk.service failed because a timeout was exceeded. See "systemctl status ovs-dpdk.service" and "journalctl -xe" for details. +/opt/stack/networking-ovs-dpdk/devstack/libs/ovs-dpdk:start_ovs_dpdk:1 exit_trap +./stack.sh:exit_trap:528 local r=1 ++./stack.sh:exit_trap:529 jobs -p +./stack.sh:exit_trap:529 jobs= +./stack.sh:exit_trap:532 [[ -n '' ]] +./stack.sh:exit_trap:538 '[' -f '' ']' +./stack.sh:exit_trap:543 kill_spinner +./stack.sh:kill_spinner:438 '[' '!' -z '' ']' +./stack.sh:exit_trap:545 [[ 1 -ne 0 ]] +./stack.sh:exit_trap:546 echo 'Error on exit' Error on exit +./stack.sh:exit_trap:548 type -p generate-subunit +./stack.sh:exit_trap:549 generate-subunit 1607341696 1042 fail +./stack.sh:exit_trap:551 [[ -z /opt/stack/logs/screen ]] +./stack.sh:exit_trap:554 /home/ubuntu/devstack/tools/worlddump.py -d /opt/stack/logs/screen Traceback (most recent call last): File "/home/ubuntu/devstack/tools/worlddump.py", line 255, in <module> sys.exit(main()) File "/home/ubuntu/devstack/tools/worlddump.py", line 239, in main sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0) File "/usr/lib/python3.6/os.py", line 1017, in fdopen return io.open(fd, *args, **kwargs) ValueError: can't have unbuffered text I/O Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'> OSError: [Errno 9] Bad file descriptor ubuntu@ubuntu:~/devstack$ ubuntu@ubuntu:~/devstack$ systemctl status ovs-dpdk.service ● ovs-dpdk.service - LSB: Open vSwitch DPDK Loaded: loaded (/etc/init.d/ovs-dpdk; generated) Active: failed (Result: timeout) since Mon 2020-12-07 12:05:38 UTC; 15s ago Docs: man:systemd-sysv-generator(8) Process: 21377 ExecStart=/etc/init.d/ovs-dpdk start (code=killed, signal=TERM) Tasks: 2 (limit: 9830) CGroup: /system.slice/ovs-dpdk.service ├─21984 sudo ovs-vsctl --may-exist add-port br-enp24s0f1 enp24s0f1 -- set Interface enp24s0f1 type=dpdk options:dpdk-devargs=0000:18:00.1 other_config:pci_address=0000:18:00.1 other_config:driver=igb_uio other_config:p └─21985 ovs-vsctl --may-exist add-port br-enp24s0f1 enp24s0f1 -- set Interface enp24s0f1 type=dpdk options:dpdk-devargs=0000:18:00.1 other_config:pci_address=0000:18:00.1 other_config:driver=igb_uio other_config:previo Dec 07 12:00:41 ubuntu sudo[21775]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/opt/stack/DPDK-v18.11/usertools/dpdk-devbind.py -b igb_uio 0000:5e:00.1 Dec 07 12:00:41 ubuntu sudo[21775]: pam_unix(sudo:session): session opened for user root by (uid=0) Dec 07 12:00:42 ubuntu sudo[21775]: pam_unix(sudo:session): session closed for user root Dec 07 12:00:42 ubuntu ovs-dpdk[21377]: add-port command: ovs-vsctl --may-exist add-port br-enp24s0f1 enp24s0f1 -- set Interface enp24s0f1 type=dpdk options:dpdk-devargs=0000:18:00.1 other_config:pci_address=0000:18:00.1 Dec 07 12:00:42 ubuntu sudo[21984]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/ovs-vsctl --may-exist add-port br-enp24s0f1 enp24s0f1 -- set Interface enp24s0f1 type=dpdk options:dpdk-devargs=0000:18:00.1 other_ Dec 07 12:00:42 ubuntu sudo[21984]: pam_unix(sudo:session): session opened for user root by (uid=0) Dec 07 12:00:42 ubuntu ovs-vsctl[21985]: ovs|00001|vsctl|INFO|Called as ovs-vsctl --may-exist add-port br-enp24s0f1 enp24s0f1 -- set Interface enp24s0f1 type=dpdk options:dpdk-devargs=0000:18:00.1 other_config:pci_address=0000:18 Dec 07 12:05:38 ubuntu systemd[1]: ovs-dpdk.service: Start operation timed out. Terminating. Dec 07 12:05:38 ubuntu systemd[1]: ovs-dpdk.service: Failed with result 'timeout'. Dec 07 12:05:38 ubuntu systemd[1]: Failed to start LSB: Open vSwitch DPDK. ubuntu@ubuntu:~/devstack$ Thanks & Regards, Elan This message may contain confidential and privileged information. If it has been sent to you in error, please reply to advise the sender of the error and then immediately delete it. If you are not the intended recipient, do not read, copy, disclose or otherwise use this message. The sender disclaims any liability for such unauthorized use. NOTE that all incoming emails sent to Gigamon email accounts will be archived and may be scanned by us and/or by external service providers to detect and prevent threats to our systems, investigate illegal or inappropriate behavior, and/or eliminate unsolicited promotional emails (“spam”).

1 0

[neutron] Bug deputy report (week starting on 2020-11-30)
by Bernard Cafarelli 07 Dec '20

07 Dec '20

Hi neutrinos, time to start last bug deputy rotation for 2020, here are the bugs reported in week 49. A pretty normal range of bugs from CI fixes to scaling enhancements, most in progress. Two are unassigned right now: OVN tempest failure ( https://bugs.launchpad.net/neutron/+bug/1906490) and DHCP agent leases fix with segments (https://bugs.launchpad.net/neutron/+bug/1906406) New RFE: * [RFE] add new extension "device-profile" for port - https://bugs.launchpad.net/neutron/+bug/1906602 New extension for device_profile notion of Cyborg project - already discussed and approved Critical * oom killer kills mysqld process on the node running fullstack tests - https://bugs.launchpad.net/neutron/+bug/1906366 Patch to limit resources used in FT tests: https://review.opendev.org/c/openstack/neutron/+/764907 * SSH failures in the neutron-ovn-tempest-ovs-release-ipv6-only job - https://bugs.launchpad.net/neutron/+bug/1906490 Unassigned * neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_dhcp_port_status_active is failing often - https://bugs.launchpad.net/neutron/+bug/1906654 https://review.opendev.org/c/openstack/neutron/+/755313 may help here, to confirm High * [ovn] Tempest tests failing while creating security group driver with KeyError: 'remote_address_group_id' - https://bugs.launchpad.net/neutron/+bug/1906500 Broke after a recent merge (and not tested in gates), fix at https://review.opendev.org/c/openstack/neutron/+/765101 * [OVN Octavia Provider] OVN provider not setting member offline correctly on create when admin_state_up=False - https://bugs.launchpad.net/neutron/+bug/1906568 Fix for tempest test in https://review.opendev.org/c/openstack/ovn-octavia-provider/+/765213 Medium * [segments] dnsmasq can't delete lease for instance due to mismatch between client ip and local addr - https://bugs.launchpad.net/neutron/+bug/1906406 With routed provider networks, DHCP agent at startup receives all leases, not only those from its segment Unassigned * MTU for networks with enabled vlan transparency should be 4 bytes lower - https://bugs.launchpad.net/neutron/+bug/1906318 Assigned to slaweq * [L2][scale] add a trunk size config option for bundle flow install - https://bugs.launchpad.net/neutron/+bug/1906487 New option to help with large set of flows installation: https://review.opendev.org/c/openstack/neutron/+/765072 Low * dns-integration api extension shouldn't be enabled by ovn_l3 plugin if there is no corresponding ML2 extension driver enabled - https://bugs.launchpad.net/neutron/+bug/1906311 Patch https://review.opendev.org/c/openstack/neutron/+/764831 * [L3] router HA port concurrently deleting - https://bugs.launchpad.net/neutron/+bug/1906375 Happened on rocky but code did not change since, patch in progress https://review.opendev.org/c/openstack/neutron/+/764913 * [L2] current binding port get type errors - https://bugs.launchpad.net/neutron/+bug/1906381 Patch in progress at https://review.opendev.org/c/openstack/neutron/+/764917 * Wrong error message when removing interface from router - https://bugs.launchpad.net/neutron/+bug/1906508 Suggested fix https://review.opendev.org/c/openstack/neutron/+/765129 -- Bernard Cafarelli

1 0

[security] Auto-renewing trusted user certificates for Openstack services using the ACME protocol
by Stanislav Dimov 07 Dec '20

07 Dec '20

Hello all, I have recently started exploring Openstack with the goal of using it to replace my current private cloud infrastructure. I have been reading the docs about security and I noticed that there isn't really a (straight forward) way of securing Openstack services communication with user provided, trusted, SSL certificates. I believe this should not be the case. My current infrastructure uses a privately hosted CA, that supports the ACME protocol. All my hosts submit CSRs to it, and respond to the ACME challenges in order to get it signed. All certificates are short-lived (1h), but never expire thanks to the ACME automation. I have achieved this through an open source project called Smallstep Step CA and Smallstep Step CLI tools. It is dead easy to set up. All of the tools needed to achieve this can also be containerized, for simplicity. Thus, I propose the following solution (keep in mind I am not an Openstack developer): Addition of an ACME client, with a configurable ACME URL, to all (or as many as possible) Openstack services, that can submit CSRs to an ACME server (basically almost identical to the already implemented Openstack Let's Encrypt functionality for public endpoints). Also, optionally, the creation of a new Openstack service, using the Smallstep Step CA, which can sign the CSRs, and thus eliminate the need for a manual setup of a separate Smallstep CA host. I am providing some links to the Smallstep repositories and documentation for easier access: https://github.com/smallstep/certificates https://github.com/smallstep/cli https://github.com/smallstep/hello-mtls https://smallstep.com/docs/ Thank you for your time and consideration. Kind regards, Stanislav

1 0

octavia flavor question
by Satish Patel 07 Dec '20

07 Dec '20

Folks, I am deployed Octavia and now trying to understand how does flavor play there role using this guide https://docs.openstack.org/octavia/latest/admin/flavors.html In my octavia.conf i have statically defined nova flavor so when i spin up LB it always use m1.amphora (UUID: 8651951f-e478-4efb-b298-cf3150d38472) which has 1vCPU/2GB memory. # grep amp_flavor_id /etc/octavia/octavia.conf amp_flavor_id = 8651951f-e478-4efb-b298-cf3150d38472 Question: how do I create or tell octavia to use multiple nova flavors for example, I want 3 flavors m1.amphora - 1vCPU/2GB/20GB disk m2.amphora - 4vCPU/4GB/20GB disk m3.amphora - 8vCPU/8GB/40GB disk (with HugePage and CPUpinning support) I want my users to have the option to select whatever flavor they like. how does that fit here? I didn't find any document explaining that feature to select your desired nova flavor. Did I miss something?

3 3

[tripleo][ussuri] overcloud deploy increase timeout/wai time to load initramfs to deploy os
by Ruslanas Gžibovskis 06 Dec '20

06 Dec '20

Hi all, To deploy initramfs takes longer time (around 17 min). And while it is loading, overcloud deploy sais: ' Went to status ERROR due to "Message: No valid host was found. , Code: 500" '. the remote compute is in a different continent, so speed is quite slow. is it possible to increase wait time? Thank you. Regards. -- Ruslanas Gžibovskis +370 6030 7030

1 0

[rdo][centos8][repos] CentOS8 RDO-USSURI repo missing openvswitch2.13
by Ruslanas Gžibovskis 06 Dec '20

06 Dec '20

Hi all, I have an issue, in CentOS8 repos for RDO. It is refering to openvswitch2.13 name, but in the repos I can see openvswitch packages with version in version field, looks like in dependency field it is missing dash (-) or smth... here is a paste of repos and error message [0]. Or any other way how this could be fixed? thanks. and packages I can see [1] here. [0] http://paste.openstack.org/show/xEalilL1rRJSDTzNCGt1/ [1] http://paste.openstack.org/show/VLWrNFpRLatwq5qXP2kD/ -- Ruslanas Gžibovskis +370 6030 7030

3 3

[tripleo] TripleO CI Summary: Unified Sprint 36
by Rafael Folco 05 Dec '20

05 Dec '20

Greetings, The TripleO CI team has just completed **Unified Sprint 36** (Nov 13 thru Dec 03). The following is a summary of completed work during this sprint cycle: - Added Tempest cleanup support: https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/76240… - Ported jobs running against tripleo-ansible stable/train to centos-8 - ansible-role-collect-logs role is packaged now and available for train onwards - All the jobs are migrated to vexxhost and rdo-cloud nodeset is - removed - Continued merging changes to wire-up content provider jobs across TripleO related projects to remove docker.io dependencies: https://review.opendev.org/#/q/topic:new-ci-job - Designed a dependency pipeline to early detect breakages in the base OS: https://hackmd.io/14KFQiyWSBCRsfJmBZNARw?both - component standalone-upgrade - Enable the component-ci-testing repo if it is installed: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/764767 - https://review.rdoproject.org/r/#/c/31200/ Adds tripleo component standalone-upgrade-master/victoria/ussuri - Redundant content providers for the upgrade jobs to reduce resource usage: https://review.opendev.org/761188 - Tempest scenario manager improvements: 1. https://review.opendev.org/c/openstack/tempest/+/753689 2. https://review.opendev.org/c/openstack/tempest/+/754081 3. https://review.opendev.org/c/openstack/tempest/+/755072 - Promoter configuration fix: https://review.rdoproject.org/r/#/c/28014/ - Created a new job in downstream for 16.2 component pipeline which deploys multiple overcloud stack using a single undercloud node: - https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/763786 - https://review.opendev.org/c/openstack/tripleo-quickstart/+/761061 - Added OSP 13 ansible openstack collection tests job - Wrote test cases for cockpit (telegraf container bump to python3): https://review.rdoproject.org/r/#/c/30492/ - Removed ubi-8 based jobs: https://review.opendev.org/760426 - Ansible-collections-openstack improvements: 1. ttps://review.opendev.org/c/openstack/ansible-collections-openstack/+/764379 <https://review.opendev.org/c/openstack/ansible-collections-openstack/+/7643…> 2. https://review.opendev.org/c/openstack/ansible-collections-openstack/+/7643… 3. https://review.opendev.org/c/openstack/ansible-collections-openstack/+/7644… 4. https://review.opendev.org/c/openstack/ansible-collections-openstack/+/7640… Upstream promotions [1] Ruck/Rover recorded notes [2]. NEXT SPRINT =========== The planned work for the next sprint is to continue work started in the previous sprint: The Ruck and Rover shifts for the upcoming weeks are scheduled as follows: Dec 7: Sorin Sbarnea (zbr), Marios Andreou (marios) Dec 14: Chandan Kumar (chandankumar), Amol Kahat (akahat) Dec 21: Bhagyashri Shewale (bhagyashris), Soniya Vyas (soniya29) Dec 28: Sandeep Yadav (ysandeep), Sagi Shnaidman (sshnaidm) Jan 4: Wesley Hayutin (weshay), Poja Jadhav (pojadhav) Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes to be tracked in hackmd [2]. Thanks, --rfolco [1] http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?org… [2] https://hackmd.io/R0kCgz_7SHSix_cNgoC9pg -- Folco

1 0

[Diversity] Diversity & Inclusion Meeting 12/7
by Amy Marrich 05 Dec '20

05 Dec '20

The Diversity & Inclusion WG invites members of all OSF projects to our next meeting Monday, December 7th, at 17:00 UTC in the #openstack-diversity channel. The agenda can be found at https://etherpad.openstack.org/p/diversity -wg-agenda. Please feel free to add any other topics you wish to discuss at the meeting. Thanks, Amy (spotz)

1 0

[baremetal-sig][ironic] Tue Dec 8, 2pm UTC: Ironic and Redfish Interop Profiles
by Arne Wiebalck 05 Dec '20

05 Dec '20

Dear all, The Bare Metal SIG will meet on Tue Dec 8 at 2pm UTC. This time there will be a 10 minute "topic-of-the-day" presentation by Richard Pioso: 'An Introduction to Ironic Redfish Interoperability Profiles' So, if you would like to learn how to validate if hardware is fit for use with Redfish and Ironic, find all the details for this meeting on the SIG's etherpad: https://etherpad.opendev.org/p/bare-metal-sig Everyone is welcome, don't miss out! Cheers, Arne

2 1