May 2020 - openstack-discuss - lists.openstack.org

Ussuri RDO Release Announcement
by Amy Marrich 28 May '20

28 May '20

If you're having trouble with the formatting, this release announcement is available online https://blogs.rdoproject.org/2020/05/rdo-ussuri-released/ --- *RDO Ussuri Released* The RDO community is pleased to announce the general availability of the RDO build for OpenStack Ussuri for RPM-based distributions, CentOS Linux and Red Hat Enterprise Linux. RDO is suitable for building private, public, and hybrid clouds. Ussuri is the 21st release from the OpenStack project, which is the work of more than 1,000 contributors from around the world. The release is already available on the CentOS mirror network at http://mirror.centos.org/centos/8/cloud/x86_64/openstack-ussuri/. The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Linux and is a member of the CentOS Cloud Infrastructure SIG. The Cloud Infrastructure SIG focuses on delivering a great user experience for CentOS Linux users looking to build and maintain their own on-premise, public or hybrid clouds. All work on RDO and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first. PLEASE NOTE: At this time, RDO Ussuri provides packages for CentOS8 only. Please use the previous release, Train, for CentOS7 and python 2.7. *Interesting things in the Ussuri release include:* - Within the Ironic project, a bare metal service that is capable of managing and provisioning physical machines in a security-aware and fault-tolerant manner, UEFI and device selection is now available for Software RAID. - The Kolla project, the containerised deployment of OpenStack used to provide production-ready containers and deployment tools for operating OpenStack clouds, streamlined the configuration of external [Ceph]( https://ceph.io/) integration, making it easy to go from Ceph-Ansible-deployed Ceph cluster to enabling it in OpenStack. *Other improvements include:* - Support for IPv6 is available within the Kuryr project, the bridge between container framework networking models and OpenStack networking abstractions. - Other highlights of the broader upstream OpenStack project may be read via https://releases.openstack.org/ussuri/highlights.html. - A new Neutron driver networking-omnipath has been included in RDO distribution which enables the Omni-Path switching fabric in OpenStack cloud. - OVN Neutron driver has been merged in main neutron repositon from networking-ovn. *Contributors* During the Ussuri cycle, we saw the following new RDO contributors: - Amol Kahat - Artom Lifshitz - Bhagyashri Shewale - Brian Haley - Dan Pawlik - Dmitry Tantsur - Dougal Matthews - Eyal - Harald Jensås - Kevin Carter - Lance Albertson - Martin Schuppert - Mathieu Bultel - Matthias Runge - Miguel Garcia - Riccardo Pittau - Sagi Shnaidman - Sandeep Yadav - SurajP - Toure Dunnon Welcome to all of you and Thank You So Much for participating! But we wouldn’t want to overlook anyone. A super massive Thank You to all 54 contributors who participated in producing this release. This list includes commits to rdo-packages and rdo-infra repositories: - Adam Kimball - Alan Bishop - Alan Pevec - Alex Schultz - Alfredo Moralejo - Amol Kahat - Artom Lifshitz - Arx Cruz - Bhagyashri Shewale - Brian Haley - Cédric Jeanneret - Chandan Kumar - Dan Pawlik - David Moreau Simard - Dmitry Tantsur - Dougal Matthews - Emilien Macchi - Eric Harney - Eyal - Fabien Boucher - Gabriele Cerami - Gael Chamoulaud - Giulio Fidente - Harald Jensås - Jakub Libosvar - Javier Peña - Joel Capitao - Jon Schlueter - Kevin Carter - Lance Albertson - Lee Yarwood - Marc Dequènes (Duck) - Marios Andreou - Martin Mágr - Martin Schuppert - Mathieu Bultel - Matthias Runge - Miguel Garcia - Mike Turek - Nicolas Hicher - Rafael Folco - Riccardo Pittau - Ronelle Landy - Sagi Shnaidman - Sandeep Yadav - Soniya Vyas - Sorin Sbarnea - SurajP - Toure Dunnon - Tristan de Cacqueray - Victoria Martinez de la Cruz - Wes Hayutin - Yatin Karel - Zoltan Caplovic *The Next Release Cycle* At the end of one release, focus shifts immediately to the next, Victoria, which has an estimated GA the week of 12-16 October 2020. The full schedule is available at https://releases.openstack.org/victoria/schedule.html. Twice during each release cycle, RDO hosts official Test Days shortly after the first and third milestones; therefore, the upcoming test days are 25-26 June 2020 for Milestone One and 17-18 September 2020 for Milestone Three. *Get Started* There are three ways to get started with RDO. To spin up a proof of concept cloud, quickly, and on limited hardware, try an All-In-One Packstack installation. You can run RDO on a single node to get a feel for how it works. For a production deployment of RDO, use the TripleO Quickstart and you’ll be running a production cloud in short order. Finally, for those that don’t have any hardware or physical resources, there’s the OpenStack Global Passport Program. This is a collaborative effort between OpenStack public cloud providers to let you experience the freedom, performance and interoperability of open source infrastructure. You can quickly and easily gain access to OpenStack infrastructure via trial programs from participating OpenStack public cloud providers around the world. *Get Help* The RDO Project participates in a Q&A service at https://ask.openstack.org. We also have our users(a)lists.rdoproject.org for RDO-specific users and operrators. For more developer-oriented content we recommend joining the dev(a)lists.rdoproject.org mailing list. Remember to post a brief introduction about yourself and your RDO story. The mailing lists archives are all available athttps://mail.rdoproject.org. You can also find extensive documentation on RDOproject.org. The #rdo channel on Freenode IRC is also an excellent place to find and give help. We also welcome comments and requests on the CentOS devel mailing list and the CentOS and TripleO IRC channels (#centos, #centos-devel, and #tripleo on irc.freenode.net) however we have a more focused audience within the RDO venues. *Get Involved* To get involved in the OpenStack RPM packaging effort, check out the RDO contribute pages, peruse the CentOS Cloud SIG page, and inhale the RDO packaging documentation. Join us in #rdo and #tripleo on the Freenode IRC network and follow us on Twitter @RDOCommunity. You can also find us on Facebook and YouTube. Amy Marrich (spotz) https://www.rdoproject.org http://community.redhat.com

1 0

[tc] Monthly update (to become Weekly)
by Mohammed Naser 28 May '20

28 May '20

Hello everyone, Here’s an update of what’s happened in the OpenStack TC during the last month. Moving forward, I’ll be sending an update on a weekly basis. You can get more information by checking for changes in openstack/governance repository. # CHANGES PENDING - Rename ansible-role-lunasa-hsm deliverable: https://review.opendev.org/#/c/731313/ - Add whitebox-tempest-plugin under QA: https://review.opendev.org/#/c/714480/ - Select migrate-to-focal goal for Victoria cycle: https://review.opendev.org/#/c/731213/ - Clarify the support for linux distro: https://review.opendev.org/#/c/727238/ - Add QA branchless projects also in py3.5 support list: https://review.opendev.org/#/c/729325/ - Remove congress and tricircle project teams: https://review.opendev.org/#/c/728818/ - [draft] Add assert:supports-standalone: https://review.opendev.org/#/c/722399/ # PROJECT UPDATES - Adding Storlets in the list of keeping py2: https://review.opendev.org/#/c/720714/ - Transfer Policy team co-lead role: https://review.opendev.org/#/c/722101/ - Add keystoneauth to the python3.5 list: https://review.opendev.org/#/c/721093/ - Add note that OpenStackSDK kept 3.5 support: https://review.opendev.org/#/c/721083/ - Retire i18n-specs: https://review.opendev.org/#/c/721723/ - Migrate from mock to built in unittest.mock: https://review.opendev.org/#/c/722924/ - Migrate from olso.rootwrap to oslo.privsep: https://review.opendev.org/#/c/718177/ - Appoint Yasufumi Ogawa as Tacker PTL: https://review.opendev.org/#/c/719091/ # GOAL UPDATES - Infrastructure project team is now the TaCT SIG: https://review.opendev.org/#/c/721621/ - Add ansible role for managing Luna SA HSM: https://review.opendev.org/#/c/721348/ - Drop deprecated release-management flag for xstatic repos: https://review.opendev.org/#/c/721046/ # ABANDONED CHANGES - Remove CentOS 8 from tested runtimes for Ussuri: https://review.opendev.org/#/c/720005/ - Goals: add container-images: https://review.opendev.org/#/c/720107/ # GENERAL UPDATES - Add goal to migrate CI/CD jobs to Ubuntu Focal: https://review.opendev.org/#/c/728158/ - Fix hacking min version to 3.0.1: https://review.opendev.org/#/c/730161/ - Appoint Javier Peña as rpm-packaging PTL: https://review.opendev.org/#/c/728088/ - Add repository for oslo.metrics: https://review.opendev.org/#/c/725848/ - Require PTL signoff for project-update changes: https://review.opendev.org/#/c/727704/ - Allow for faster addition of projects: https://review.opendev.org/#/c/726932/ - Retire syntribos: https://review.opendev.org/#/c/726507/ - Loosen voting for community goal proposals: https://review.opendev.org/#/c/724142/ - Add TrilioVault Charms to OpenStack Charms: https://review.opendev.org/#/c/720533/ - Cleanup retired repositories: https://review.opendev.org/#/c/724334/ - Move i18n into a SIG: https://review.opendev.org/#/c/721605/ # OTHER - Cap jsonschema 3.2.0 as the minimal version: https://review.opendev.org/#/c/730948/ - Fix bullet list formatting for drop-py2 goal: https://review.opendev.org/#/c/729314/ Thanks and sorry for the delay in getting those out. -- Mohammed Naser VEXXHOST, Inc.

1 0

Re: Uniting TC and UC
by Mohamed Elsakhawy 28 May '20

28 May '20

Hi All, As you may know already, there has been an ongoing discussion to merge UC and TC under a single body. Three options were presented, along with their impact on the bylaws. http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012806… We had several discussions in the UC on the three options as well as the UC items that need to be well-represented under the common committee, and here’s what we propose for the merger: 1- There’s consensus on utilizing option 1, thus requiring no bylaws change and merging AUC and ATC : 1- No bylaws change As bylaws changes take a lot of time and energy, the simplest approach would be to merge the TC and UC without changing the bylaws at all. The single body (called TC) would incorporate the AUC criteria by adding all AUC members as extra-ATC. It would tackle all aspects of our community. To respect the letter of the bylaws, the TC would formally designate 5 of its members to be the 'UC' and those would select a 'UC chair'. But all tasks would be handled together. 2- Two main core UC responsibilities need to continue to be represented under TC - Representing the user community to the broader community (Board of directors, TC...) - Working with user groups worldwide to keep the community vibrant and informed. 3- The current active workgroups under UC’s governance will be moved to be under TC’s governance. 4- AUC and ATC to be merged into a single list, potentially called AC (Active Contributor) 5- We hope to have the merger finalized before the upcoming UC elections in September 2020. In addition to discussions over the mailing list, we also have the opportunity of "face-to-face" discussions at the upcoming PTG. Thanks Mohamed --melsakhawy

8 11

[blazar][ptg] IRC meetings cancelled due to PTG
by Pierre Riteau 28 May '20

28 May '20

Hello, The Blazar community will meet several times through the PTG event next week. Let's cancel the IRC meetings originally scheduled on June 2 and June 4. Talk to you soon, Pierre Riteau (priteau)

1 0

[glance] "true is not of type boolean" when creating protected image
by Bernd Bausch 28 May '20

28 May '20

This problem occurs on a stable Ussuri Devstack. I issue this command to create a protected image: $ openstack image create myimage2 --file ~/myvm1.qcow2 --disk-format qcow2 --protected and get this error message back: BadRequestException: 400: Client Error for url: http://192.168.1.200/image/v2/images, 400 Bad Request: On instance['protected']:: 'type': 'boolean'}: {'description': 'If true, image will not be deletable.',: 'True': Provided object does not match schema 'image': 'True' is not of type 'boolean': Failed validating 'type' in schema['properties']['protected']: It seems that image creation fails because 'True' is not of type 'boolean' Am I looking at a bug or am I doing something wrong? I do have a great workaround: First, create the image without the protected flag. Then: $ openstack image set --protected myimage2 This works.

2 1

[all] [kolla] Kolla Builder - next generation
by Radosław Piliszek 28 May '20

28 May '20

Hello fellow OpenStackers and Kollars (or Koalas) in particular, today is the day when I finally sit down to write up my thoughts on Kolla, and I mean *Kolla*, the container image building project for OpenStack and friends, nothing more (nor less). *** Some background on Kolla to get everyone on the same page (or close enough): Kolla builds images for production use. Kolla is upstream for TripleO containers and Kolla-Ansible deployment. These are container images, think OCI and Docker in particular since Kolla actually relies on Dockerfile format to specify build recipes and Docker runs building (TripleO runs buildah on the same recipes). Kolla supports three distributions as bases: CentOS (TripleO does override this for RHEL as well), Debian and Ubuntu. In the interim periods Kolla supports two distribution releases to ease/smooth the transition process for operators (like currently CentOS 7 and 8 in Train, while Ussuri is 8 only). What is more, each distro has two flavours (or 'types' as they are called for now): binary and source. This 'type' applies only to OpenStack software. Binary means that Kolla uses downstream packages (rpm/deb) while source means to use pip and install from official tarballs (or repos if it is master) and PyPI for deps, utilising venv to keep it separate from distro stuff. Finally, there is particular target architecture: x86_64, aarch64 and ppc64le. All of above affect the support matrix which is based on 3 or 4 dimensions (distro, flavour, arch and sometimes distro version). [1] Kolla offers a high level of customisability via various overrides levels. See docs for details [2] Kolla engine offers hierarchical approach to image building, under the assumption that more than one image is often deployed on the same machine so layer sharing is beneficial. Kolla helps collect sources for OpenStack projects to build desired versions of them. All recipes are templated using Jinja2 syntax. Images contain both run-time and, mostly in the case of 'source' flavour, build-time tools and libraries. *** That's it for the background. :-) So where is the problem you might ask? Oh, there are plenty. The general is that Kolla has lots of logic in templates. Thankfully, Kolla has macros for most stuff but still Jinja2 limitations make it hard to document exceptions (no inline comments in arrays anyone?). We lose visibility into real dependency graphs and may be easily reinstalling same stuff, any optimisations are limited and would require parsing of both Jinja2 and Dockerfile syntax. In general the current approach is ugly and becomes unwieldy (e.g. getting warnings about empty continuation lines). There is layering hell: dependency on long &&ed commands which are templated out to avoid useless layers. Support matrix is great for an overview. There is a hidden layer to it though, different combinations may support different features. This is not documented so far and not so easy to follow from sources. This stems from the fact that different extra components have different availability in distros, but this is not easily apparent from sources. Also, there is no way to turn feature on/off. You might do an override, but then again, which to keep, which not to? Which is which? Oh! Finally, the images tend to be heavy due to inclusion of build-time deps. *** That's it for issues (hopefully!) My idea is to reuse Kolla engine where it shines: sources collection, system of plugins, hierarchical building; but replace the part that smells - Jinja2 templating. Giving up on Jinja2 might encompass giving up on Dockerfile syntax, but that is optional and depends on what makes it more amenable to avoid further pitfalls. With giving up on Jinja2, the idea is to generate building recipes fully programmatically from Python. It would be possible to introduce "Features" - sets of packages to install based on the (distro, arch) tuple. This would result in more flexibility - turning them on/off (some could be optional, some not). There could be more than one optimization strategy regarding when packages get installed: you want only standalone blah-blah? Then Kolla won't be installing XYZ and ABC just because ugma-ugma and tickle-tickle require them and you "could save some space" (TM). In the same vein, features could declare which components are build-time and which are run-time and this would make it straightforward to separate the sides. The above effort could well be coordinated with different projects to reuse bindep contents. So far Kolla does not use bindep because it often installs too much and not enough at the same time. Do note it would still be bindep-less for external services. There would still need to exist a general mechanism for providing custom command executions required by some images. For contributors and cores this new approach would bring more sanity as to the scope of proposed changes. Also, it would be possible to get quick insight into feature support and autogenerate docs for that as well. Similarly, current concept of unbuildable images would no longer be required because unbuildability would be dictated by lack of support for a required feature. *** Looking forward to your opinions/thoughts. *** [1] https://docs.openstack.org/kolla/train/support_matrix.html [2] https://docs.openstack.org/kolla/train/admin/image-building.html -yoctozepto

5 9

Re:[Cyborg]Edge sync up at the PTG
by yumeng bao 28 May '20

28 May '20

Hi Ildikó, Thanks for the post! Cyborg team would be glad to join ! I have added this edge group meeting session to our PTG schedule : https://etherpad.opendev.org/p/cyborg-victoria-goals Please help to add links for the discussion if there is any. Regards， Yumeng

2 2

[kolla] Virtual PTG
by Mark Goddard 28 May '20

28 May '20

Hi, Just a reminder that it's the virtual PTG next week. We have an Etherpad [1] which has lots of information, and remember to register (free) [2] if you haven't already. Kendall sent some useful information [3] recently also. If you plan to attend, please add your name to the Etherpad so we know who to expect for each session (although you are welcome to turn up if you have not). The Etherpad is currently quite light on proposed discussion topics. Please do spend some time to consider what we should be using this time to discuss. It doesn't have to be a shiny new feature, it could be a long standing bug that we should agree on how to address, or a discussion on how to improve our processes. If we are left with time to spare however, I think it will be well spent on some reflection, maintenance, and planning. Cheers, Mark [1] https://etherpad.opendev.org/p/kolla-victoria-ptg [2] https://virtualptgjune2020.eventbrite.com/ [3] http://lists.openstack.org/pipermail/openstack-discuss/2020-May/015054.html

1 0

[sdk][osc][cli][ptg][project_cleanup] PTG Week planning
by Artem Goncharov 28 May '20

28 May '20

Hi everyone interested :) The PTG planning for the OpenStackSDK and the CLI is there: https://etherpad.opendev.org/p/SDK-VictoriaPTG-Planning <https://etherpad.opendev.org/p/SDK-VictoriaPTG-Planning> Please enter yourself there if interested and add topics you want to discuss. We have currently 2 slots: 01 June: 15:00 - 17:00 UTC Juno 03 June: 15:00 - 17:00 UTC Bexar And we plan to be using meetpad (fingers crossed). Interesting topics: - June 01 15:00-16:00 project cleanup - long standing topic across the projects - June 01 16:00-17:00 openstackclient-first - a one time failed community goal, and now maybe multi-release goal for joining all individual service clients into OSC to give users seamless UX Hear from you soon, Artem

2 1

[glance][ptg] Victoria Virtual PTG updates
by Abhishek Kekane 28 May '20

28 May '20

Hello All, Greetings!!! First virtual PTG is around the corner and if you haven't already registered, please do so as soon as possible [1]. I have created a Virtual PTG planning etherpad [2] and also added day wise topics along with timings we are going to discuss. Kindly let me know if you have any concerns with allotted time slots. As a reminder, these are the time slots for our discussion. Monday 01 June 2020 1400 UTC to 1700 UTC Folsom Tuesday 02 June 2020 2100 UTC to 00 UTC Austin Wednesday 03 June 2020 2100 UTC to 00 UTC Austin Thursday 04 June 2020 2100 UTC to 00 UTC Folsom Friday 05 June 2020 1400 UTC to 1700 UTC Folsom We will be using meetpad [3] for our discussion, kindly try to use it once before the actual discussion. I will publish the meeting URL before the PTG and which will be the same throughout the PTG. [1] https://virtualptgjune2020.eventbrite.com/ [2] https://etherpad.opendev.org/p/Glance-Victoria-PTG-planning [3] https://meetpad.opendev.org Thank you, Abhishek

1 0

[manila][ptg] Victoria Virtual PTG Items
by Goutham Pacha Ravi 28 May '20

28 May '20

Hello Zorillas and other interested Stackers, If you haven't already registered to be part of the virtual PTG, please do so as soon as possible [1]. There's now an etherpad posted to gather topics for the Ussuri Team Retrospective [2]. Please take a look and add your thoughts before the event. Also take a look at our Virtual PTG planning etherpad [3], and let me know if you have a problem with the draft schedule posted. As a reminder, these are the time slots for our meeting: == Monday, 1st June 2020 == 1400 UTC - 1700 UTC == Tuesday, 2nd June 2020 == 1500 UTC - 1700 UTC 2100 UTC - 2300 UTC == Friday, 5th June 2020 == 1400 UTC - 1700 UTC Our meetings are on Zoom and will be recorded. The Zoom Room is called "Cactus" and is linked out of the PTG Schedule page [4]. So make sure to bookmark it :) Looking forward to a happy week of designing with you all! -- Goutham Ravi [1] https://virtualptgjune2020.eventbrite.com/ [2] https://etherpad.opendev.org/p/manila-ussuri-retrospective [3] https://etherpad.opendev.org/p/vancouver-ptg-manila-planning [4] http://ptg.openstack.org/ptg.html

1 0

[nova][ptg] Healthchecks API
by Ghanshyam Mann 28 May '20

28 May '20

Hello Everyone, This is one of the PTG topic[1] but it will be good if we can get some discussion to happen before PTG and utilize the PTG time. I am sorry for starting this thread late. To provide the healthchecks API for nova, I have done 1 PoC which is basically extending the oslo healthchecks middleware plugins to add the real checks. - https://review.opendev.org/#/c/731396/ I am writing the details on this etherpad[2] where you can see the response and the flow of new plugins. Also describing briefly here: oslo provides healthchecks middleware with plugin framework. oslo provides two basic plugins which does file/port based health checks - disable_by_file - disable_by_files_ports New Idea is to extend those checks with new plugins on nova side: Nova can provide three plugins which are configurable[3]: 1. Nova_DB_healthcheck: Checks if API, cell0 and at least one cell DB is up. If so then return Healthy otherwise Unhealthy 2. Nova_MQ_healthcheck: Checks if at least one cells MQ is up. If so then return Healthy otherwise Unhealthy 3. Nova_services_healthcheck: Checks if at least one cell has at least one conductor and one compute service running. If so then return Healthy otherwise Unhealthy All plugins will return the dict of results for example DB dict of API, all cells DB with status, Please refer the example response in later part. TODO: Auth part for various plugins Flow diagrame of new plugins: - Nova_DB_healthcheck: -->API DB is up | --> cell0 DB and at least one cell DB is up I --> Return OK - Nova_MQ_healthcheck: -->API MQ is up | --> At leask: one cell MQ is up I --> Return OK - Nova_services_healthcheck: --> At least one cell has at least one conductor and one compute running --> TODO: need to check other services for example scheduler at least I --> Return OK Result: 200 OK if all enabled plugins return OK otherwise 503 Service Unavailable. Opinion/thoughts? [1] https://etherpad.opendev.org/p/nova-victoria-ptg [2]https://etherpad.opendev.org/p/nova-healthchecks [3] https://review.opendev.org/#/c/731396/1/etc/nova/api-paste.ini@107

1 0

[sdk][api-sig][ptg] Victoria PTG planning
by Artem Goncharov 28 May '20

28 May '20

Hi everyone interested :) The PTG planning for the API-SIG is there: https://etherpad.opendev.org/p/API-SIG-VictoriaPTG-Planning <https://etherpad.opendev.org/p/SDK-VictoriaPTG-Planning> Please enter yourself there if interested and add topics you want to discuss. We have currently 1 slot, so hurry up not to loose the chance: 02 June: 16:00 - 17:00 UTC Liberty And we plan to be using meetpad (fingers crossed). Hear from you soon, Artem

2 1

[all][interop WG] Call for Participationin for Virtual PTG interop WG and seek ambassadors
by prakash RAMCHANDRAN 27 May '20

27 May '20

Hi all, We are looking for participation & presenters, contributors form different OpenStack and adjacent projects in Open Infrastructure , k8s, CNCF, OPNFV, ONAP, OCN etc. Please register in etherpad and review for anything you like to share wrt user level API for Multi Cloud, Hybrid Cloud and Edge cloud Interop capability or compatibility. The bridge will be tested this Friday May 29th before we finalize the location or bridge, Add you name topic and your inputs for rapid delivery slots of say 5-10 mts each on June 1st. https://etherpad.opendev.org/p/interop_virtual_ptg_planning_june_2020 Schedule June 1 (UTC 13-15 hrs = 6AM-8 AM PDT = 9AM-11 AM EST = 9PM-11PM BJT) First hr will focus on Project team like Nova, Neutron, Ciner, Keystone, Glance & Swift.Second Hour - Open for new ideas Additionally we would like ambassadors who can register and attend other Virtual PTG projects and provide us feedback after attending them as what they can provide to us for Interop WG from all other projects. ThanksFor Interop WGChair PrakashVice Chair Mark

1 0

[tripleo] Ussuri containers
by Wesley Hayutin 27 May '20

27 May '20

Greetings, FYI we had a squatter in docker.io steal away the tripleoussuri namespace. We are investigating using quay but we need to do quite a bit of testing for it. For now please remind folks that the default container registry namespace is not tripleo$release for ussuri. We're using "tripleou" [1]. If you want to pull from the rdo registry, you will find those containers in tripleoussuri for now [2], but will be moved to use tripleou as well for consistency. Note the following update to tripleo-common as well [3] Full story is that we attempted to contact docker.io support several times as they have a policy around removing squatting namespaces. The support team never responded. Our goal is drop docker.io and use quay.io but taking a step by step phased approach to that. Thanks all [1] https://hub.docker.com/u/tripleou [2] https://registry.rdoproject.org:8443/oapi/v1/namespaces/tripleoussuri/image… [3] https://review.opendev.org/#/c/731244/

1 0

[Nova][CentOS] extrem slow first boot Centos 8.1 with failures
by Nicolas Ghirlanda 27 May '20

27 May '20

Hello all, I created a centos image with a pretty similar kickstart config like from that link https://github.com/CentOS/sig-cloud-instance-build/pull/159/commits/2c542bb… Packer is building the image fine, also the post installation tasks after a reboot in qemu (which is running fine and fast), so everything looks good so far. Then upload to openstack and on first boot, the instance is booting fine until that point of boot: [[0;32m OK [0m] Started D-Bus System Message Bus. [[0;32m OK [0m] Started Hardware RNG Entropy Gatherer Daemon. Starting OpenSSH rsa Server Key Generation... Starting System Security Services Daemon... [ 9.913211] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 9.988763] input: PC Speaker as /devices/platform/pcspkr/input/input6 [ 10.137969] sd 0:0:0:0: Attached scsi generic sg0 type 0 [ 10.332234] cirrus 0000:00:02.0: vgaarb: deactivate vga console [ 10.419654] Console: switching to colour dummy device 80x25 [ 10.421488] [TTM] Zone kernel: Available graphics memory: 2017692 kiB [ 10.422946] [TTM] Initializing pool allocator [ 10.424036] [TTM] Initializing DMA pool allocator [ 10.425630] [drm] fb mappable at 0xFC000000 [ 10.426612] [drm] vram aper at 0xFC000000 [ 10.427436] [drm] size 33554432 [ 10.428108] [drm] fb depth is 16 [ 10.428884] [drm] pitch is 2048 [ 10.449202] fbcon: cirrusdrmfb (fb0) is primary device [ 10.465230] Console: switching to colour frame buffer device 128x48 [ 10.499394] cirrus 0000:00:02.0: fb0: cirrusdrmfb frame buffer device [ 10.524033] [drm] Initialized cirrus 1.0.0 20110418 for 0000:00:02.0 on minor 0 On the console there is no further update from here. In the boot log it slowly progresses, but with Failures of starting services e.g. Starting RPC Bind... Starting Security Auditing Service... [ 9.500362] audit: type=1400 audit(1590589814.846:4): avc: denied { read } for pid=784 comm="auditd" name="group" dev="sda1" ino=259654 scontext=system_u:system_r:auditd_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=0 [ 9.517988] audit: type=1400 audit(1590589814.863:5): avc: denied { read } for pid=783 comm="rpcbind" name="passwd" dev="sda1" ino=259634 scontext=system_u:system_r:rpcbind_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=0 [[0;1;31mFAILED[0m] Failed to start RPC Bind. See 'systemctl status rpcbind.service' for details. or [[0;32m OK [0m] Started OpenSSH ecdsa Server Key Generation. [[0;32m OK [0m] Started GSSAPI Proxy Daemon. [[0;32m OK [0m] Started OpenSSH ed25519 Server Key Generation. [[0;32m OK [0m] Started OpenSSH rsa Server Key Generation. [[0;1;31mFAILED[0m] Failed to start Authorization Manager. See 'systemctl status polkit.service' for details. [[0;1;33mDEPEND[0m] Dependency failed for Dynamic System Tuning Daemon. [[0;1;31mFAILED[0m] Failed to start NTP client/server. See 'systemctl status chronyd.service' for details. Also cloud-init is not able to assign an ip. [[0;32m OK [0m] Started D-Bus System Message Bus. Starting Network Manager... [ 550.962041] cloud-init[1070]: Cloud-init v. 18.5 running 'init' at Wed, 27 May 2020 14:37:46 +0000. Up 460.76 seconds. [ 550.964516] cloud-init[1070]: ci-info: +++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++ [ 550.967275] cloud-init[1070]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+ [ 550.970075] cloud-init[1070]: ci-info: | Device | Up | Address | Mask | Scope | Hw-Address | [ 550.972524] cloud-init[1070]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+ [ 550.975221] cloud-init[1070]: ci-info: | ens3 | False | . | . | . | fa:16:3e:a3:7d:d5 | [ 550.977970] cloud-init[1070]: ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | host | . | [ 550.980623] cloud-init[1070]: ci-info: | lo | True | ::1/128 | . | host | . | [ 550.983300] cloud-init[1070]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+ [ 550.986164] cloud-init[1070]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++ [ 550.988469] cloud-init[1070]: ci-info: +-------+-------------+---------+-----------+-------+ [ 550.991092] cloud-init[1070]: ci-info: | Route | Destination | Gateway | Interface | Flags | [ 550.993576] cloud-init[1070]: ci-info: +-------+-------------+---------+-----------+-------+ [ 550.996326] cloud-init[1070]: ci-info: +-------+-------------+---------+-----------+-------+ After that time, the system is up, but no hostname is set (due to cloud-init issues). If I now reboot, everything is working fine, cloud-init is working etc. If I set selinux from enforcing to permissive in the kickstart config, the issues are not showing up, but I need selinux to stay enforcing. Anyone with the same issues or any hints appreciated. regards Nicolas

1 1

[ironic] [ptg] PTG week schedule
by Dmitry Tantsur 27 May '20

27 May '20

Hi ironicers! The PTG is next week, and I'd like to share our plans for it. We don't have anything planned on Monday, and the weekly meeting is cancelled as well. 2-hour slots are reserved Tuesday to Friday 14:00 UTC - 16:00 UTC in the Liberty room. The up-to-date schedule and notes are available on https://etherpad.opendev.org/p/Ironic-VictoriaPTG-Planning, the current version is as follows: Tue, Jun 2nd: 14:00-15:00: Releases, perceptions and our place in the world: governance and perceptions 15:00-16:00: Releases, perceptions and our place in the world: standalone future Wed, Jun 3rd: 14:00-15:00: Deployment Efficency 15:00-16:00: Aborting non-waiting transient states Thu, Jun 4th: 14:00-15:00: Fishy future 15:00-16:00: IPAM integrations outside of Neutron Fri, Jun 5th: 14:00-15:00: Operator suggestions/feedback/painpoints 15:00-16:00: Ideas and brainstorming I would like to highlight the operators feedback slot on Friday at 14:00 UTC. We would like to hear more opinions from people running ironic! Please join us. Hear from you soon, Dmitry

1 0

[kolla-ansible] External Ceph and cinder-volumes
by Alfredo De Luca 27 May '20

27 May '20

Hi all. I am sing kolla-ansible 4.0 with and external ceph already deployed. I have in globals.yml the following setting: enable_ceph: "no" glance_backend_ceph: "yes" nova_backend_ceph: "yes" cinder_backend_ceph: "yes" enable_cinder_backend_lvm: "no" When I run kolla-ansible I got an error about not having cinder-volumes group Cannot process volume group cinder-volumes Do you know why it keeps looking for the cinder volumes? I ve done it before with no problem but maybe something in the global not correctly configured? Cheers -- */Alfredo*

2 7

[cyborg] Edge sync up at the PTG
by Ildiko Vancsa 27 May '20

27 May '20

Hi Cyborg Team, I’m reaching out to check if you are available during next week to continue discussions on edge related topics that we started in Shanghai. The OSF Edge Computing Group is meeting next week and we have a slot reserved to sync up with OpenStack projects such as Cyborg on Wednesday (June 3) at 1300 UTC - 1400 UTC. Would the team be available to join at that time? Thanks, Ildikó

1 0

[tripleo][operators] Removal of mistral from the TripleO Undercloud
by Kevin Carter 27 May '20

27 May '20

Hello stackers, In the pursuit to remove Mistral from the TripleO undercloud, we've discovered an old capability that we need to figure out how best to handle. Currently, we provide the ability for an end-user (operator / deployer) to pass in "N" Mistral workflows as part of a given deployment plan which is processed by python-tripleoclient at runtime [0][1]. From what we have documented, and what we can find within the code-base, we're not using this feature by default. That said, we do not remove something if it is valuable in the field without an adequate replacement. The ability to run arbitrary Mistral workflows at deployment time was first created in 2017 [2] and while present all this time, its documented [3] and intra-code-base uses are still limited to samples [4]. As it stands now, we're on track to making Mistral inert this cycle and if our progress holds over the next couple of weeks the capability to run arbitrary Mistral workflows will be the only thing left within our codebase that relies on Mistral running on the Undercloud. So the question is what do we do with functionality. Do we remove this ability out right, do we convert the example workflow [5] into a stand-alone Ansible playbook and change the workflow runner to an arbitrary playbook runner, or do we simply leave everything as-is and deprecate it to be removed within the next two releases? Although it would be perfectly acceptable to keep Mistral around a little while longer, it would also be a bummer if this one capability was the only reason we were not able to remove Mistral from the Undercloud. Any and all feedback from operators, deployers, developers, former developers, etc would be greatly appreciated. Thank you in advance. [0] https://opendev.org/openstack/python-tripleoclient/src/commit/af719b795bc9b… [1] https://opendev.org/openstack/python-tripleoclient/src/commit/af719b795bc9b… [2] https://review.opendev.org/#/c/457874 [3] https://specs.openstack.org/openstack/tripleo-specs/specs/pike/tripleo-deri… [4] https://opendev.org/openstack/tripleo-heat-templates/src/commit/48da6a13936… [5] https://opendev.org/openstack/tripleo-common/src/commit/cc4f9c57b708259e9e9… Kevin Carter IRC: kecarter

5 9

[tripleo] TripleO Ussuri has been released
by Wesley Hayutin 27 May '20

27 May '20

Greetings, I'm pleased to announce that TripleO Ussuri has been released today. Special thanks to Chandan Kumar and Emilien Macchi for the extra time and effort in getting everything in place for the release. Thanks to everyone else as well for your contributions, bug reports, patches and participation! Some highlights from the TripleO release: * The mistral to ansible conversion blueprint is complete * The nova-less-deploy feature is complete * The TripleO-Operator-Ansible feature is complete * The RDO and CI teams completed work on component based DLRN rpm repos and CI. * TLS and TripleO deployments tested together in upstream CI. * Support for CentOS-8 * Many other improvements and updates Congratulations to the following individual contributors for their reviews across ALL openstack projects!! Alex Schultz, the #1 reviewer for all projects in Ussuri Herve Beraud, the #3 reviewer for all projects in Ussuri Emilien Macchi, the #4 reviewer for all projects in Ussuri Followed closely behind in reviews by Stephen Finucane Dmitry Tantsur Slawek Kaplonski Thanks to all our TripleO contributors!! https://www.stackalytics.com/?release=ussuri&metric=commits&project_type=op… Oh by the way TripleO fixed 558 bugs this release as well :) Additional statistics can be found here: https://docs.google.com/presentation/d/1RV30OVxmXv1y_z33LuXMVB56TA54Urp7oHI… <https://docs.google.com/presentation/d/1RV30OVxmXv1y_z33LuXMVB56TA54Urp7oHI…> Release notes can be found per TripleO project: https://docs.openstack.org/releasenotes/ At the time of writing this email not all the release notes have been published. Thanks all and I'm looking forward to seeing you at the PTG!

2 1

[PTG][neutron] Draft of the agenda
by Slawek Kaplonski 27 May '20

27 May '20

Hi neutrinos, I just prepared draft of the agenda for the Neutron team for the upcoming virtual PTG. It's available in the etherpad [1]. Please take a look into it and reach out to me if You: * want to change day/time of topic(s) which You are interested in mostly due to e.g. some conflicts which You may have with some other meetings, * if You want to add some new topics which, to make sure that agenda is modified to include that, * if You see anything else which You thinks that should be changed there :) See You all on the virtual PTG next week :) [1] https://etherpad.opendev.org/p/neutron-victoria-ptg -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[all][policy] Secure Default Policies Popup team meeting
by Ghanshyam Mann 27 May '20

27 May '20

Hello Everyone, To speed up the work on Secure Default Policies Popup team[1], we are going to start the biweekly meeting to discuss the pending things to finish if any or project need help on implementing the new default policy. We encourage projects interested to implement the new defaults or at least those listed on the wiki page [2] to join that meeting. Meeting details are on the wiki[3] also I have proposed the patch to book the slot: https://review.opendev.org/#/c/730935/ Agenda is slightly open for now which we will be moving to concrete as we progress and timing is biweekly-even on Thursday starting from this week on 28th 1800 UTC. We can adjust the timing also if needed. NOTE: First meeting on this week Thursday 28th 1800UTC on #openstack-meeting [1] https://governance.openstack.org/tc/reference/popup-teams.html#secure-defau… [2] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… [3] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… -gmann

1 0

[openstack-helm] Stepping down as core reviewer
by Steve Wilkerson 26 May '20

26 May '20

Hey everyone, I hope you're all staying safe and well. I switched jobs a few months back that took me in a different direction from my previous employer, and as a result I've not had the time to remain involved with the OpenStack community (especially as my day job no longer involves working with OpenStack). I think it's best that I step down from the openstack-helm core reviewer team, as I can't realistically make the time to do my due diligence in providing thorough, useful code reviews. I greatly appreciate the opportunities that working with the OpenStack community at large has provided me over the past 5 years. Cheers, srwilkers

2 1

[kolla-ansible][kolla][cinder]Cinder volume is not attached to instance
by Sanjay K 26 May '20

26 May '20

Hello Openstackers, I have used Cinder stein release (v14.0.2) source base and deployed in All-In-One setup on ubuntu16.04 using Kolla-Ansible. The cinder image was built using python 2.7 somewhere which i am trying to deploy in my new setup having python3.5. All services are deployed and running fine. However when I try to attach one volume to an instance, it shows below error in /var/log/kolla/cinder/cinder-volume.log. 2020-05-26 08:10:07.026 35 INFO oslo.privsep.daemon [req-db287360-876c-425e-b8df-a4f3b0db3163 bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'cinder.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpan8c8r/privsep.sock'] 2020-05-26 08:10:10.952 35 INFO oslo.privsep.daemon [req-db287360-876c-425e-b8df-a4f3b0db3163 bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] Spawned new privsep daemon via rootwrap 2020-05-26 08:10:10.647 1071 INFO oslo.privsep.daemon [-] privsep daemon starting 2020-05-26 08:10:10.721 1071 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 2020-05-26 08:10:10.725 1071 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none 2020-05-26 08:10:10.726 1071 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1071 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server [req-db287360-876c-425e-b8df-a4f3b0db3163 bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] Exception during message handling: ProcessExecutionError: Unexpected error while running command. Command: tgtadm --lld iscsi --op show --mode target Exit code: 107 Stdout: u'' Stderr: u'tgtadm: failed to send request hdr to tgt daemon, Transport endpoint is not connected\n' 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/manager.py", line 4517, in attachment_update 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server connector) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/manager.py", line 4445, in _connection_create 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server volume, connector) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/drivers/lvm.py", line 830, in create_export 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server volume_path) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/targets/iscsi.py", line 210, in create_export 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server **portals_config) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/utils.py", line 818, in _wrapper 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server return r.call(f, *args, **kwargs) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/retrying.py", line 206, in call 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server return attempt.get(self._wrap_exception) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/retrying.py", line 247, in get 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server six.reraise(self.value[0], self.value[1], self.value[2]) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/retrying.py", line 200, in call 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server attempt = Attempt(fn(*args, **kwargs), attempt_number, False) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/volume/targets/tgt.py", line 126, in create_iscsi_target 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server (out, err) = cinder.privsep.targets.tgt.tgtadm_show() 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 244, in _wrap 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server return self.channel.remote_call(name, args, kwargs) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 203, in remote_call 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server raise exc_type(*result[2]) 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server ProcessExecutionError: Unexpected error while running command. 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server Command: tgtadm --lld iscsi --op show --mode target 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server Exit code: 107 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server Stdout: u'' 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server Stderr: u'tgtadm: failed to send request hdr to tgt daemon, Transport endpoint is not connected\n' 2020-05-26 08:10:11.550 35 ERROR oslo_messaging.rpc.server When I check inside cinder-volume container, the tgtd is not running. SO I started tgtd from root user. Now the volume attachment happens for few seconds and then it is detached. 2020-05-26 08:14:39.221 35 INFO cinder.volume.manager [req-5da81388-33ce-4994-9114-52442a08ac2a bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] attachment_update completed successfully. 2020-05-26 08:14:52.622 35 INFO cinder.volume.manager [req-cbaa1b6c-4fa8-4ece-b08b-8ca09f78ecaf bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] Terminate volume connection completed successfully. 2020-05-26 08:14:52.840 35 INFO cinder.volume.targets.tgt [req-cbaa1b6c-4fa8-4ece-b08b-8ca09f78ecaf bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] Removing iscsi_target for Volume ID: d713ce55-1fa6-4d96-a69e-69368494f8f9 2020-05-26 08:14:53.739 35 WARNING py.warnings [req-cbaa1b6c-4fa8-4ece-b08b-8ca09f78ecaf bc13e1a279854a6fa1d553c7b7a0214c 9bdfc4ffefaa4edb823ff9ba5383ab42 - default default] /var/lib/kolla/venv/lib/python2.7/site-packages/sqlalchemy/orm/evaluator.py:99: SAWarning: Evaluating non-mapped column expression 'updated_at' onto ORM instances; this is a deprecated use case. Please make use of the actual mapped columns in ORM-evaluated UPDATE / DELETE expressions. "UPDATE / DELETE expressions." % clause I also used the cinder 14.0.4 source tag to build the new image, but the same error persists. Can you please point me what is the issue in my test case or setup. Appreciate your help. best regards, Sanjay

2 1

[Horizon] Failing to configure the Instance Launch dialog
by Bernd Bausch 26 May '20

26 May '20

On an Ussuri Devstack, I would like to launch instances with ephemeral storage from Horizon and remove the volume storage switch from the instance launch dialog. Prior to Train, the dialog was configured with the LAUNCH_INSTANCE_DEFAULTS dictionary in /local_settings.py/. This is my dictionary; I set /create_volume /to /False /and /hide_create_volume /to /True/: LAUNCH_INSTANCE_DEFAULTS = { 'config_drive': False, 'create_volume': False, 'hide_create_volume': True, 'disable_image': False, 'disable_instance_snapshot': False, 'disable_volume': False, 'disable_volume_snapshot': False, 'enable_scheduler_hints': True, } Starting with Train, the dictionary is in /.../openstack_dashboard/defaults.py/, but the changes I made there have no effect. Adding the dictionary to /local_settings.py/ has no effect either. To test whether /defaults.py/ is used at all, I added a syntactically incorrect line, and indeed, the syntax error is logged in /horizon_error.log/. I wonder what I am doing wrong. I am convinced it worked like this on Stein. Any ideas? Bernd Bausch

2 3

[TripleO] External network on compute node
by Reza Bakhshayeshi 26 May '20

26 May '20

Hi all. The default way of compute node for accessing Internet if through undercloud. I'm going to assign an IP from External network to each compute node with default route. But the deployment can't assign an IP to br-ex and fails with: " raise AddrFormatError('invalid IPNetwork %s' % addr)", "netaddr.core.AddrFormatError: invalid IPNetwork ", Actually 'ip_netmask': '' is empty during deployment for compute nodes. I've added external network to compute node role: External: subnet: external_subnet and for network interface: - type: ovs_bridge name: bridge_name mtu: get_param: ExternalMtu dns_servers: get_param: DnsServers use_dhcp: false addresses: - ip_netmask: get_param: ExternalIpSubnet routes: list_concat_unique: - get_param: ExternalInterfaceRoutes - - default: true next_hop: get_param: ExternalInterfaceDefaultRoute members: - type: interface name: nic3 mtu: get_param: ExternalMtu use_dhcp: false primary: true Any suggestion would be grateful. Regards, Reza

2 6

[Nova][Scheduler] Reducing race-conditions and re-scheduling during creation of multiple high-ressources instances or instances with anti-affinity.
by Laurent Dumont 25 May '20

25 May '20

Hey everyone, We are seeing a pretty consistent issue with Nova/Scheduler where some instances creation are hitting the "max_attempts" limits of the scheduler. Env : Red Hat Queens Computes : All the same hardware and specs (even weight throughout) Nova : Three nova-schedulers This can be due to two different factors (from what we've seen) : - Anti-affinity rules are getting triggered during the creation (two claims are done within a few milliseconds on the same compute) which counts as a retry (we've seen this when spawning 40+ VMs in a single server group with maybe 50-55 computes - or even less 14 instances on 20ish computes). - We've seen another case where MEMORY_MB becomes an issue (we are spinning new instances in the same host-aggregate where VMs are already running. Only one VM can run per compute but there are no anti-affinity groups to force that between the two deployments. The ressource requirements prevent anything else from getting spun on those). - The logs look like the following : - Unable to submit allocation for instance 659ef90e-33b8-42a9-9c8e-fac87278240d (409 {"errors": [{"status": 409, "request_id": "req-429c2734-2f2d-4d2d-82d1-fa4ebe12c991", "detail": "There was a conflict when trying to complete your request.\n\n Unable to allocate inventory: Unable to create allocation for 'MEMORY_MB' on resource provider '35b78f3b-8e59-4f2f-8cad-eaf116b7c1c7'. The requested amount would exceed the capacity. ", "title": "Conflict"}]}) / Setting instance to ERROR state.: MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f6d06cca-e9b5-4199-8220-e3ff2e5c2a41. - I do believe we are hitting this issue as well : https://bugs.launchpad.net/nova/+bug/1837955 - In all the cases where the Stacks creation failed, one instance was left in the Build state for 120 minutes and then finally failed. >From what we can gather, there are a couple of parameters that be be tweaked. 1. host_subset_size (Return X number of host instead of 1?) 2. randomize_allocation_candidates (Not 100% on this one) 3. shuffle_best_same_weighed_hosts (Return a random of X number of computes if they are all equal (instance of the same list for all scheduling requests)) 4. max_attempts (how many times the Scheduler will try to fit the instance somewhere) We've already raised "max_attempts" to 5 from the default of 3 and will raise it further. That said, what are the recommendations for the rest of the settings? We are not exactly concerned with stacking vs spreading (but that's always nice) of the instances but rather making sure deployments fail because of real reasons and not just because Nova/Scheduler keeps stepping on it's own toes. Thanks!

3 6

[all][interop] Please suggest capabilities to deprecate-remove and add to the OpenStack Powered Programs
by prakash RAMCHANDRAN 25 May '20

25 May '20

Hi all, Please help the Interop WG to review and finalize the draft version for the next interop guidelines. https://review.opendev.org/gitweb?p=openstack/interop.git;a=blob;f=2020.06.… Link for the DNS add-on: https://review.opendev.org/gitweb?p=openstack/interop.git;a=blob;f=add-ons/… Link for the Heat add-on: https://review.opendev.org/gitweb?p=openstack/interop.git;a=blob;f=add-ons/… This guidelines is intended to cover "Stein" , "Train", "Ussuri" and "Victoria" releases of OpenStack. We request that PTLs or Core team members please confirm any updates you may want to include or exclude in link above. The Projects currently covered under OpenStack Powered Programs include "Keystone", "Glance", "Nova", "Neutron", "Cinder", "Swift" and add-on programs "Designate", "Heat". A more human-readable view of the capabilities can be found in RefStack: https://refstack.openstack.org/#/guidelines We would like have feedback by next Interop call on Friday. - Can attend 10 AM PDT call refer etherpad - https://etherpad.opendev.org/p/interop If you have any additional topics to discuss at PTG please add them to https://etherpad.opendev.org/p/interop_virtual_ptg_planning_june_2020 The conference Bridge will be assigned once we know the number of participants. Please add your names/topics to ether pad interop ptg planning. Optionally reply to this email with your inputs/patches. Please Register to attend the Interop WG slot scheduled for Virtual PTG on June 1 6-8 AM PDT / 9-11 AM EST / 14-16 UTC / 9-11 PM BJT https://www.openstack.org/ptg/ Thanksfor Interop WGChair - Prakash RamchandranVice Chair - Mark Voelker

4 4

Bug deputy report for week of 2020-05-18
by Maciej Jozefczyk 25 May '20

25 May '20

Hello, Hey, Here's the bug deputy report for last week: ===== High ===== * https://bugs.launchpad.net/neutron/+bug/1879301 [OVN] networking-ovn-tempest-dsvm-ovs-release-python2 job starts to fail on tempest py2 installation fix already merged: https://review.opendev.org/#/c/728817/ * https://bugs.launchpad.net/neutron/+bug/1879307 Recent stable/rocky change breaks networking-calico's interface driver fix proposed: https://review.opendev.org/#/c/729167/ * https://bugs.launchpad.net/neutron/+bug/1879950 [OVN] neutron-ovn-db-sync-util wipes out hash ring nodes fix proposed: https://review.opendev.org/#/c/729997/ ===== Medium ===== * https://bugs.launchpad.net/neutron/+bug/1879215 [L3] Unexcepted HA router scheduled instance shows up after manully scheduling Unassigned. * https://bugs.launchpad.net/neutron/+bug/1879502 [ovn] neutron-ovn-db-sync-util fails with PortContext has no session fix proposed: https://review.opendev.org/#/c/729264/ * https://bugs.launchpad.net/neutron/+bug/1879747 [DOC] Manual install & Configuration in neutron fix proposed: https://review.opendev.org/#/c/729894/ ===== Low ===== * https://bugs.launchpad.net/neutron/+bug/1879407 [OVN] Modifying FIP that is no associated causes ovn_revision_numbers to go stale unassigned, no big deal * https://bugs.launchpad.net/neutron/+bug/1879716 Policy is not enforced for network mtu config unassigned ==== Reopened ==== https://bugs.launchpad.net/neutron/+bug/1742401 Fullstack tests neutron.tests.fullstack.test_securitygroup.TestSecurityGroupsSameNetwork fails often ===== Incomplete ===== * https://bugs.launchpad.net/neutron/+bug/1878719 DHCP Agent's iptables CHECKSUM rule causes skb_warn_bad_offload kernel * https://bugs.launchpad.net/neutron/+bug/1878905 when I use dvr mode for neutron,it's not functional * https://bugs.launchpad.net/neutron/+bug/1879009 attaching extra port to server raise duplicate dns-name error fix proposed: https://review.opendev.org/#/c/729440/ (in progress) See you, Maciej -- Best regards, Maciej Józefczyk

1 0

[neutron] Team meeting on Monday 25.05.2020 cancelled
by Slawek Kaplonski 25 May '20

25 May '20

Hi, As today is the Memorial Day in the USA, and many from team members are on holidays, lets cancel today's team meeting. Next week there is virtual PTG so we will meet there :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[swift] Rolling upgrade, any version relationships?
by Mark Kirkwood 25 May '20

25 May '20

Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are: - If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)? - If not is there a preferred version to jump to first? - Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time? regards Mark

3 5

[keystone][devstack][openstackclient] Problem with Keystone setup in stable/ussuri devstack install
by Neil Jerram 23 May '20

23 May '20

+ ./stack.sh:main:1091 : create_keystone_accounts + lib/keystone:create_keystone_accounts:314 : local admin_project ++ lib/keystone:create_keystone_accounts:315 : oscwrap project show admin -f value -c id WARNING: Failed to import plugin identity. ++ functions-common:oscwrap:2370 : return 2 + lib/keystone:create_keystone_accounts:315 : admin_project='openstack: '\''project show admin -f value -c id'\'' is not an openstack command. See '\''openstack --help'\''. I believe this is a completely mainline Keystone setup by devstack. The top-level stack.sh code is echo_summary "Starting Keystone" if [ "$KEYSTONE_AUTH_HOST" == "$SERVICE_HOST" ]; then init_keystone start_keystone bootstrap_keystone fi create_keystone_accounts ... bootstrap_keystone succeeded but create_keystone_accounts failed as shown above, trying to execute openstack project show "admin" -f value -c id IIUC, the rootmost problem here is "WARNING: Failed to import plugin identity.", indicating that python-openstackclient is failing to import its openstackclient.identity.client module. But I don't know any more about why that would be. Any ideas? Many thanks, Neil

2 3

[all][qa] Gate Status: week 20-21
by Ghanshyam Mann 23 May '20

23 May '20

Hello Everyone, I am highlighting the few of the gate issues we fixed or in-progress recently in case you are facing the same in your projects gate or 3rd party CI. 1. Bug# 1880219. A job using older Tempest (ubuntu xenial node jobs) on stable stein onwards gate or 3rd party CI fail. This is failing the same type of job I reported a few weeks back but with different issues and happening on 3rd party CI as I have not seen such a job or failure on upstream yet. I am reproducing the scenario to test the fixes[1]. Earlier issue[2] was fixed by using the older Tempest. Now it started failing as we use the master constraint on stein and train and master constraints are no more compatible with older Tempest[3]. The solution for this will be to use stable constraints if older Tempest is not used. This fix needs modification on devstack as well as the tempest side. Devstack changes[4] working fine but I need to fix it on the Tempest side also. 2. Bug#1880217. In-tree Tempest plugins python2 jobs on stable/stein or Train failing This might be a rare case but still possible as we do have in-tree plugins for older stable branches. For in-tree plugins (in case it's still present even we migrated most of the tempest plugins into separate repo ), 'all-plugins' tox env which is supposed to detect the system-wide installed plugins will not work now for python2.7 jobs. This is because Tempest is now py3-only and so create py3 venv even on py2 jobs. 'all-plugins' venv which is py3 no longer detect the tempest plugins installed as python2.7 site-packages. Failure example: https://zuul.opendev.org/t/openstack/build/7d3f36f464bb4642bb1647f4b0b8e91f… If you encounter this issue then a quick fix for that will be to install the in-tree plugins in Tempest venv via TEMPEST_PLUGINS and run tests with 'all' tox venv. Example: https://review.opendev.org/#/c/730378/2 3. hacking fix for new flake8 3.8.0 merged in master might need to be backported to stable branches also but not for all projects. But to have a safe guard against future flake8 3.9.0(which can also cause same issue), you can backport the hacking min version fix: - http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html Have a happy weekend and feel free to append more gate bugs here you have encountered or fixed recently. [1] https://review.opendev.org/#/c/729611/3 [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014601.html [3] https://review.opendev.org/#/c/720264/ [4] https://review.opendev.org/#/c/729722/6 -gmann

1 0

[swift] Erasure code hardware offloading
by Mark Kirkwood 22 May '20

22 May '20

Hi, Can Swift make use of an Erasure Code offload card (e.g https://community.mellanox.com/s/article/understanding-erasure-coding-offlo… regards Mark

2 1

[charms] Propose Aurelien Lourot for OpenStack Charms core
by Frode Nordahl 22 May '20

22 May '20

Hello all, I would like to propose adding Aurelien Lourot as core developer for the Charms project. -- Frode Nordahl

4 3

[release] Release countdown for week R-20, May 25 - 29
by Sean McGinnis 22 May '20

22 May '20

Welcome back to the release countdown emails! These will be sent at major points in the Victoria development cycle, which should conclude with a final release on October 16, 2020. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the Victoria development cycle, assessing Victoria community goals and approving Victoria specs. General Information ------------------- Victoria is a 22 week development cycle, which is already underway. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/victoria/schedule.html By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the Ussuri release are continued in the Victoria series with a similar release model. If you intend to drop a deliverable, or modify its release model, please do so before the victoria-1 milestone by proposing a change to the deliverable file at: https://opendev.org/openstack/releases/src/branch/master/deliverables/victo… Upcoming Deadlines & Dates -------------------------- Virtual Victoria PTG: June 1-5 Victoria-1 milestone: June 18

1 0

[neutron][operators]
by Dincer Celik 22 May '20

22 May '20

Hi, You know in public cloud world, most users prefer using VPCs (aka tenant networks for us), and when they need to connect two separate VPC, they have VPC peering functionality. For OpenStack, we have routers, RBACs, static routes and floating IPs, but there is nothing functioning like a real VPC peering. I see this was discussed at Stein PTG[1] under L3 topics but seems there is no progress about it. So I would like to ask what is the most common way to let two separate tenant networks under different accounts talk to each other? Thanks! Dincer [1] https://etherpad.opendev.org/p/neutron-stein-ptg

1 0

[charms] OpenStack Charms 20.05 release is now available
by Peter Matulis 22 May '20

22 May '20

The OpenStack Charms team is happy to announce the 20.05 charms release, introducing support for OpenStack Ussuri and Ceph Octopus on Ubuntu 18.04 LTS and Ubuntu 20.04 LTS. This release also brings several new and valuable features to the existing OpenStack Charms deployments for Queens, Rocky, Stein, Train, Ussuri and many other stable combinations of Ubuntu + OpenStack. Please see the Release Notes for full details: https://docs.openstack.org/charm-guide/latest/2005.html == Highlights == * OpenStack Ussuri OpenStack Ussuri is now supported on Ubuntu 18.04 LTS (via UCA) and Ubuntu 20.04 LTS natively. * Ceph Octopus The Octopus release of Ceph is now supported, starting with OpenStack Ussuri. * New charms: MySQL 8 The MySQL 8 charms (mysql-innodb-cluster and mysql-router) have been promoted to supported status. These charms allow for a completely native cloud database HA cluster (no need for the hacluster charm). The legacy solution (percona-cluster charm) is not supported on Ubuntu 20.04 LTS. Migration documentation is available. * New charms: Masakari The Masakari charms (masakari and masakari-monitors) and the requisite pacemaker-remote charm have been promoted to supported status, starting with OpenStack Stein. Masakari provides automated recovery of OpenStack instances (shared storage required). * New charms: OVN The OVN charms (neutron-api-plugin-ovn, ovn-central, ovn-chassis, and ovn-dedicated-chassis) have been promoted to supported status. OVN provides virtual networking for Open vSwitch and is the preferred default for new deployments of OpenStack Ussuri. * Ceph iSCSI Ceph iSCSI gateway functionality is available as a tech preview. These gateways provide iSCSI targets backed by a Ceph cluster. == OpenStack Charms team == The OpenStack Charms team can be contacted on the #openstack-charms IRC channel on Freenode. == Thank you == Lots of thanks to the below 67 charm contributors who squashed 89 bugs, enabled support for a new release of OpenStack, improved documentation, and added exciting new functionality! James Page Alex Kavanagh Frode Nordahl Liam Young Peter Matulis David Ames Corey Bryant Chris MacNaughton Ghanshyam Mann Aurelien Lourot Sahid Orentino Ferdjaoui Stamatis Katsaounis Ryan Beisner inspurericzhang Tiago Pasqualini Andrew McLeod ShangXiao Edward Hope-Morley Felipe Reyes kangyufei Adam Dyess Andreas Jaeger wangfaxin Xuan Yandong Arif Ali José Pekkarinen Chris Johnston Hemanth Nakkina Alexandros Soumplis Jose Guedez Qitao Tytus Kurek Seyeong Kim Dmitrii Shcherbakov Dongdong Tao Drew Freiberger Haw Loeung Jorge Niedbalski Andrea Ieri Xav Paice Qiu Fossen Yanos Angelopoulos Syed Mohammad Adnan Karim JiaSiRui Sérgio Filipe Marques Manso Vladimir Grevtsev George Kraft Marco Filipe Moutinho da Silva Xiyue Wang Adam Dyess Jose Delarosa Alexander Balderson Facundo Ciccioli Jacek Nykis Eduardo Sousa Thobias Trevisan Rodrigo Barbieri Alejandro Santoyo Gonzalez Sean McGinnis Claudio Pisa Shuo Liu Matus Kosut Stephen Muss Camille Rodriguez zhangboye Aggelos Kolaitis Nicolas Bock -- OpenStack Charms Team

2 1

[neutron] Drivers meeting 22.05.2020
by Slawek Kaplonski 22 May '20

22 May '20

Hi, Due to lack of agenda for the next drivers meeting, lets cancel it this week. See You all next week. BTW. Please maybe check Nate's email [1] - this isn't exactly RFE but something which IMO drivers team should check too :) [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014820.html -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[kolla] Kolla klub meeting
by Mark Goddard 21 May '20

21 May '20

Hi, Just a reminder that we will host a project onboarding session for the next Kolla Klub meeting on Thursday 21st May. We'll cover a variety of topics on the many different ways you can contribute to the project. Gaël Therond will also fill us in on some of the questions put to him since the last meeting about his case studies. https://docs.google.com/document/d/1EwQs2GXF-EvJZamEx9vQAOSDB5tCjsDCJyHQN5_… Thanks, Mark

1 1

review of tripleo-ci-centos-8-standalone-on-multinode-ipa
by Wesley Hayutin 21 May '20

21 May '20

Greetings, Please review the following new standalone + ipa deployment [1]. The keystone folks would like this deployment to run by default on all THT changes vs. the current set of files [2]. Looking for any strong objects to running the job on any THT change and or more files to target the job against in [2] If you need specific information about the job please contact rlandy @ redhat pojadhav @ redhat Thanks [1] https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-o… [2] https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/zuul…

1 0

[Heat] Heat Stack Create Authorization Failed
by İzzettin Erdem 21 May '20

21 May '20

Hello everyone, When I launch a cluster on Sahara, it gives a heat stack authorization error. I also use Murano service and it is working with heat. I could not find the solution and I discuss this error with the Sahara-devel team. They are also searching for this. Could you help me, please? Both the error log of Sahara and Heat are below. Sahara-engine: http://paste.openstack.org/show/793671/ Heat-engine: http://paste.openstack.org/show/793673/ Thanks. Regards.

4 4

[ptg][cyborg][neutron][nova]Smartnic support integration via cyborg neutron and nova? - Cyborg Pre-PTG Meeting
by yumeng bao 21 May '20

21 May '20

Hi, As agreed on the IRC, tomorrow, we will have a cyborg Pre-PTG Zoom Meeting. Shaohe Feng from Intel will introduce SmartNIC support from the following aspects: * the cyborg-neutron-nova integration workflow when creating a server with a port of SmartNIC which is managed by cyborg *what changes needs to be done in nova, neutron to support SmartNIC feature? Please find the Pre-PTG meeting Link below: Zoom URL: https://us04web.zoom.us/j/9575741145?pwd=cVZydHl6blYzTGtNVlU4QU96ZWxiZz09 Meeting ID：957 574 1145 Password：9XhFhK The reserved meeting time is in total one and a half hours, but divided into three time slots(if time is not enough, we can extend another half hour time slot). *May 22 Friday 2:30 UTC -3:00 UTC *May 22 Friday 3:00 UTC -3:30UTC *May 22 Friday 3:30 UTC -4:00UTC * *And Please be aware: The three slots use the same URL, same Meeting ID and same Password. This means, when time of the slot is running out,and if you're kicked out of the Zoom, then you will need to login to the same Zoom URL again to join the next time slot. (And there’s also possibility that we don’t need to do the re-login, this depends on the number of people.） For others interested but not quite familiar with this topic, here’s some background information: Cyborg, served as a general purpose management framework for acceleration resources, is trying to support various types of accelerators.(FPGA, GPU, SPDK and so on) We continuously got questions from operators if cyborg supports SmartNIC, and this motivated us to support SmartNIC. Actually, we’ve made a discussion on SmartNIC with neutron team at the Shanghai Summit[1] 2019. We are now planning to continue discussing SmartNICs support integration in the coming Virtual PTG and try to move forward in Victoria Release. Cyborg will discuss about this topic internally first, and then later discuss with nova and neutron at PTG cross-project session[2][3]. [1] https://etherpad.opendev.org/p/Shanghai-Neutron-Cyborg-xproj [2] https://etherpad.opendev.org/p/nova-victoria-ptg [3] https://etherpad.opendev.org/p/cyborg-victoria-goals Regards， Yumeng

1 0

[nova][cinder][linstor] VM creation failed with invalid libvirt xml <source dev='None'/>
by Volodymyr Litovka 21 May '20

21 May '20

Hi colleagues, does anybody here use Linstor as backend for Openstack? I can not launch VM (synopsis below) while everything looks fine until the last step :-) Few words about Openstack's storage configuration: - controllers: cinder-api, cinder-scheduler, cinder-volume (modified with latest driver from Linstor's git), linstor-client, python-linstor - compute nodes: linstor-controller, linstor-satellite, linstor-client, python-linstor; node types - Combined - cinder.conf[1] - volume-type registered with corresponding properties[2] I'm able to manipulate volumes - create/delete[3] and every existing volume has the corresponding drbd/linstor repr[4] and LVM LV[5] on the compute nodes, so, from storage point of view, it looks as correct and ready to use. The problem is when I try to create VM using $ openstack server create --flavor [...] --network [...] --volume 15c4fa98-76a5-44d6-9701-c89a8a848b82 fearst Nova fails[6] because of invalid VM's libvirt XML definition (note "source dev='None'" below): <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source dev='None'/> <target dev='vda' bus='virtio'/> <serial>15c4fa98-76a5-44d6-9701-c89a8a848b82</serial> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> I do not have another working DRBD/Linstor/Openstack installation and can't even imagine, where to look into the problem - whether it is in Nova, Cinder or in Linstor configuration itself. I will appreciate if anybody can give me some guidance on where to seek for the solution to the problem. In my lab I'm using Openstack Train on Ubuntu 18.04.4: nova-compute 2:20.1.1-0ubuntu1~cloud0 nova-compute-kvm 2:20.1.1-0ubuntu1~cloud0 nova-compute-libvirt 2:20.1.1-0ubuntu1~cloud0 nova-api 2:20.1.1-0ubuntu1~cloud0 nova-common 2:20.1.1-0ubuntu1~cloud0 nova-conductor 2:20.1.1-0ubuntu1~cloud0 nova-novncproxy 2:20.1.1-0ubuntu1~cloud0 nova-scheduler 2:20.1.1-0ubuntu1~cloud0 cinder-api 2:15.1.0-0ubuntu1~cloud0 cinder-common 2:15.1.0-0ubuntu1~cloud0 cinder-scheduler 2:15.1.0-0ubuntu1~cloud0 cinder-volume 2:15.1.0-0ubuntu1~cloud0 python3-cinder 2:15.1.0-0ubuntu1~cloud0 python3-cinderclient 1:5.0.0-0ubuntu2~cloud0 python3-nova 2:20.1.1-0ubuntu1~cloud0 python3-novaclient 2:15.1.0-0ubuntu2~cloud0 Thank you! ==================================== [1] cinder.conf [DEFAULT] enabled_backends = linstor default_volume_type = linstor [linstor] storage_availability_zone = nova volume_backend_name = linstor volume_driver = cinder.volume.drivers.linstordrv.LinstorDrbdDriver linstor_autoplace_count = 2 linstor_default_volume_group_name=DfltRscGrp linstor_default_uri=linstor://runner linstor_default_storage_pool_name=drbdpool linstor_default_resource_size=1 linstor_volume_downsize_factor=4096 where 'runner' is virtual ip of linstor-controller, protected by pacemaker. ==================================== [2] openstack volume type show linstor +--------------------+--------------------------------------+ | Field | Value | +--------------------+--------------------------------------+ | access_project_ids | None | | description | None | | id | d2025962-503a-4f37-93bd-b766bb346a42 | | is_public | True | | name | linstor | | properties | volume_backend_name='linstor' | | qos_specs_id | None | +--------------------+--------------------------------------+ ==================================== [3] openstack volume show fearst +--------------------------------+-----------------------------------------------------+ | Field | Value | +--------------------------------+-----------------------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | true | | consistencygroup_id | None | | created_at | 2020-05-21T08:42:11.000000 | | description | None | | encrypted | False | | id | 15c4fa98-76a5-44d6-9701-c89a8a848b82 | | migration_status | None | | multiattach | False | | name | fearst | | os-vol-host-attr:host | ctrl1@linstor#linstor | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 7acec404393344cbabde07a22bbe6b3f | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | available | | type | linstor | | updated_at | 2020-05-21T08:42:18.000000 | | user_id | 1048a19e94234451a61c3d9d46907ceb | | volume_image_metadata | {'signature_verified': 'False', | | | 'image_id': '15e992e8-a601-44cd-a2f0-8f34588a4d18',| | | 'image_name': 'cirros', | | | 'checksum': '1d3062cd89af34e419f7100277f38b2b', | | | 'container_format': 'bare', | | | 'disk_format': 'qcow2', | | | 'min_disk': '0', | | | 'min_ram': '0', | | | 'size': '16338944'} | +--------------------------------+-----------------------------------------------------+ ==================================== [4] linstor r l ╭───────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ Node ┊ Port ┊ Usage ┊ Conns ┊ State ┊ ╞═══════════════════════════════════════════════════════════════════════════════════╡ ┊ CV_15c4fa98-76a5-44d6-9701-c89a8a848b82 ┊ cmp1 ┊ 7000 ┊ Unused ┊ Ok ┊ UpToDate ┊ ┊ CV_15c4fa98-76a5-44d6-9701-c89a8a848b82 ┊ cmp2 ┊ 7000 ┊ Unused ┊ Ok ┊ UpToDate ┊ ╰───────────────────────────────────────────────────────────────────────────────────╯ Note, that cmp3 is arbiter and has no 'by-disk' representation: root@cmp1:~# ls -lR /dev/drbd* brw-rw---- 1 root disk 147, 1000 May 21 11:42 /dev/drbd1000 /dev/drbd/by-disk/sds: lrwxrwxrwx 1 root root 17 May 21 11:42 CV_15c4fa98-76a5-44d6-9701-c89a8a848b82_00000 -> ../../../drbd1000 /dev/drbd/by-res/CV_15c4fa98-76a5-44d6-9701-c89a8a848b82: lrwxrwxrwx 1 root root 17 May 21 11:42 0 -> ../../../drbd1000 root@cmp2:~# ls -lR /dev/drbd* brw-rw---- 1 root disk 147, 1000 May 21 11:42 /dev/drbd1000 /dev/drbd/by-disk/sds: lrwxrwxrwx 1 root root 17 May 21 11:42 CV_15c4fa98-76a5-44d6-9701-c89a8a848b82_00000 -> ../../../drbd1000 /dev/drbd/by-res/CV_15c4fa98-76a5-44d6-9701-c89a8a848b82: lrwxrwxrwx 1 root root 17 May 21 11:42 0 -> ../../../drbd1000 root@cmp3:~# ls -lR /dev/drbd* brw-rw---- 1 root disk 147, 1000 May 21 11:42 /dev/drbd1000 /dev/drbd/by-res/CV_15c4fa98-76a5-44d6-9701-c89a8a848b82: lrwxrwxrwx 1 root root 17 May 21 11:42 0 -> ../../../drbd1000 ==================================== [5] root@cmp1:~# lvdisplay [ ... ] --- Logical volume --- LV Path /dev/sds/CV_15c4fa98-76a5-44d6-9701-c89a8a848b82_00000 LV Name CV_15c4fa98-76a5-44d6-9701-c89a8a848b82_00000 VG Name sds LV UUID jGzcIQ-yRpH-RKxP-ExsH-JSn6-kTFn-8kpKYO LV Write Access read/write LV Creation host, time cmp1, 2020-05-21 11:42:14 +0300 LV Pool name thin LV Status available # open 2 LV Size 5.00 GiB Mapped size 0.02% Current LE 1280 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:4 root@cmp2:~# lvdisplay [ ... ] --- Logical volume --- LV Path /dev/sds/CV_15c4fa98-76a5-44d6-9701-c89a8a848b82_00000 LV Name CV_15c4fa98-76a5-44d6-9701-c89a8a848b82_00000 VG Name sds LV UUID 8pVwMo-e3ed-Sy0w-06IZ-M7Mh-YVIL-BtKZEG LV Write Access read/write LV Creation host, time cmp2, 2020-05-21 11:42:14 +0300 LV Pool name thin LV Status available # open 2 LV Size 5.00 GiB Mapped size 0.02% Current LE 1280 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:4 ==================================== [6] Traceback of nova-compute: 3304 ERROR nova.compute.manager [] Traceback (most recent call last): 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2419, in _build_and_run_instance 3304 ERROR nova.compute.manager [] block_device_info=block_device_info) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3474, in spawn 3304 ERROR nova.compute.manager [] power_on=power_on) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6261, in _create_domain_and_network 3304 ERROR nova.compute.manager [] destroy_disks_on_failure) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 3304 ERROR nova.compute.manager [] self.force_reraise() 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 3304 ERROR nova.compute.manager [] six.reraise(self.type_, self.value, self.tb) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise 3304 ERROR nova.compute.manager [] raise value 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6230, in _create_domain_and_network 3304 ERROR nova.compute.manager [] post_xml_callback=post_xml_callback) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6164, in _create_domain 3304 ERROR nova.compute.manager [] guest.launch(pause=pause) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 143, in launch 3304 ERROR nova.compute.manager [] self._encoded_xml, errors='ignore') 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 3304 ERROR nova.compute.manager [] self.force_reraise() 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 3304 ERROR nova.compute.manager [] six.reraise(self.type_, self.value, self.tb) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise 3304 ERROR nova.compute.manager [] raise value 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 138, in launch 3304 ERROR nova.compute.manager [] return self._domain.createWithFlags(flags) 3304 ERROR nova.compute.manager [] File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 190, in doit 3304 ERROR nova.compute.manager [] result = proxy_call(self._autowrap, f, *args, **kwargs) 3304 ERROR nova.compute.manager [] File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 148, in proxy_call 3304 ERROR nova.compute.manager [] rv = execute(f, *args, **kwargs) 3304 ERROR nova.compute.manager [] File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 129, in execute 3304 ERROR nova.compute.manager [] six.reraise(c, e, tb) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise 3304 ERROR nova.compute.manager [] raise value 3304 ERROR nova.compute.manager [] File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 83, in tworker 3304 ERROR nova.compute.manager [] rv = meth(*args, **kwargs) 3304 ERROR nova.compute.manager [] File "/usr/lib/python3/dist-packages/libvirt.py", line 1110, in createWithFlags 3304 ERROR nova.compute.manager [] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 3304 ERROR nova.compute.manager [] libvirt.libvirtError: Cannot access storage file 'None': No such file or directory -- Volodymyr Litovka "Vision without Execution is Hallucination." -- Thomas Edison

1 0

[neutron][release] Missing tarballs for networking-mlnx, networking-l2gw and vmware-nsx
by Mark Goddard 21 May '20

21 May '20

Hi, There are no tarballs published for ussuri networking-mlnx or vmware-nsx. The most recently published tarballs were in February. Is this expected? networking-l2gw also has no ussuri tarballs, although master was updated recently. Thanks, Mark

7 8

[kolla] deprecating kolla-cli
by Mark Goddard 21 May '20

21 May '20

Hi, We previously planned to retire kolla-cli due to lack of interest, but at the last minute there was an offer to help support it. That support is no longer available, and we are not aware of the project having gained traction since our original decision. We are therefore adding a deprecation notice [1] to the Ussuri release notes, with a plan to remove it from governance in Victoria if there is no response to the call for help. [1] https://review.opendev.org/729855 Thanks, Mark

1 0

[keystone][ldap]
by Amjad Kotobi 20 May '20

20 May '20

Hi all, I’m integrating keystone with LDAP, and having “service account” e.g. Nova, keystone etc.. which are in database. As soon as connecting it to ldap all authentication getting failed, how can I have both “service account” and “LDAP users” connected to Keystone? Here is my keystone.conf ################### [ldap] url = ldap://XXXXX user = uid=XXX,cn=sysaccounts,cn=etc,dc=XXX,dc=de password = dkrzprox user_tree_dn = cn=users,cn=accounts,dc=XXX,dc=de user_objectclass = posixAccount user_id_attribute = uid user_name_attribute = uid user_allow_create = false user_allow_update = false user_allow_delete = false group_tree_dn = cn=groups,cn=accounts,dc=XXX,dc=de group_objectclass = groupOfNames group_id_attribute = cn group_name_attribute = cn group_member_attribute = member group_desc_attribute = description group_allow_create = false group_allow_update = false group_allow_delete = false use_pool = true use_auth_pool = true debug_level = 4095 query_scope = sub [identity] driver = ldap ##################### OS: Centos7 OpenStack-Release: Train Any idea or example of options gonna be great! Thank you

3 2

[keystone]
by Amjad Kotobi 20 May '20

20 May '20

Hi, I have keystone running in HA multiple nodes, and after reboot one of the node it started in logs to pop up below messages “""WARNING keystone.server.flask.application [req-fe993224-cc7e-4e5f-83a8-1e925e60b995 24dfa0beaaaa48848b4eee68e449f2de d1fecbdb49ef4871b94b201b7856eed3 - default default] Could not recognize Fernet token: TokenNotFound: Could not recognize Fernet token””” And in command line “”"Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-1145c696-6275-4c78-9486-a690b98548ed)””" During command lines when request lands on machine which hasn’t rebooted or so, the result comes but on the rest fore-mentioned message always in place. Openstack release Train, and deployed manually. How can I tackle it down? Any ideas? Thanks Amjad

2 1

[docs][all] Important changes in recent openstackdocstheme updates
by Andreas Jaeger 20 May '20

20 May '20

A couple of changes recently merged into openstackdocstheme to fix problems reported. These had some surprises in it and we'd like to inform you about the changes: * Config options are now prefixed with openstackdocs_, the old names will be removed in a future release * The 'project' config option is now only respected (and displayed in the left menu) if 'openstackdocs_auto_name = False' is set. By default, the theme uses the package name (from setup.cfg) * The HTML files show the version number by default (with exception of releasenotes and api docs) calculated from git. If you want to use your own version number or disable it, set 'openstackdocs_auto_version = False' and manually configure the 'version' and 'release' options. * Previously, the theme always used 'pygments_style = "native"' and overrode the setting of 'sphinx' that many repos have. Now the setting is respected. For a few repos this lead to unreadable code snippets. If you see this or want to go back to the previous theme, configure 'pygments_style = "native"'. * Many projects have written PDF documents. openstackdocstheme can now optionally link to them. Set 'openstackdocs_pdf_link' to True to show the icon with path. Note that the PDF file is placed on docs.openstack.org in the top of the html files while in check/gate it's in a separate PDF folder. Thus, the site preview will show in check/gate a broken link - but it works fine, check [2]. * Both reno (since version 3.1.0) and openstackdocsstheme are now declared parallel safe, the CI jobs automatically build releasenotes in parallel [1]. You can modify your local tox job to do this by adding the '-j auto' parameter to your 'sphinx-build' invocation. We're releasing openstackdocstheme version 2.2.1 soon with two further fixes: * PDF documents will now show the version number like html document, no need to configure versions in conf.py for this anymore [3]. * small bug fix (if you set auto_name = False in doc/source/conf.py, this hit so far 5 repos)[4]. Everything is documented in the documentation of openstackdocstheme [2]. If there are any questions, best ask in #openstack-oslo. Andreas has started pushing changes to update projects with topic:reno-openstackdocstheme. Hope that's all for Victoria on the openstackdocstheme, Stephen and Andreas [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014902.html [2] https://docs.openstack.org/openstackdocstheme/ [3] https://review.opendev.org/729554 [4] https://review.opendev.org/729031 -- Andreas Jaeger aj(a)suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB

1 0