August 2020 - openstack-discuss - lists.openstack.org

[tripleo] Overcloud without provisioning error: os-net-config command not found
by Marco Marino 06 Aug '20

06 Aug '20

Hi, I'm trying to deploy an overcloud using pre-provisioned nodes. I have only 2 nodes, 1 compute and 1 controller and here is the command I'm using: openstack overcloud deploy --templates --disable-validations -e /usr/share/openstack-tripleo-heat-templates/environments/deployed-server-environment.yaml -e /home/stack/templates/node-info.yaml -e /home/stack/templates/ctlplane-assignments.yaml -e /home/stack/templates/hostname-map.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -n /home/stack/os-deploy-custom-config/network_data.yaml --overcloud-ssh-user stack --overcloud-ssh-key /home/stack/.ssh/id_rsa Here is the content of custom files: (undercloud) [stack@undercloud ~]$ cat templates/node-info.yaml parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 1 ComputeCount: 1 (undercloud) [stack@undercloud ~]$ cat templates/ctlplane-assignments.yaml resource_registry: OS::TripleO::DeployedServer::ControlPlanePort: /usr/share/openstack-tripleo-heat-templates/deployed-server/deployed-neutron-port.yaml parameter_defaults: DeployedServerPortMap: controller-0-ctlplane: fixed_ips: - ip_address: 192.168.199.200 subnets: - cidr: 192.168.199.0/24 network: tags: 192.168.199.0/24 compute-0-ctlplane: fixed_ips: - ip_address: 192.168.199.210 subnets: - cidr: 192.168.199.0/24 network: tags: 192.168.199.0/24 (undercloud) [stack@undercloud ~]$ cat templates/hostname-map.yaml parameter_defaults: HostnameMap: overcloud-controller-0: controller-0 overcloud-novacompute-0: compute-0 http://paste.openstack.org/show/796634/ <-- Here is the complete output for overcloud deploy command. It seems that the error is /var/lib/tripleo-config/scripts/run_os_net_config.sh: line 59: os-net-config: command not found" os-net-config is provided by "delorean-component-tripleo" repository. So my question is: should I pre install Openstack repositories on pre-provisioned nodes in addition to operating system installation and network configuration? Thank you, Marco

2 1

[horizon] Victoria virtual mid-cycle poll
by Ivan Kolodyazhny 06 Aug '20

06 Aug '20

Hi team, As discussed at Horizon's Virtual PTG [1], we'll have a virtual mid-cycle meeting around Victoria-2 milestone. We'll discuss Horizon current cycle development priorities and the future of Horizon with modern JS frameworks. Please indicate your availability to meet for the first session, which will be held during the week of July 27-31: https://doodle.com/poll/3neps94amcreaw8q Please respond before 12:00 UTC on Tuesday 4 August. [1] https://etherpad.opendev.org/p/horizon-v-ptg Regards, Ivan Kolodyazhny, http://blog.e0ne.info/

1 3

[cinder][nova] Local storage in compute node
by Eric K. Miller 06 Aug '20

06 Aug '20

Hi, I'm research methods to get around high storage latency for some applications where redundancy does not matter, so using local NVMe drives in compute nodes seems to be the practical choice. However, there does not appear to be a good solution from what I have read. For example, BlockDeviceDriver has been deprecated/removed, LVM is only supported via iSCSI (which is slow) and localization of LVM volumes onto the same compute node as VMs is impossible, and other methods (PCI pass-through, etc.) would require direct access to the local drives, where device cleansing would need to occur after a device was removed from a VM, and I don't believe there is a hook for this. Ephemeral storage appears to be an option, but I believe it has the same issue as PCI pass-through, in that there is no abiilty to automatically cleanse a device after it has been used. In our default configuration, ephemeral storage is redirected to use Ceph, which solves the cleansing issue, but isn't suitable due to its high latency. Also, ephemeral storage appears as a second device, not the root disk, so that complicates a few configurations we have. Is there any other way to write an operating system image onto a local drive and boot from it? Or preferably assign an LVM /dev/mapper path as a device in libvirt (no iSCSI) after configuring a logical volume? or am I missing something? Thanks! Eric

4 14

[ops] [keystone] "Roles are not immutable"
by Massimo Sgaravatto 06 Aug '20

06 Aug '20

After updating to Train from rocky (on stein we just performed the db-sync), we tried the new "keystone-status update check" command which says that the admin role is not immutable [*]. As far as I understand this is something that was done to prevent deleting/modifying the default roles (that could cause major problems). But how am I supposed to fix this? The "--immutable" option for the "openstack role set" command, documented at: https://docs.openstack.org/python-openstackclient/latest/cli/command-object… is not available in Train. Thanks, Massimo [*] +-------------------------------------------+ | Check: Check default roles are immutable | | Result: Failure | | Details: Roles are not immutable: admin | +-------------------------------------------+

1 0

[tripleo] Deploy overcloud without provisioning
by Marco Marino 06 Aug '20

06 Aug '20

Hi, I'm trying to deploy an overcloud using tripleo with pre provisioned nodes. My configuration is quite simple: - 1 controller and 1 compute nodes on which I already installed CentOS 8.2 - Both nodes have a dedicated idrac interface with an ip in 192.168.199.0/24. Please note that this interface is not visible with "ip a" or "ifconfig". It's a dedicated IDRAC interface - Both nodes have a NIC configured in the subnet 192.168.199.0/24 (192.168.199.200 and 192.168.199.201) - Undercloud uses 192.168.199.0/24 as pxe/provisioning network (but I don't need provisioning) Question: should I import nodes with "openstack overcloud node import nodes.yaml" even if I don't need the provisioning step? Furthermore, on the undercloud I created one file: /home/stack/templates/node-info.yaml with the following content parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 1 ComputeCount: 1 Question: How can I specify that "node X with ip Y should be used as a controller and node Z with ip K should be used as a compute"?? Should I set the property with the following command? openstack baremetal node set --property capabilities='profile:control' controller1 Thank you, Marco

2 3

[horizon] patternfly?
by Mohammed Naser 06 Aug '20

06 Aug '20

Hi everyone, I was wondering if anyone in the community has explored the idea of implementing PatternFly inside Horizon. It seems like it shares a lot of the similar ideas that we use and we could really benefit from using a common library that already exists with a lot of good UX thought behind it. I know it's based on React which is a bit of a leap from where Horizon is today. However, I'd be curious if the Horizon team is interested in figuring out a plan to make a migration to something like this happen. Personally, I think I would be able to provide resources to have someone do this work. However, if this seems like a huge stretch and architecture change where it actually makes more sense to stand this up from scratch (and implement an architecture where the dashboard talks directly to the APIs?), perhaps we should explore that. I also would like to hear more from our extended community too, as I think we really need to improve our user interface experience. Thanks, Mohammed -- Mohammed Naser VEXXHOST, Inc.

11 31

[TC] [PTG] Victoria vPTG Summary of Conversations and Action Items
by Kendall Nelson 06 Aug '20

06 Aug '20

Hello Everyone! I hope you all had a productive and enjoyable PTG! While it’s still reasonably fresh, I wanted to take a moment to summarize discussions and actions that came out of TC discussions. If there is a particular action item you are interested in taking, please reply on this thread! For the long version, check out the etherpad from the PTG[1]. Tuesday ====== Ussuri Retrospective ---------------------------- As usual we accomplished a lot. Some of the things we accomplished were around enumerating operating systems per release (again), removing python2 support, and adding the ideas repository. Towards the end of the release, we had a lot of discussions around what to do with leaderless projects, the role of PTLs, and what to do with projects that were missing PTL candidates for the next release. We discussed office hours, their history and reason for existence, and clarified how we can strengthen communication amongst ourselves, the projects, and the larger community. TC Onboarding -------------------- It was brought up that those elected most recently (and even new members the election before) felt like there wasn’t enough onboarding into the TC. Through discussion about what we can do to better support returning members is to better document the daily, weekly and monthly tasks TC members are supposed to be doing. Kendall Nelson proposed a patch to start adding more detail to a guide for TC members already[2]. It was also proposed that we have a sort of mentorship or shadow program for people interested in joining the TC or new TC members by more experienced TC members. The discussion about the shadow/mentorship program is to be continued. TC/UC Merge ------------------ Thierry gave an update on the merge of the committees. The simplified version is that the current proposal is that UC members are picked from TC members, the UC operates within the TC, and that we are already setup for this given the number of TC members that have AUC status. None of this requires a by-laws change. One next step that has already begun is the merging of the openstack-users ML into openstack-discuss ML. Other next steps are to decide when to do the actual transition (disbanding the separate UC, probably at the next election?) and when to setup AUC’s to be defined as extra-ATC’s to be included in the electorate for elections. For more detail, check out the openstack-discuss ML thread[3]. Wednesday ========= Help Wanted List ----------------------- We settled on a format for the job postings and have several on the list. We talked about how often we want to look through, update or add to it. The proposal is to do this yearly. We need to continue pushing on the board to dedicate contributors at their companies to work on these items, and get them to understand that it's an investment that will take longer than a year in a lot of cases; interns are great, but not enough. TC Position on Foundation Member Community Contributions ---------------------------------------------------------------------------------- The discussion started with a state of things today - the expectations of platinum members, the benefits the members get being on the board and why they should donate contributor resources for these benefits, etc. A variety of proposals were made: either enforce or remove the minimum contribution level, give gold members the chance to have increased visibility (perhaps giving them some of the platinum member advantages) if they supplement their monetary contributions with contributor contributions, etc. The #ACTION that was decided was for Mohammed to take these ideas to the board and see what they think. OpenStack User-facing APIs -------------------------------------- Users are confused about the state of the user facing API’s; they’ve been told to use the OpenStackClient(OSC) but upon use, they discover that there are features missing that exist in the python-*clients. Partial implementation in the OSC is worse than if the service only used their specific CLI. Members of the OpenStackSDK joined discussions and explained that many of the barriers that projects used to have behind implementing certain commands have been resolved. The proposal is to create a pop up team and that they start with fully migrating Nova, documenting the process and collecting any other unresolved blocking issues with the hope that one day we can set the migration of the remaining projects as a community goal. Supplementally, a new idea was proposed- enforcing new functionality to services is only added to the SDK (and optionally the OSC) and not the project’s specific CLI to stop increasing the disparity between the two. The #ACTION here is to start the pop up team, if you are interested, please reply! Additionally, if you disagree with this kind of enforcement, please contact the TC as soon as possible and explain your concerns. PTL Role in OpenStack today & Leaderless Projects --------------------------------------------------------------------- This was a veeeeeeeerrrry long conversation that went in circles a few times. The very short version is that we, the TC, are willing to let project teams decide for themselves if they want to have a more deconstructed kind of PTL role by breaking it into someone responsible for releases and someone responsible for security issues. This new format also comes with setting the expectation that for things like project updates and signing up for PTG time, if someone on the team doesn’t actively take that on, the default assumption is that the project won’t participate. The #ACTION we need someone to take on is to write a resolution about how this will work and how it can be done. Ideally, this would be done before the next technical election, so that teams can choose it at that point. If you are interested in taking on the writing of this resolution, please speak up! Cross Project Work ------------------------- -Pop Up Teams- The two teams we have right now are Encryption and Secure Consistent Policy Groups. Both are making slow progress and will continue. -Reducing Community Goals Per Cycle- Historically we have had two goals per cycle, but for smaller teams this can be a HUGE lift. The #ACTION is to clearly outline the documentation for the goal proposal and selection process to clarify that selecting only one goal is fine. No one has claimed this action item yet. -Victoria Goal Finalization- Currently, we have three proposals and one accepted goal. If we are going to select a second goal, it needs to be done ASAP as Victoria development has already begun. All TC members should review the last proposal requesting selection[4]. -Wallaby Cycle Goal Discussion Kick Off- Firstly, there is a #ACTION that one or two TC members are needed to guide the W goal selection. If you are interested, please reply to this thread! There were a few proposed goals for VIctoria that didn’t make it that could be the starting point for W discussions, in particular, the rootwrap goal which would be good for operators. The OpenStackCLI might be another goal to propose for Wallaby. Detecting Unmaintained Projects Early --------------------------------------------------- The TC liaisons program had been created a few releases ago, but the initial load on TC members was large. We discussed bringing this program back and making the project health checks happen twice a release, either the start or end of the release and once in the middle. TC liaisons will look at previously proposed releases, release activity of the team, the state of tempest plugins, if regular meetings are happening, if there are patches in progress and how busy the project’s IRC channel is to make a determination. Since more than one liaison will be assigned to each project, those liaisons can divvy up the work how they see fit. The other aspect that still needs to be decided is where the health checks will be recorded- in a wiki? In a meeting and meeting logs? That decision is still to be continued. The current #ACTION currently unassigned is that we need to assign liaisons for the Victoria cycle and decide when to do the first health check. Friday ===== Reducing Systems and Friction to Drive Change ---------------------------------------------------------------- This was another conversation that went in circles a bit before realizing that we should make a list of the more specific problems we want to address and then brainstorm solutions for them. The list we created (including things already being worked) are as follows: - TC separate from UC (solution in progress) - Stable releases being approved by a separate team (solution in progress) - Making repository creation faster (especially for established project teams) - Create a process blueprint for project team mergers - Requirements Team being one person - Stable Team - Consolidate the agent experience - Figure out how to improve project <--> openstack client/sdk interaction. If you feel compelled to pick one of these things up and start proposing solutions or add to the list, please do! Monitoring in OpenStack (Ceilometer + Telemetry + Gnocchi State) ----------------------------------------------------------------------------------------- This conversation is also ongoing, but essentially we talked about the state of things right now- largely they are not well maintained and there is added complexity with Ceilometers being partially dependent on Gnocchi. There are a couple of ideas to look into like using oslo.metrics for the interface between all the tools or using Ceilometer without Gnocchi if we can clean up those dependencies. No specific action items here, just please share your thoughts if you have them. Ideas Repo Next Steps ------------------------------- Out of the Ussuri retrospective, it was brought up that we probably needed to talk a little more about what we wanted for this repo. Essentially we just want it to be a place to collect ideas into without worrying about the how. It should be a place to document ideas we have had (old and new) and keep all the discussion in one place as opposed to historic email threads, meetings logs, other IRC logs, etc. We decided it would be good to periodically go through this repo, likely as a forum session at a summit to see if there is any updating that could happen or promotion of ideas to community goals, etc. ‘tc:approved-release’ Tag --------------------------------- This topic was proposed by the Manila team from a discussion they had earlier in the week. We talked about the history of the tag and how usage of tags has evolved. At this point, the proposal is to remove the tag as anything in the releases repo is essentially tc-approved. Ghanshyam has volunteered to document this and do the removal. The board also needs to be notified of this and to look at projects.yaml in the governance repo as the source of truth for TC approved projects. The unassigned #ACTION item is to review remaining tags and see if there are others that need to be modified/removed/added to drive common behavior across OpenSack components. Board Proposals ---------------------- This was a pretty quick summary of all discussions we had that had any impact on the board and largely decided who would mention them. Session Feedback ------------------------ This was also a pretty quick topic compared to many of the others, we talked about how things went across all our discussions (largely we called the PTG a success) logistically. We tried to make good use of the raising hands feature which mostly worked, but it lacks context and its possible that the conversation has moved on by the time it’s your turn (if you even remember what you want to say). OpenStack 2.0: k8s Native ----------------------------------- This topic was brought up at the end of our time so we didn’t have time to discuss it really. Basically Mohammed wanted to start the conversation about adding k8s as a base service[5] and what we would do if a project proposed required k8s. Adding services that work with k8s could open a door to new innovation in OpenStack. Obviously this topic will need to be discussed further as we barely got started before we had to wrap things up. So. The tldr; Here are the #ACTION items we need owners for: - Start the User Facing API Pop Up Team - Write a resolution about how the deconstructed PTL roles will work - Update Goal Selection docs to explain that one or more goals is fine; it doesn’t have to be more than one - Two volunteers to start the W goal selection process - Assign two TC liaisons per project - Review Tags to make sure they are still good for driving common behavior across all openstack projects Here are the things EVERYONE needs to do: - Review last goal proposal so that we can decide to accept or reject it for the V release[4] - Add systems that are barriers to progress in openstack to the Reducing Systems and Friction list - Continue conversations you find important Thanks everyone for your hard work and great conversations :) Enjoy the attached (photoshopped) team photo :) -Kendall (diablo_rojo) [1] TC PTG Etherpad: https://etherpad.opendev.org/p/tc-victoria-ptg [2] TC Guide Patch: https://review.opendev.org/#/c/732983/ [3] UC TC Merge Thread: http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014736.html [4] Proposed V Goal: https://review.opendev.org/#/c/731213/ [5] Base Service Description: https://governance.openstack.org/tc/reference/base-services.html

3 2

Re: [nova] If any spec freeze exception now?
by yumeng bao 05 Aug '20

05 Aug '20

Hi gibi and all, I wanna mention the SRIOV SmartNIC Support Spec https://review.opendev.org/#/c/742785 This spec is proposed based on feedback from our PTG discussion, yet there are still open questions need to be nailed down. Since this spec involves nova neutron and cyborg, it will probably take a long time to get ideas from different aspects and reach an agreement. Can we keep this as an exception and keep review it to reach closer to an agreement? Hopefully we can reach an agreement in Victoria, and start to land in W. Xinran and I were trying to attend nova’s weekly meeting to discuss this spec, but the time too late for us. :( We will find if there is any other way to sync and response more actively to all your comments and feedback. And just to point out, nova operations support are still one of cyborg’s high priority goals in Victoria, we will keep focus on it and won’t sacrifice time of this goal. Regards， Yumeng

3 3

[swift][ceph] Container ACLs don't seem to be respected on Ceph RGW
by Jason Anderson 05 Aug '20

05 Aug '20

Hi all, Just scratching my head at this for a while and though I’d ask here in case it saves some time. I’m running a Ceph cluster on the Nautilus release and it’s running Swift via the rgw. I have Keystone authentication turned on. Everything works fine in the normal case of creating containers, uploading files, listing containers, etc. However, I notice that ACLs don’t seem to work. I am not overriding "rgw enforce swift acls”, so it is set to the default of true. I can’t seem to share a container or make it public. (Side note, confusingly, the Ceph implementation has a different syntax for public read/write containers, ‘*’ as opposed to ‘*:*’ for public write for example.) Here’s what I’m doing (as admin) swift post —write-acl ‘*’ —read-acl ‘*’ public-container swift stat public-container Account: v1 Container: public-container Objects: 1 Bytes: 5801 Read ACL: * Write ACL: * Sync To: Sync Key: X-Timestamp: 1595883106.23179 X-Container-Bytes-Used-Actual: 8192 X-Storage-Policy: default-placement X-Storage-Class: STANDARD Last-Modified: Wed, 05 Aug 2020 03:42:11 GMT X-Trans-Id: tx000000000000000662156-005f2a2bea-23478-default X-Openstack-Request-Id: tx000000000000000662156-005f2a2bea-23478-default Accept-Ranges: bytes Content-Type: text/plain; charset=utf-8 (as non-admin) swift upload public-container test.txt Warning: failed to create container 'public-container': 409 Conflict: BucketAlreadyExists Object HEAD failed: https://ceph.example.org:7480/swift/v1/public-container/README.md 403 Forbidden swift list public-container Container GET failed: https://ceph.example.org:7480/swift/v1/public-container?format=json 403 Forbidden [first 60 chars of response] b'{"Code":"AccessDenied","BucketName”:”public-container","RequestId":"tx0' Failed Transaction ID: tx000000000000000662162-005f2a2c2a-23478-default What am I missing? Thanks in advance! /Jason

1 2

[tc][ptl] Proposal: Distributed Project Leadership
by Nate Johnston 05 Aug '20

05 Aug '20

The governing structure for OpenStack projects has long been for a Project Technical Lead (PTL) to be elected to serve as a singular focus for that project. While the PTL role varies significantly from project to project, the PTL has many responsibilities for managing the development and release process for a project as well as representing the project both internally and externally. There have been a number of projects that have expressed an interest in functioning in a mode that would not require a PTL, but would rather devolve the responsibilities of the PTL into various liaison roles, that could in turn be held by one or more contributors. This topic was discussed by the TC at the recent virtual PTG, and we now have a proposal to put forth to the community for comment. Jean-Phillipe Evrard and I worked up a more detailed proposal for everyone to comment on and review. We are calling this the 'distributed project leadership model'. Most importantly, this is an opt-in process where projects interested in pursuing a distributed project leadership model should opt in to it, but for projects satisfied with the status quo nothing would change. I encourage everyone who is interested to examine the proposal and comment: https://review.opendev.org/744995 Thank you, Nate Johnston

1 0