October 2021 - openstack-discuss - lists.openstack.org

[kolla-ansible][neutron]BGP Problems
by Franck VEDEL 27 Oct '21

27 Oct '21

Hello, I'm trying to get BGP / BGP speaker / DRAgent to work with Neutron on one side and a Cisco router on the other. I work under Centos / Kolla-ansible / Wallaby. I adapt this technical documentation (https://docs.openstack.org/neutron/latest/admin/config-bgp-dynamic-routing.…) to my configuration but I am having difficulties (the cisco router sends a TCP SYN frame to the router in Openstack, TCP port 179, but the router sends back an RST. I don't understand why, no security on the router port). Do you have any links to help, tutorials, that would be of great help to me. Thanks in advance. Franck

2 2

[openstack-ansible][osa][ptg] Yoga PTG summary
by Dmitriy Rabotyagov 27 Oct '21

27 Oct '21

1 0

Devstack stack run issue
by Venkata Krishna Reddy 27 Oct '21

27 Oct '21

3 3

[tc][all][ Yoga Virtual PTG Summary
by Ghanshyam Mann 27 Oct '21

27 Oct '21

Hello Everyone, I am summarizing the Technical Committee discussion that happened in the Yoga cycle PTG last week. Recordings: Day1: https://www.youtube.com/watch?v=uQRKYummsHM Day2: https://www.youtube.com/watch?v=IHj7cpBlbYo Day3: https://www.youtube.com/watch?v=RT492bi6Xto TC + Community leaders interaction ------------------------------------------ This is our first session in PTG to interact with community leaders and ask for feedback on TC. Below are the topics we discussed in this feedback session * What is TC responsibility: This is the first feedback we started this session, to discuss what is the role of TC other than accepting/ removing the projects and community-wide goals. does TC need to have a strong vision and start working on that? Why not TC do more technical work than spend time on composing resolutions or so? All those were valid points and we all agreed that TC should be doing more technical work and providing more guidance to the community on the cross-project works for example RBAC, Unified limit (more of similar way API SIG is doing on API side). * Improving community-wide goal process: Another feedback was on the effectiveness of community-wide goals. A few of the community-wide goals were not ready when they were selected. Even a few like pdf goal changed the implementation in milestone-2 and projects have to update the previous implementation. Or sometimes we try too much to do instead of small steps like IPV6 testing. With all feedback, we agreed to have a more concrete checklist when we select any goal and also check project bandwidth to complete the work. * Fewer contributors: Fewer contributors issue is from many projects, to solve that at some extent, TC will encourage projects to apply in outreachy/mentorship programs where few projects have received a good amount of help. * Improving CI: Improving CI is the next topic which we discussed in TC slots on Thursday. One key topic in this was "Request to be able to recheck on a single test instead of the whole check test" as many projects like Ironic gate has more often neutron failure and they have to recheck the complete set of jobs. The issue can be from DHCP, memory etc. We do not think allowing job-specific recheck is the right direction here and instead, we should encourage projects to solve the failures we get in the gate which can improve the code stability. Also, we should encourage projects to monitor the gate failures. >From the above discussion, we are taking below two action items for TC: 1. TC to start composing the technical guidelines on various topics like 'unified limit' etc similar way API SIG has done. 2. TC to add a more specific checklist for community-wide goal readiness and make them more successful as they are currently. TC interop sync on guidelines ----------------------------------- We discussed interop tests requirements. One of the points brought that if starter kits impact interop guidelines to consider but it does not. Can cinder be separated from "OpenStack Powered Compute" if it can be used as standalone? The Interop team need to check if there are cloud without cinder and how they apply for interop certification. On tests requirement by interop, our current process stays same that if interop defined the capabilities in new guidelines then the test writing can be requested or even an interop group can propose the test in Tempest or Tempest plugins. Technical Writing (doc) SIG need a chair and more maintainers ----------------------------------------------------------------------- We are merging this SIG repo into TC responsibility. There are not many activities nowadays on this SIG or repo it own so it should not impact the TC work as such. We agreed to distribute this SIG repo in * TC: api-site, constellations, and openstack-manuals repos under * FC SIG: contributor-guide, training-guides, and upstream-institute-virtual-environment * For training-labs which is for COA, I will check with Foundation on ownership or it. Thanks, Stephen for serving as chair for this SIG. Yoga testing runtime: Review & Update if needed ---------------------------------------------------------- We discussed/iterate over the Yoga testing runtime on new distro and python versions to tests. Distro to tests: * We agreed on including "Debian Bullseye" as a distro in the testing runtime. * centos9-stream release time is not clear so we are keeping centos8-stream for now. * centos8 is planned to be EOL at end of 2021, If you see failure on or stable branch testing, then you can update it to centos8-stream (devstack support is already there) But we do not need to change the testing runtime for stable branches as such. Python Versions to test: * py3.8 as voting (no change from currently tested) * py3.9 as voting * py3.10 as non voting. * keep py3.6 until we move from centos8-stream to centos9-stream I will update the testing runtime doc and job template for that. TC tags review ------------------ You might have seen the email from yoctozepto about a query on TC tag usage[1]. As we did not get any response in the email, we did final checks in PTG. We agreed to start the tag framework removal proposal on ML and if there is no objection for a week or so then we will remove it. Improvement in project governance ------------------------------------------ In the past couple of cycles, we have seen few projects that are not so active or not getting more attention from users. This discussion was to brainstorm the process to keep monitoring such projects and especially when there is a new project application how we can monitor those in initial time to check if they have all the required things working to stay as official OpenStack project. The first thing we discussed introducing the different levels of the project something similar to CNCF (sandbox, incubated, graduated) or OpenStack used to have (incubation -> incubated projects). But it is not known if these levels will work well in our governance. With the low cost to maintain projects in OpenStack governance (as long as they are active). Being an OpenStack official project is one way to give that project more visibility in the market and attract users/contributors. With all these points, we agreed not to introduce the different levels in the project hierarchy but we will introduce a new process of 'Tech Pre-review' for new project applications so that we can monitor the new projects if they are doing good and all minimum required things are working smoothly. As the next step, we will be working to define the "Tech Pre-review" checklist and timeline etc. Release name process change ----------------------------------- We want to discuss two points 1. allow community members to vote on release name 2. some pre-trademark (non-paid) checks, in this topic but due to time limit we discussed first only. As you might know, we changed the electorate on release naming from community to TC which has been objected to in the past when we changed it and during the naming voting also in the past couple of cycles. We have simplified the name proposal criteria in the past which solved the issue that occurred in past on TC removing the name. We agreed on: * The community can propose names that meet the convention. * The TC will ensure that the name is not offensive based on community feedback. * Allow community members to vote on the proposed names. * After Z cycle, go back to 'A' and go through the alphabet (obviously don't re-use names). Pain Point targeting ----------------------- Two key pain points we collected during TC weekly meetings were 1. Rabbitmq 2. OpenStackClient support. But the etherpad has a lot more pain points not only from projects but also from operator or project has collected those from operators. We need to iterate the list again which we will continue doing in TC weekly meetings. TC members will encourage projects to fix or have a look into those pain points if the operator has added them. TC also can help a few of the pain points to solve consistently by defining the technical guidelines for such common pain points. As the next step, we will continue reviewing the list in etherpad[2]. 'Skyline' application to become an official OpenStack Project ------------------------------------------------------------------------ We discussed the new project application Skyline which is a dashboard project for OpenStack. One of the key points raised is about non-integrated projects UI-plugins (like every project has horizon plugins for their UI). How and who is going to implement the new UI plugins on the project's side? Also, there is more work needed on repos and python packaging side for which discussion is going in separate mailing thread[3]. Overall, we are ok to proceed with the proposal once the python package/repos things are set up correctly. Discussion on new RBAC ------------------------------ There were a lot of discussions on the new RBAC on various project sessions. it was clear from the discussion or points raised in those sessions that we are not using the system scope in the way it should be used. We should keep system-level operation only for system scoped and project level for the project. If any system users want to perform the project level operation then they can request the project token and then request. Also, 'admin' term at all levels domain, system, and the project is the most confusing term and we would like to change that to 'manager' or so. The discussion is not yet over and due to the time limit, we wanted to continue it after PTG too. As a summary, we selected this (current way) as Yoga cycle goal which we will re-iterate. I will set up the call to continue the discussion and send on openstack-discuss mailing list. For a detailed summary, please refer to L122 in policy popup etherpad[4]. Community-wide goals ---------------------------- Based on the feedback received in TC+community leaders sessions, we would like to make sure we have the proper checklist to know this goal is ready to start (in terms of implementation details as well as projects bandwidth). Rico and I had worked on such a checklist in the past but did not push for review which we should do now. We are also restructuring the community-wide goal timeline by decoupling them from the release cycle. At a time we will select one or more goals to start with one or more milestones. They can be multi-cycle goals or single based on the goal nature and work it needs. This way we will make sure that we complete the work needed to be done for that goal irrespective of the current cycle release. Xena Retrospective ----------------------- What went well? * Xena cycle tracker status: Completed the 9 items out of 13, others in progress. * Think having the weekly meetings went well instead of just office hours. * Got good feedback from the community about what they expect from the TC. * We have been good at keeping our open reviews low. What to change or any other feedback? * Be more technical, provide more technical guidance. * Get a feedback loop from the community and work on that feedback. Pop Up Team Check In ---------------------------- * Secure RBAC / Policy As discussed in RBAC discussion, there are a lot of work still needed along with community-wide goal. We decided to continue this pop-up team for the Yoga cycle at least. * Encryption Slow progress but still more work to do (waiting on changes in Barbican). The Team is hoping for more progress in the coming cycle. We will continue this pop-up team. Meeting check ----------------- We will continue the weekly meeting at the same time they are currently and also continue the monthly video call. We will continue using google meet until we figure out the recording things in meetpad or so. We did not discuss or check on office hours whether to continue those or not which we will continue in weekly meetings. User Survey Feedback Responses --------------------------------------- We will start working on analyzing the feedback from the user survey. Jay will summarize the feedback and then we will start composing the action items accordingly. k8s Steering Committee Cross community Discussions ----------------------------------------------------------------- This is our last hour of the PTG. This is cross-community sessions we continue organizing in PTG. From Kubernetes steering committee, we had Dims, Bob, Christoph joined along with OpenStack TC and a few non-tc members. The first topic we talked about is the different levels in the CNCF ladder and project confirmation process. This area is mostly taken care of by CNCF and Kubernetes community more concentrate on the technical part. Like OpenStack, Kubernetes is also seeing a slow down in project/SIG activities especially on community leaders. Kubernetes is trying mentor/cohort and more distributed roles by documenting the responsibility of that role that will help contributors to discuss it with their employer. OpenStack Upstream investment opportunity is one of the things we discussed and how successful that is. One of the benefits of this is we can communicate the help-needed things on various platforms/forums. ELK is one (the only as of now) example of success. Like OpenStack, Kubernetes also lacks in CI/CD resourcing and funding. This seems like a common problem in many OSS communities. Yoga cycle tracker ---------------------- Like we did in Xena cycle, TC is defining the cycle-wise tracker to complete the set of things TC would like to work on. For the Yoga cycle also we will continue with the tracker. Based on the PTG discussion, we have collected 9 items for the Yoga cycle to complete. I have captured those in the below etherpad which will be using it for progress tracking also. - https://etherpad.opendev.org/p/tc-yoga-tracker [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-September/02480… [2] https://etherpad.opendev.org/p/pain-point-elimination [3] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025397.… [4] https://etherpad.opendev.org/p/policy-popup-yoga-ptg [5] https://etherpad.opendev.org/p/tc-xena-tracker -gmann

1 0

[TripleO] PTG Summary
by James Slagle 26 Oct '21

26 Oct '21

TripleO had PTG sessions on Monday, Tuesday, and Wednesday, followed by a Directord+Task-Core hackfest on Thursday. The main etherpad for the PTG is at https://etherpad.opendev.org/p/tripleo-yoga-topics (includes links to recordings) Overall Summary =============== Our sessions were well attended with good cross collaboration from external stakeholders (storage, network, compute, etc). We had a good mix of topics, which I felt helped maintain interest and engagement. Topics ranged from future proposals, integrations, CI, and infrastructure. Our attendance was between 30-40 for most sessions. The underlying theme of the week was about the proposed migration from Ansible to Directord+task-core. Overall, I felt we accomplished clarifying the scope of the proposed change and articulating the reasons why the change is necessary, as well as understanding the benefits. With the heavy current investment in Ansible specific components, we emphasized the need for migration tooling wherever possible. No significant objections were raised about the approach. However, we settled on the next steps being agreement in the spec, and agreement around the end to end integration in TripleO. We continued to tie most other topics we discussed back to this topic to make sure we were considering the future impacts. The hackfest on Thursday further helped to understand the proposal and illustrated several key concepts that are presently missing in TripleO. Namely, those of dynamic service dependency management, job result caching and fingerprinting, and the messaging architecture. We had around 30 participants during the hackfest. We took a team photo on Thursday which we can be seen at https://slagle.fedorapeople.org/TripleO-Yoga-PTG-Team-Photo.png Thanks for the participation everyone! Sessions ======== Monday ------ 1310-1350 Directord/task-core introduction and overview https://etherpad.opendev.org/p/tripleo-directord-task-core The overview of Directord and task-core was presented, and we clarified the scope of the change, which is primarily about replacing Ansible in TripleO. As part of the change, we will need to migrate from all Ansible specific implementations (playbooks, roles, tasks, modules, actions, etc) to pure Python or Directord orchestrations and components. We emphasized the need for migration tooling wherever possible. No major objections were raised, and I felt the team understood the proposed benefits. Next steps are to finish the spec and continue on the TripleO integration. 1405-1445 task-core task graphs and execution concepts https://etherpad.opendev.org/p/tripleo-task-core-execution Further illustrated the benefits and functionality gained with task-core, particularly around the dynamic dependency ordering for tasks. Explained the benefits for services with a lot of external deployment tasks, such as Octavia. 1500-1540 (CI) Tripleo Health https://etherpad.opendev.org/p/tripleo-ptg-tripleo-health Really great work by the CI team on http://ci-health.tripleo.org/ and https://opendev.org/openstack/tripleo-ci-health-queries. We need to get the whole team committed to maintaining the queries for the most benefit. 1540-1620 OS Migrate update/feedback session https://etherpad.opendev.org/p/yoga-os-migrate Update on new features added to os-migrate, and version/compatibility testing. Tuesday ------- 1310-1350 ceph integration https://etherpad.opendev.org/p/tripleo-ceph-yoga Plan to remove support for ceph-ansible, and add cephadm ingress with haproxy+keepalived. Upgrade challenges around migrating from PCS to ingress. We also need to keep ceph integration as a priority and proving ground for migration to Directord+task-core. 1405-1445 healthchecks https://etherpad.opendev.org/p/tripleo-healthchecks-yoga Participation from Sean Mooney from the Nova team to present the proposed solution in Nova. Agreement this would work well for TripleO. We need to continue the collaboration and participate in the pending Nova spec to help drive the healthcheck model. 1500-1540 (HA) Pacemaker brainstorming https://etherpad.opendev.org/p/tripleo-yoga-pacemaker General consensus is that we still need pacemaker for some tasks. But, we need to evaluate each usage on a case by case basis and migrate services out of pacemaker where there is opportunity. 1540-1620 FIPS Updates & Secret Management https://etherpad.opendev.org/p/yoga-tripleo-fips Plan to go forward with the spec for secret management in TripleO. Wednesday --------- 1310-1350 (CI) tripleo-repos updates https://etherpad.opendev.org/p/ci-tripleo-repos Update on tripleo-repos and new functionality (yum config, tripleo-get-hash). Covered ansible integration (for CI). 1405-1445 (CI) Update on CI and centos9 prep work https://etherpad.opendev.org/p/centos-stream-9-upstream Discussed needed CI jobs and CentOS 9 timeline. 1500-1540 Dependency clashes installing branched and non-branches repos in Zuul jobs https://etherpad.opendev.org/p/branched-unbranched-dependency-clashes Discussed limiting the usage of upper constraints as the overall approach. 1540-1620 RDO Releases Agreed that providing the list of last tested hashes to RDO for releases that TripleO doesn't support is sufficient. We reviewed the RDO/TripleO support statement around intermediary releases that TripleO doesn't plan to maintain. https://review.rdoproject.org/r/c/rdo-website/+/36342 Hackfest ======== The team participated in the hackfest by following through the etherpad at https://etherpad.opendev.org/p/tripleo-directord-hackfest. Several participants were able to get the environment up. We were able to work through various issues as they arose. Good discussion around UX improvements that are needed, and the apparent benefits for different services due to the dynamic dependencies, job caching, execution time, etc. Also shared the POC of the end to end integration within TripleO. -- -- James Slagle --

1 0

Guest's secondary/virtual IP
by lejeczek 26 Oct '21

26 Oct '21

Hi guys. What I expected turns out not to be enough, must be something trivial - what am I missing? I set a port with --allowed-address and on the instance/guest using the port I did: -> $ ip add add 10.0.1.99/24 dev eth1 yet that IP other guest cannot reach. many thanks, L.

2 2

[skyline][tc] October 2021 - PTG Summary
by Boxiang Zhu 26 Oct '21

26 Oct '21

Hi, Well, it's Skyline Team's first time to attend the PTG : ) Thank you for your attention and discussion of Skyline. And welcome more friends to join us! You can see all the notes in a read-only etherpad here: - skyline etherpad: https://etherpad.opendev.org/p/r.b44d07e6504a4b38514b2717f01aca62 - tc etherpad: https://etherpad.opendev.org/p/r.a07136b01df95515f32c4cb779faa45c#L395 ## reorganize to one Python package per Git repository Now skyline-apiserver has divided the source code into six sections: - skyline-apiserver: core source code of skyline apiserver - skyline-config: config library with yaml to parse config file - skyline-console: git submodule of skyline-console - skyline-log: log library with loguru - skyline-nginx: generate the nginx.conf file with openstack environment - skyline-policy-manager: generate policy yaml file Generally, we just focus on skyline-apiserver. ## stop using Git's submodule feature Is there any reason why we should stop using git's submodule? ## use Python package configuration and tooling consistent with existing OpenStack projects Now skyline-apiserver use yaml to parse config and poetry to manage dependency. ## call Python entrypoints directly from tox instead of using tox as a wrapper around make Scripts[0] get converted to console_scripts entry points, plugins[1] would allow for arbitrary entry points. https://python-poetry.org/docs/pyproject/#scripts https://python-poetry.org/docs/pyproject/#plugins ## constrain Python dependencies and test dependencies in tox configuration (pip -c) Yet, there is no constraints like pip -c in poetry. Find some discussion about this in poetry. But all are closed, not merged. https://github.com/python-poetry/poetry/pull/4005 https://github.com/python-poetry/poetry-core/pull/172 ## track Python dependencies and test dependencies in requirements files Now we track python depentdencies and test dependencies in poetry's tool.poetry.dependencies and tool.poetry.dev-dependencies ## get Python package versions from Git tags rather than hard-coded in configuration files Sure, but hard-coded package version is not serious problem. ## rely more on Oslo libraries (currently only oslo.policy is used) Reuse wherever possible. oslo.policy is used to generate the policy yaml for skyline to use. ## Horizon has plugin support so that projects like Manila can add dashboard support. Skyline's architecture and technology stacks are different from horizon. So we will continue to explore how to gracefully add non-core module functionality via plug-ins. At the same time, we will also improve core component functions. Best Regards Boxiang Zhu

2 1

[nova][placement] Yoga PTG summary
by Sylvain Bauza 26 Oct '21

26 Oct '21

Well, it's a try given we discussed for 4 days and it could be a large summary ;) You can see all the notes in a read-only etherpad here : https://etherpad.opendev.org/p/r.e70aa851abf8644c29c8abe4bce32b81 ### Cross-project discussions # Cyborg cross-project discussion with Nova We agreed on adding a OWNED_BY_NOVA trait for all Resource Providers creating by Nova so Cyborg would provide their own OWNED_BY_CYBORG trait for knowing which inventories are used by either Nova or Cyborg. Cyborg contributors need to modify https://review.opendev.org/c/openstack/nova-specs/+/780452 We also agreed on the fact that https://blueprints.launchpad.net/nova/+spec/cyborg-suspend-and-resume is a specless blueprint. # Oslo cross-project discussion with Nova gmann agreed on providing a new flag for oslopolicy-sample-generator for adding deprecated rules in the generated policy file. # RBAC popup team discussion with Nova Eventually, we found no consensus for this topic as there were some left open questions about system-scope. FWIW, the popup team then discussed on Friday with the TC about this, so please look at the TC etherpad if you want to know more. A side impact is about https://review.opendev.org/c/openstack/nova-specs/+/793011 which is now punted until we figure out a good path. # Neutron cross-project discussion with Nova Again, about RBAC for external events api interaction, we discussed about the scopes and eventually punted the discussion. About specific events related to Neutron backends, ralonso accepted to provide a documentation explaining what backends sends which events, and we accepted to merge https://review.opendev.org/c/openstack/nova/+/813419 as a short-term solution while we would like to get a long-term solution by having Neutron providing the event information by the port binding information. About testing move operations, we agreed on continuing to have a ML2/OVS multinode job. During another Nova session, we also agreed on changing libvirt to directly unplug (not unbind) ports during VM shutdown. # Interop cross-project disussion with Nova We agreed on reviewing https://review.opendev.org/c/openinfra/interop/+/811049/3/guidelines/2021.1… # Cinder cross-project discussion with Nova We discussed about https://blueprints.launchpad.net/nova/+spec/volume-backed-server-rebuild and we said the workflow should be something like user > nova > cinder > nova. We also discussed about some upgrade questions for this one, but we eventually agreed on it. We also discussed about devstack-plugin-nfs gate issues and how to contact the nova events API for resize. # Manila integration with libvirt driver The first spec looks promising https://review.opendev.org/c/openstack/nova-specs/+/813180 There were discussions about cloud-init and the Ironic case, but we agree on the smaller scope for the Yoga release that's proposed in the spec. ### Nova specific topics ### # Xena retrospective We agreed on stopping to have an Asian-friendly meeting timeslot once per month as unfortunately, no contributors went to meetings when we had them. We also agreed on modifying the review-priority label for abling contributors to use it too, but we first need to provide a documentation explaining it before we change Gerrit. # Continuing to use Storyboard for Placement or not ? We eventually said we would look at how create a script for moving Storyboard (SB) stories to Launchpad (as either features or bugs) but we still want to continue verifying whether contributors use Storyboard for Placement bugs or feature requests and bauzas accepted to provide visibility on Placement SB stories during the weekly meeting. gmann also agreed on asking contributors to move off the #openstack-placement IRC channel to #openstack-nova so we would delete this IRC channel eventually during this cycle. # Yoga release deadlines for Nova No real change in deadlines compared to the Xena timeframe. We agreed on having two spec review days, one around mid-Nov (before yoga-1) and one around mid-Dec (before Christmas period) in order to prioritize implementation reviews after Jan 1st even if we can continue to review specs until yoga-2. # python version pinning with tox We agreed on the fact this is a pain for developers. We then had a consensus on modifying tox to accept multiple python versions automatically with https://review.opendev.org/c/openstack/nova/+/804292 so it would fix the issue. # Nova DB issues with DB coalation We agreed on the fact it's a problem, so we'll document the fact that Nova APIs are case-insensitive at the moment even if Python is case-sensitive, which creates problems. What we propose in order to fix this is to provide a nova-manage db upgrade command that would modify the DB tables to use COLLATE utf8_bin but we also agree we can't ask operators to migrate by one cycle and we accept the fact this command could be there for a while. # SQLAlchemy 2.0 The wave is coming and we need contributors to help us change what we have in our nova DB modules to no longer use the deprecated calls. This looks a low-hanging-fruit and I'll try to find some contributor for this. # Bumping minimum microversion from v2.1 No, we said no because $users use it. # Unified limits We agreed on providing a read-only API for knowing the limits but *not* providing proxy APIs for setting limits *as a first attempt*. We prefer operators to come back with usecases and feedback from their use of Unified Limits 1.0 before we start drafting some proxy API to Keystone. Also, we agreed on *not* having config-driven quotas. # Central vncproxy in a multiple-cells environment We understand the usecase which is very specific and we accept to create a central vncproxy service that would proxy calls to the cell-related vncproxy service but this is not a pattern we want to follow for every cell-specific nova service. # Move instances between projects. Well, we totally get the usecase but we absolutely lack of resources in order to work on this very large effort that would span multiple services. # Nova service healthchecks We agreed on providing a way for monitoring tools to ping Nova services for healthness thru http or unix socket, from every service that would return a status based on cached data. A spec has to be written. # Zombie Resource Providers no longer corresponding to Nova resources Thanks to the OWNED_BY_NOVA trait we agreed when discussing with the Cyborg team, we could find a way to know the ResourceProviders owned by Nova that are no longer in use and we could consequently delete them, or warn the operator if existing allocations are present. # NUMA balancing This is definitely a bug we will fix by providing a workaround config option that will let operators define the packing or spreading strategy they want for NUMA cell stacking (or not). # Deprecation of the novaclient shell command Yes, now that the SDK is able to negociate, we can deprecate the novaclient CLI. More investigation work has to be done in order to know whether we can also deprecate the novaclient library itself. # Integration with Off-Path Network Backends Lots of technicalities with this very large spec https://review.opendev.org/c/openstack/nova-specs/+/787458. Long story short, we asked the proposer to add a few more details in the spec about upgrade scenarios, move operations and testing, but we also told that we can't accept this spec until the Neutron one lands as there are some usage from the extended Neutron APIs that are proposed in the Nova spec. # 'pc' and 'q35' machine types We agreed on changing the default machine type to 'q35' but we also agreed on *not* deprecating the 'pc' type. Some documentation has to be written in order to explain the intent of the migration. # Nova use of privsep sean-k-mooney agreed on working on a patch to remove usage of CAP_DAC_OVERRIDE and on another patch to provide the new capabilities for the monolith privsep context (a few other topics were discussed but I skipped them from my summary as they're not significant for either operators or other contributors but a very few people - don't get me wrong, I like them but I don't wanna add more verbosity to an already large summary) ### Nova painpoints (taken from https://etherpad.opendev.org/p/pain-point-elimination) # Ironic hashring failure recovery That's a latent bug but we don't want the Ironic virt driver to modify the host value of every instance. We'd rather prefer some operator action in order to communicate Nova it has to shuffle a few things. More brainstorm has honestly to be done with this as we haven't clearly drafted a designed solution yet. # Problems with shelve, unshelve and then shelve back Well, this is a nasty bug and we need to fix it, agreed. We also have a testing gap we need to close. # Naming cinder volumes after nova instance name We said yes, why not. We also considered that 'delete on terminate' has to change so it does delete the volume when you delete the instance (for a boot-for-volume case) # Orphaned instances due to underlying network issues We agreed on the fact it would be nice to provide a tool for knowing such orphans and we also think it's import for instance force-delete API call to complete successfully in such case. # reminiscent guests on a recovered compute node while instance records were purged Well, we should avoid to purge instance records from the Nova DB if we are still unable to correctly delete the compute bits unless the operator explicitly wants to (in the case of a non-recoverable compute for example). A potential solution can be to add a config flag to *not* archive deleted rows on instances that are running on down computes. # Nova APIs leaking hypervisor hardware details No, we don't want this but we can try to add more traits for giving more visibility on a case-by-case basis # RabbitMQ replacement Well, we're unfortunately lacking resources on this. We recommend the community to investigate on use of a NATS backend for oslo.messaging # Placement strictness with move operations We agree this is a bug that needs to be fixed. Okay, if you reach this point, you're very brave. Kudos to you. I don't really want to reproduce this exercice often, but I just hope it helps you summarizing a thousand-line large etherpad. -Sylvain

2 2

Octavia loadbalancer status offline
by Md. Hejbul Tawhid MUNNA 26 Oct '21

26 Oct '21

Hi, We have installed openstack ussuri version from ubuntu universe repository. We have installed octavia 6.2.0 version. after creating loadbalancer , listener and pool all are offline. but the LB operation is working as expected. changing the pool member is also working. octavia is installed in compute node. 5555 is listening and allowed in iptables (iptables -I INPUT -i o-hm0 -p udp --dport 5555 -j ACCEPT). amphora to octavia-worker(172.16.0.2) is reachable. Any idea to troubleshoot this issue Please find the log from octavia-worker ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// 2021-10-25 18:15:13.482 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540e192be0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:18.490 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c0ddd00>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:23.507 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c0ddd90>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:28.511 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c0ddbb0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:33.521 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540c064190>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:38.529 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f540d111790>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:43.540 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec7a3670>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:48.549 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec74d0d0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:53.554 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec75a7f0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:15:58.561 1192307 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Could not connect to instance. Retrying.: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='172.16.1.220', port=9443): Max retries exceeded with url: // (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f53ec75a6a0>: Failed to establish a new connection: [Errno 111] Connection refused')) 2021-10-25 18:16:04.707 1192307 INFO octavia.controller.worker.v1.tasks.database_tasks [req-806e5e6b-46b0-4a52-900f-7c2d22d4442d - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Mark ALLOCATED in DB for amphora: c661b828-1690-4866-8152-f745c43e0977 with compute id c9133819-b8e0-42d6-9544-bf83e3ad4b3f for load balancer: c0bd3e21-6983-40c9-8713-859194496b37 2021-10-25 18:16:40.660 1192307 INFO octavia.controller.worker.v1.tasks.database_tasks [req-78e78f29-3bdb-4b12-ae76-b8daa4926c09 - 9a817a70161d45fd9f0b5fe2cad30f5c - - -] Mark ACTIVE in DB for load balancer id: c0bd3e21-6983-40c9-8713-859194496b37 2021-10-25 18:16:44.317 1192307 INFO octavia.controller.queue.v1.endpoints [-] Creating listener 'cc45192d-de70-4d59-857b-ac23c4fc8d07'... 2021-10-25 18:16:44.325 1192307 WARNING octavia.controller.worker.v1.controller_worker [-] Failed to fetch listener cc45192d-de70-4d59-857b-ac23c4fc8d07 from DB. Retrying for up to 60 seconds. 2021-10-25 18:17:35.375 1192307 INFO octavia.controller.queue.v1.endpoints [-] Creating pool '9d23855f-d849-4ad9-9de1-66ab5cd268eb'... 2021-10-25 18:17:35.382 1192307 WARNING octavia.controller.worker.v1.controller_worker [-] Failed to fetch pool 9d23855f-d849-4ad9-9de1-66ab5cd268eb from DB. Retrying for up to 60 seconds. 2021-10-25 18:18:02.814 1192307 INFO octavia.controller.queue.v1.endpoints [-] Creating member '29bb41e5-457c-43ba-9149-5af55e73fe38'... 2021-10-25 18:18:02.825 1192307 WARNING octavia.controller.worker.v1.controller_worker [-] Failed to fetch member 29bb41e5-457c-43ba-9149-5af55e73fe38 from DB. Retrying for up to 60 seconds. Regards, Munna

3 4

Re: Openstack magnum
by Vikarna Tathe 26 Oct '21

26 Oct '21

Hi Yasemin, You can find all the stable releases in the below link https://builds.coreos.fedoraproject.org/browser?stream=stable&arch=x86_64 Vikarna On Sun, 24 Oct 2021 at 17:28, Yasemin DEMİRAL (BILGEM BTE) < yasemin.demiral(a)tubitak.gov.tr> wrote: > Hi, > > How can I dowloand fcos 33? I can't find any link for dowloanding it. > > *Yasemin DEMİRAL* > <http://www.tubitak.gov.tr/tr/icerik-sorumluluk-reddi> > > Senior Researcher at TUBITAK BILGEM B3LAB > > Safir Cloud Scrum Master > > > ------------------------------ > *Kimden: *"Vikarna Tathe" <vikarnatathe(a)gmail.com> > *Kime: *"Ammad Syed" <syedammad83(a)gmail.com> > *Kk: *"openstack-discuss" <openstack-discuss(a)lists.openstack.org> > *Gönderilenler: *19 Ekim Salı 2021 16:23:20 > *Konu: *Re: Openstack magnum > > Hi Ammad, > Thanks!!! It worked. > > On Tue, 19 Oct 2021 at 15:00, Vikarna Tathe <vikarnatathe(a)gmail.com> > wrote: > >> Hi Ammad, >> Yes, fcos34. Let me try with fcos33. Thanks >> >> On Tue, 19 Oct 2021 at 14:52, Ammad Syed <syedammad83(a)gmail.com> wrote: >> >>> Hi, >>> >>> Which fcos image you are using ? It looks like you are using fcos 34. >>> Which is currently not supported. Use fcos 33. >>> >>> On Tue, Oct 19, 2021 at 2:16 PM Vikarna Tathe <vikarnatathe(a)gmail.com> >>> wrote: >>> >>>> Hi All, >>>> I was able to login to the instance. I see that kubelet service is in >>>> activating state. When I checked the journalctl, found the below. >>>> >>>> >>>> >>>> >>>> >>>> >>>> *Oct 19 05:18:34 kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: >>>> Started Kubelet via Hyperkube (System Container).Oct 19 05:18:34 >>>> kubernetes-cluster-6cdrblcpckny-master-0 bash[6521]: Error: statfs >>>> /sys/fs/cgroup/systemd: no such file or directoryOct 19 05:18:34 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: Main >>>> process exited, code=exited, status=125/n/aOct 19 05:18:34 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: >>>> Failed with result 'exit-code'.Oct 19 05:18:44 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: kubelet.service: >>>> Scheduled restart job, restart counter is at 18.Oct 19 05:18:44 >>>> kubernetes-cluster-6cdrblcpckny-master-0 systemd[1]: Stopped Kubelet via >>>> Hyperkube (System Container).* >>>> >>>> Executed the below command to fix this issue. >>>> *mkdir -p /sys/fs/cgroup/systemd* >>>> >>>> >>>> Now I am getiing the below error. Has anybody seen this issue. >>>> >>>> >>>> >>>> *failed to get the kubelet's cgroup: mountpoint for cpu not found. >>>> Kubelet system container metrics may be missing.failed to get the container >>>> runtime's cgroup: failed to get container name for docker process: >>>> mountpoint for cpu not found. failed to run Kubelet: mountpoint for not >>>> found* >>>> >>>> On Mon, 18 Oct 2021 at 14:09, Vikarna Tathe <vikarnatathe(a)gmail.com> >>>> wrote: >>>> >>>>> >>>>>> Hi Ammad, >>>>>> Thanks for responding. >>>>>> >>>>>> Yes the instance is getting created, but i am unable to login >>>>>> though i have generated the keypair. There is no default password for this >>>>>> image to login via console. >>>>>> >>>>>> openstack server list >>>>>> >>>>>> +--------------------------------------+--------------------------------------+---------+------------------------------------+----------------------+----------+ >>>>>> | ID | Name >>>>>> | Status | Networks | Image >>>>>> | Flavor | >>>>>> >>>>>> +--------------------------------------+--------------------------------------+---------+------------------------------------+----------------------+----------+ >>>>>> | cf955a75-8cd2-4f91-a01f-677159b57cb2 | >>>>>> k8s-cluster-01-2nyejxo3hyvb-master-0 | ACTIVE | private1=10.100.0.39, >>>>>> 10.14.20.181 | fedora-coreos-latest | m1.large | >>>>>> >>>>>> >>>>>> ssh -i id_rsa core(a)10.14.20.181 >>>>>> The authenticity of host '10.14.20.181 (10.14.20.181)' can't be >>>>>> established. >>>>>> ECDSA key fingerprint is >>>>>> SHA256:ykEMpwf79/zTMwcELDSI0f66Sxbri56ovGJ+RRwKXDU. >>>>>> Are you sure you want to continue connecting (yes/no/[fingerprint])? >>>>>> yes >>>>>> Warning: Permanently added '10.14.20.181' (ECDSA) to the list of >>>>>> known hosts. >>>>>> core(a)10.14.20.181: Permission denied >>>>>> (publickey,gssapi-keyex,gssapi-with-mic). >>>>>> >>>>>> On Mon, 18 Oct 2021 at 14:02, Ammad Syed <syedammad83(a)gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> Can you check if the master server is deployed as a nova instance ? >>>>>>> if yes, then login to the instance and check cloud-init and heat agent logs >>>>>>> to see the errors. >>>>>>> >>>>>>> Ammad >>>>>>> >>>>>>> On Mon, Oct 18, 2021 at 12:03 PM Vikarna Tathe < >>>>>>> vikarnatathe(a)gmail.com> wrote: >>>>>>> >>>>>>>> Hello All, >>>>>>>> I am trying to create a kubernetes cluster using magnum. Image: >>>>>>>> fedora-coreos. >>>>>>>> >>>>>>>> >>>>>>>> The stack gets stucked in CREATE_IN_PROGRESS. See the output below. >>>>>>>> openstack coe cluster list >>>>>>>> >>>>>>>> +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ >>>>>>>> | uuid | name | keypair | >>>>>>>> node_count | master_count | status | health_status | >>>>>>>> >>>>>>>> +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ >>>>>>>> | 9381cd12-af60-40b1-b1c4-d3dcc93e926e | k8s-cluster-01 | ctrlkey | >>>>>>>> 2 | 1 | CREATE_IN_PROGRESS | None | >>>>>>>> >>>>>>>> +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ >>>>>>>> >>>>>>>> openstack stack resource show k8s-cluster-01-2nyejxo3hyvb >>>>>>>> kube_masters >>>>>>>> >>>>>>>> +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >>>>>>>> | Field | Value >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> >>>>>>>> +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >>>>>>>> | attributes | {'refs_map': None, 'removed_rsrc_list': >>>>>>>> [], 'attributes': None, 'refs': None} >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | creation_time | 2021-10-18T06:44:02Z >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | description | >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | links | [{'href': ' >>>>>>>> http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', >>>>>>>> 'rel': 'self'}, {'href': ' >>>>>>>> http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', >>>>>>>> 'rel': 'stack'}, {'href': ' >>>>>>>> http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', >>>>>>>> 'rel': 'nested'}] | >>>>>>>> | logical_resource_id | kube_masters >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | physical_resource_id | 3da2083f-0b2c-4b9d-8df5-8468e0de3028 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | required_by | ['kube_cluster_deploy', >>>>>>>> 'etcd_address_lb_switch', 'api_address_lb_switch', 'kube_cluster_config'] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_name | kube_masters >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_status | CREATE_IN_PROGRESS >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_status_reason | state changed >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | resource_type | OS::Heat::ResourceGroup >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> | updated_time | 2021-10-18T06:44:02Z >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> | >>>>>>>> >>>>>>>> +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >>>>>>>> >>>>>>>> Vikarna >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Syed Ammad Ali >>>>>>> >>>>>> -- >>> Regards, >>> >>> Syed Ammad Ali >>> >> >

1 0