June 2020 - openstack-discuss - lists.openstack.org

[stein][neutron] gratuitous arp
by Ignazio Cassano 14 May '21

14 May '21

Hello, I have a problem on stein neutron. When a vm migrate from one node to another I cannot ping it for several minutes. If in the vm I put a script that ping the gateway continously, the live migration works fine and I can ping it. Why this happens ? I read something about gratuitous arp. Please, help me ? Any workaround , please ? Best Regards Ignazio

3 33

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

Fwd: [MURANO] DevStack Murano "yaql_function" Error
by İzzettin Erdem 02 Nov '20

02 Nov '20

Thanks Andy. Regards. Forwarded Conversation Subject: [MURANO] DevStack Murano "yaql_function" Error ------------------------ Gönderen: Andy Botting <andy(a)andybotting.com> Date: 30 Nis 2020 Per, 14:13 To: İzzettin Erdem <root.mch(a)gmail.com> Cc: Rong Zhu <aaronzhu1121(a)gmail.com>, < openstack-discuss(a)lists.openstack.org> Hi İzzettin, Thanks for your interest. Complete log link is at below. It is happening > when i try to deploy or update an environment. > > http://paste.openstack.org/show/792444/ > I've encountered the error AttributeError: 'method' object has no attribute '__yaql_function__' in our environment recently. I believe it started after we upgraded to the Stein release. I haven't tracked it down yet, but calls to the YAQL function: std:Project.getEnvironmentOwner() in a Murano app seem to generate it. It's good to know that at least someone else has reproduced this. cheers, Andy ---------- Gönderen: İzzettin Erdem <root.mch(a)gmail.com> Date: 2 May 2020 Cmt, 16:08 To: Andy Botting <andy(a)andybotting.com> Hi Andy, I solved this problem, it is happening when you try to create or update an environment on existing network. If you change the network with "create new" parameter, it stuck on "creating -or updating- environment" state and after that throws connection error. So i searched Murano network errors and i found this: https://docs.oracle.com/cd/E73172_01/E73173/html/issue-21976631.html. I applied these steps and i created successfully an environment just only with "create new" network parameter. After all these, i realized my devstack environment has no internet connection and because of this murano-agent could not install. So i tried to install Murano on my OSA stable/train test environment and i came with success until last step. Murano-agent installed but this time agent could not connect to rabbitmq instance. I discussed this with OSA IRC channel and i think i have to struggle with this for a long time. Apologize for this long mail but i wanted to inform you. Thanks. Regards. ---------- Gönderen: Andy Botting <andy(a)andybotting.com> Date: 3 May 2020 Paz, 14:24 To: İzzettin Erdem <root.mch(a)gmail.com> Hi İzzettin, Glad you got it sorted. I think you've probably just worked around the issue. Using your new method, you're probably just avoiding the std:Project.getEnvironmentOwner YAQL call now. I've also worked around it in our environment, but I'd like to solve it properly. When I get some time to look into it properly, I'll let you know. cheers, Andy

3 3

[Neutron] no L3 meeting this week (2020-June-01)
by LIU Yulong 28 Oct '20

28 Oct '20

Hi there, Due to the PTG meetings this week, we will cancel the L3 meeting this week. See you guys online next week. Regards, LIU Yulong

1 2

[puppet][congress] Retiring puppet-congress
by Takashi Kajinami 10 Aug '20

10 Aug '20

Hello, As you know, Congress project has been retired already[1], so we will retire its puppet module, puppet-congress in openstack puppet project as well. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014292.ht… Because congress was directly retired instead of getting migrated to x namespace, we'll follow the same way about puppet-congress retirement and won't create x/puppet-congress. Thank you for the contribution made for the project ! Please let us know if you have any concerns about this retirement. Thank you, Takashi Kajinami

2 1

[horizon] patternfly?
by Mohammed Naser 06 Aug '20

06 Aug '20

Hi everyone, I was wondering if anyone in the community has explored the idea of implementing PatternFly inside Horizon. It seems like it shares a lot of the similar ideas that we use and we could really benefit from using a common library that already exists with a lot of good UX thought behind it. I know it's based on React which is a bit of a leap from where Horizon is today. However, I'd be curious if the Horizon team is interested in figuring out a plan to make a migration to something like this happen. Personally, I think I would be able to provide resources to have someone do this work. However, if this seems like a huge stretch and architecture change where it actually makes more sense to stand this up from scratch (and implement an architecture where the dashboard talks directly to the APIs?), perhaps we should explore that. I also would like to hear more from our extended community too, as I think we really need to improve our user interface experience. Thanks, Mohammed -- Mohammed Naser VEXXHOST, Inc.

11 31

[TC] [PTG] Victoria vPTG Summary of Conversations and Action Items
by Kendall Nelson 06 Aug '20

06 Aug '20

Hello Everyone! I hope you all had a productive and enjoyable PTG! While it’s still reasonably fresh, I wanted to take a moment to summarize discussions and actions that came out of TC discussions. If there is a particular action item you are interested in taking, please reply on this thread! For the long version, check out the etherpad from the PTG[1]. Tuesday ====== Ussuri Retrospective ---------------------------- As usual we accomplished a lot. Some of the things we accomplished were around enumerating operating systems per release (again), removing python2 support, and adding the ideas repository. Towards the end of the release, we had a lot of discussions around what to do with leaderless projects, the role of PTLs, and what to do with projects that were missing PTL candidates for the next release. We discussed office hours, their history and reason for existence, and clarified how we can strengthen communication amongst ourselves, the projects, and the larger community. TC Onboarding -------------------- It was brought up that those elected most recently (and even new members the election before) felt like there wasn’t enough onboarding into the TC. Through discussion about what we can do to better support returning members is to better document the daily, weekly and monthly tasks TC members are supposed to be doing. Kendall Nelson proposed a patch to start adding more detail to a guide for TC members already[2]. It was also proposed that we have a sort of mentorship or shadow program for people interested in joining the TC or new TC members by more experienced TC members. The discussion about the shadow/mentorship program is to be continued. TC/UC Merge ------------------ Thierry gave an update on the merge of the committees. The simplified version is that the current proposal is that UC members are picked from TC members, the UC operates within the TC, and that we are already setup for this given the number of TC members that have AUC status. None of this requires a by-laws change. One next step that has already begun is the merging of the openstack-users ML into openstack-discuss ML. Other next steps are to decide when to do the actual transition (disbanding the separate UC, probably at the next election?) and when to setup AUC’s to be defined as extra-ATC’s to be included in the electorate for elections. For more detail, check out the openstack-discuss ML thread[3]. Wednesday ========= Help Wanted List ----------------------- We settled on a format for the job postings and have several on the list. We talked about how often we want to look through, update or add to it. The proposal is to do this yearly. We need to continue pushing on the board to dedicate contributors at their companies to work on these items, and get them to understand that it's an investment that will take longer than a year in a lot of cases; interns are great, but not enough. TC Position on Foundation Member Community Contributions ---------------------------------------------------------------------------------- The discussion started with a state of things today - the expectations of platinum members, the benefits the members get being on the board and why they should donate contributor resources for these benefits, etc. A variety of proposals were made: either enforce or remove the minimum contribution level, give gold members the chance to have increased visibility (perhaps giving them some of the platinum member advantages) if they supplement their monetary contributions with contributor contributions, etc. The #ACTION that was decided was for Mohammed to take these ideas to the board and see what they think. OpenStack User-facing APIs -------------------------------------- Users are confused about the state of the user facing API’s; they’ve been told to use the OpenStackClient(OSC) but upon use, they discover that there are features missing that exist in the python-*clients. Partial implementation in the OSC is worse than if the service only used their specific CLI. Members of the OpenStackSDK joined discussions and explained that many of the barriers that projects used to have behind implementing certain commands have been resolved. The proposal is to create a pop up team and that they start with fully migrating Nova, documenting the process and collecting any other unresolved blocking issues with the hope that one day we can set the migration of the remaining projects as a community goal. Supplementally, a new idea was proposed- enforcing new functionality to services is only added to the SDK (and optionally the OSC) and not the project’s specific CLI to stop increasing the disparity between the two. The #ACTION here is to start the pop up team, if you are interested, please reply! Additionally, if you disagree with this kind of enforcement, please contact the TC as soon as possible and explain your concerns. PTL Role in OpenStack today & Leaderless Projects --------------------------------------------------------------------- This was a veeeeeeeerrrry long conversation that went in circles a few times. The very short version is that we, the TC, are willing to let project teams decide for themselves if they want to have a more deconstructed kind of PTL role by breaking it into someone responsible for releases and someone responsible for security issues. This new format also comes with setting the expectation that for things like project updates and signing up for PTG time, if someone on the team doesn’t actively take that on, the default assumption is that the project won’t participate. The #ACTION we need someone to take on is to write a resolution about how this will work and how it can be done. Ideally, this would be done before the next technical election, so that teams can choose it at that point. If you are interested in taking on the writing of this resolution, please speak up! Cross Project Work ------------------------- -Pop Up Teams- The two teams we have right now are Encryption and Secure Consistent Policy Groups. Both are making slow progress and will continue. -Reducing Community Goals Per Cycle- Historically we have had two goals per cycle, but for smaller teams this can be a HUGE lift. The #ACTION is to clearly outline the documentation for the goal proposal and selection process to clarify that selecting only one goal is fine. No one has claimed this action item yet. -Victoria Goal Finalization- Currently, we have three proposals and one accepted goal. If we are going to select a second goal, it needs to be done ASAP as Victoria development has already begun. All TC members should review the last proposal requesting selection[4]. -Wallaby Cycle Goal Discussion Kick Off- Firstly, there is a #ACTION that one or two TC members are needed to guide the W goal selection. If you are interested, please reply to this thread! There were a few proposed goals for VIctoria that didn’t make it that could be the starting point for W discussions, in particular, the rootwrap goal which would be good for operators. The OpenStackCLI might be another goal to propose for Wallaby. Detecting Unmaintained Projects Early --------------------------------------------------- The TC liaisons program had been created a few releases ago, but the initial load on TC members was large. We discussed bringing this program back and making the project health checks happen twice a release, either the start or end of the release and once in the middle. TC liaisons will look at previously proposed releases, release activity of the team, the state of tempest plugins, if regular meetings are happening, if there are patches in progress and how busy the project’s IRC channel is to make a determination. Since more than one liaison will be assigned to each project, those liaisons can divvy up the work how they see fit. The other aspect that still needs to be decided is where the health checks will be recorded- in a wiki? In a meeting and meeting logs? That decision is still to be continued. The current #ACTION currently unassigned is that we need to assign liaisons for the Victoria cycle and decide when to do the first health check. Friday ===== Reducing Systems and Friction to Drive Change ---------------------------------------------------------------- This was another conversation that went in circles a bit before realizing that we should make a list of the more specific problems we want to address and then brainstorm solutions for them. The list we created (including things already being worked) are as follows: - TC separate from UC (solution in progress) - Stable releases being approved by a separate team (solution in progress) - Making repository creation faster (especially for established project teams) - Create a process blueprint for project team mergers - Requirements Team being one person - Stable Team - Consolidate the agent experience - Figure out how to improve project <--> openstack client/sdk interaction. If you feel compelled to pick one of these things up and start proposing solutions or add to the list, please do! Monitoring in OpenStack (Ceilometer + Telemetry + Gnocchi State) ----------------------------------------------------------------------------------------- This conversation is also ongoing, but essentially we talked about the state of things right now- largely they are not well maintained and there is added complexity with Ceilometers being partially dependent on Gnocchi. There are a couple of ideas to look into like using oslo.metrics for the interface between all the tools or using Ceilometer without Gnocchi if we can clean up those dependencies. No specific action items here, just please share your thoughts if you have them. Ideas Repo Next Steps ------------------------------- Out of the Ussuri retrospective, it was brought up that we probably needed to talk a little more about what we wanted for this repo. Essentially we just want it to be a place to collect ideas into without worrying about the how. It should be a place to document ideas we have had (old and new) and keep all the discussion in one place as opposed to historic email threads, meetings logs, other IRC logs, etc. We decided it would be good to periodically go through this repo, likely as a forum session at a summit to see if there is any updating that could happen or promotion of ideas to community goals, etc. ‘tc:approved-release’ Tag --------------------------------- This topic was proposed by the Manila team from a discussion they had earlier in the week. We talked about the history of the tag and how usage of tags has evolved. At this point, the proposal is to remove the tag as anything in the releases repo is essentially tc-approved. Ghanshyam has volunteered to document this and do the removal. The board also needs to be notified of this and to look at projects.yaml in the governance repo as the source of truth for TC approved projects. The unassigned #ACTION item is to review remaining tags and see if there are others that need to be modified/removed/added to drive common behavior across OpenSack components. Board Proposals ---------------------- This was a pretty quick summary of all discussions we had that had any impact on the board and largely decided who would mention them. Session Feedback ------------------------ This was also a pretty quick topic compared to many of the others, we talked about how things went across all our discussions (largely we called the PTG a success) logistically. We tried to make good use of the raising hands feature which mostly worked, but it lacks context and its possible that the conversation has moved on by the time it’s your turn (if you even remember what you want to say). OpenStack 2.0: k8s Native ----------------------------------- This topic was brought up at the end of our time so we didn’t have time to discuss it really. Basically Mohammed wanted to start the conversation about adding k8s as a base service[5] and what we would do if a project proposed required k8s. Adding services that work with k8s could open a door to new innovation in OpenStack. Obviously this topic will need to be discussed further as we barely got started before we had to wrap things up. So. The tldr; Here are the #ACTION items we need owners for: - Start the User Facing API Pop Up Team - Write a resolution about how the deconstructed PTL roles will work - Update Goal Selection docs to explain that one or more goals is fine; it doesn’t have to be more than one - Two volunteers to start the W goal selection process - Assign two TC liaisons per project - Review Tags to make sure they are still good for driving common behavior across all openstack projects Here are the things EVERYONE needs to do: - Review last goal proposal so that we can decide to accept or reject it for the V release[4] - Add systems that are barriers to progress in openstack to the Reducing Systems and Friction list - Continue conversations you find important Thanks everyone for your hard work and great conversations :) Enjoy the attached (photoshopped) team photo :) -Kendall (diablo_rojo) [1] TC PTG Etherpad: https://etherpad.opendev.org/p/tc-victoria-ptg [2] TC Guide Patch: https://review.opendev.org/#/c/732983/ [3] UC TC Merge Thread: http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014736.html [4] Proposed V Goal: https://review.opendev.org/#/c/731213/ [5] Base Service Description: https://governance.openstack.org/tc/reference/base-services.html

3 2

[neutron][networking-midonet] Maintainers needed
by Slawek Kaplonski 28 Jul '20

28 Jul '20

Hi, Over the past couple of cycles we have noticed that new contributions and maintenance efforts for networking-midonet project were lower and lower. This impacts patches for bug fixes, new features and reviews. The Neutron core team is trying to at least keep the CI of this project more or less healthy, but we don't have enough cycles and knowledge about the details of this project code base to review more complex patches. During the PTG in Shanghai we discussed that with operators and TC members during the forum session [1] and later within the Neutron team during the PTG session [2]. During these discussions, with the help of operators and TC members, we reached the conclusion that we need to have someone responsible for maintaining project. This doesn't mean that the maintainer needs to spend full time working on this project. Rather, we need someone to be the contact person for the project, who takes care of the project's CI and review patches. Of course that'ss only a minimal requirement. If the new maintainer works on new features for the project, it's even better :) I recently spoke with members of current networking-midonet core team and they told me that they don't have cycles to focus on this project. So if we don't have any new maintainer(s) before milestone Victoria-2, which is Jul 27 - Jul 31 according to [3], we will need to mark networking-midonet as deprecated for removal. "Removal" means that in W cycle we will remove code of this project from its master branch and there will be no new releases of it. We will still keep code in stable branches which were already released until its EOL. So if You are using this project now, or if You have customers who are using it, please consider the possibility of maintaining it. Otherwise, please be aware that it is highly possible that the project will be deprecated and moved out from the official OpenStack projects. [1] https://etherpad.openstack.org/p/PVG-Neutron-stadium-projects-the-path-forw… [2] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning-restored - Lines 379-421 [3] https://releases.openstack.org/victoria/schedule.html -- Slawek Kaplonski Senior software engineer Red Hat

1 1

[ceilometer][octavia] polling meters
by Ionut Biru 14 Jul '20

14 Jul '20

Hello guys, I was trying to add in polling.yaml and pipeline from ceilometer the following: - network.services.lb.active.connections - network.services.lb.health_monitor - network.services.lb.incoming.bytes - network.services.lb.listener - network.services.lb.loadbalancer - network.services.lb.member - network.services.lb.outgoing.bytes - network.services.lb.pool - network.services.lb.total.connections But it doesn't work, I think they are for the old lbs that were supported in neutron. I found https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollst… but this is not available in stein or train. I was wondering if there is a way to meter loadbalancers from octavia. I mostly want for start to just meter if a loadbalancer was deployed and has status active. -- Ionut Biru - https://fleio.com

3 17

[All][Neutron][Devstack] OVN as the Default Devstack Neutron Backend
by Slawek Kaplonski 09 Jul '20

09 Jul '20

Hi, The Neutron team wants to propose a switch of the default Neutron backend in Devstack from OVS (neutron-ovs-agent, neutron-dhcp-agent, neutron-l3-agent) to OVN with its own ovn-metadata-agent and ovn-controller. We discussed that change during the virtual PTG - see [1]. In this document we want to explain reasons why we want to do that change. OVN in 75 Words --------------- Open Virtual Network is managed under the OVS project, and was created by the original authors of OVS. It is an attempt to re-do the ML2/OVS control plane, using lessons learned throughout the years. It is intended to be used in projects such as OpenStack and Kubernetes. OVN has a different architecture, moving us away from Python agents communicating with the Neutron API service via RabbitMQ to C daemons communicating via OpenFlow and OVSDB. Here’s a heap of information about OpenStack’s integration of OVN: * OpenStack Boston Summit talk on OVN [2] * Upstream OpenStack networking-ovn documentation [3] and [4] * OSP 13 OVN documentation, including how to install it using Director [5] Neutron OVN driver was developed as a Neutron stadium project, "networking-ovn". In the Ussuri cycle, networking-ovn was merged into the main Neutron repository. Why? ---- In the Neutron team we believe that OVN and the Neutron OVN driver are built with a modern architecture that offers better foundations for a simpler and more performant solution. We see increased participation in kubernetes-ovn, resulting in a larger core OVN community, and we would like OpenStack to benefit from this Kubernetes driven OVN investment. Neutron OVN driver currently has got some feature parity gaps comparing to ML2/OVS (see [6] for details) but our team is working hard to close those gaps and we believe that this driver is the future for Neutron and that’s why we want to make it the default Neutron ML2 backend in the Devstack configuration. What Does it Mean? ------------------ Since most Openstack projects use Neutron in their CI and gate jobs, this change has the potential for a large impact. But this backend is already tested with various jobs in the Neutron CI and it works fine. Recently (See [7]) we also proposed to add an OVN based job to the Devstack’s check queue. Similarly the default Neutron backend in TripleO was changed in the Stein cycle and there were no any significant issues related strictly to this change. It worked well for other projects. Of course in the Neutron project we will be still gating other drivers, like ML2/Linuxbridge and ML2/OVS - nothing will change here, except for the names of some of the jobs. The Neutron team is *NOT* going to deprecate any of the other existing ML2 drivers. We will be still maintaining Linuxbridge, OVS and other in-tree drivers in the same way as it is now. Action Plan ----------- We want to make this change before the Victoria-2 milestone to not make such changes too late in the release cycle. Our action plan is as below: 1. Share the plan and get feedback from the upstream community (this thread) 2. Move OVN related Devstack code from a plugin defined in the Neutron repo to Devstack repo - we don’t want to force everyone else to add “enable_plugin neutron” in their local.conf file to use default Neutron backend, 3. Switch default Neutron backend in Devstack to be OVN, a. Switch definition of base devstack CI jobs that it will run Neutron with OVN backend, 4. Propose DNM patches depend on patch from point 3 and 3a to main OpenStack projects to check if it will not break anything in the gate of those projects. 5. If all will be running fine, merge patches proposed in points 3 and 3a. [1] https://etherpad.opendev.org/p/neutron-victoria-ptg - Lines 185 - 193 [2] https://www.youtube.com/watch?v=sgc7myiX6ts [3] https://docs.openstack.org/neutron/latest/admin/ovn/index.html [4] https://docs.openstack.org/neutron/latest/ovn/index.html [5] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13… [6] https://docs.openstack.org/neutron/latest/ovn/gaps.html [7] https://review.opendev.org/#/c/736021/ -- Slawek Kaplonski Senior software engineer Red Hat

6 8

[ironic] 2nd Victoria meetup
by Dmitry Tantsur 08 Jul '20

08 Jul '20

Hi all, Since we're switching to 6 releases per year cadence, I think it makes sense to have short virtual meetups after every release. The goal will be to sync on priorities, exchange ideas and define plans for the upcoming 2 months of development. Fooling around is also welcome! Please vote for the best 2 hours slot next week: https://doodle.com/poll/3r9tbhmniattkty8. I tried to include more potential time zones, so apologies for so many options. Please cast your vote until Friday, 12pm UTC, so that I can announce the final time slot this week. Dmitry

2 4

[glance] Global Request ID issues in Glance
by Markus Hentsch 07 Jul '20

07 Jul '20

Hello everyone, while I was experimenting with the Global Request ID functionality of OpenStack [1], I identified two issues in Glance related to this topic. I have written my findings below and would appreciate it if you could take a look and confirm whether those are intended behaviors or indeed issues with the implementation. In case of the latter please advice me which bug tracker to report them to. 1. The Glance client does not correctly forward the global ID When the SessionClient class is used, the global_request_id is removed from kwargs in the constructor using pop() [2]. Directly after this, the parent constructor is called using super(), which in this case is Adapter from the keystoneauth1 library. Therein the global_request_id is set again [3] but since it has been removed from the kwargs, it defaults to None as specified in the Adapter's __init__() header. Thus, the global_request_id passed to the SessionClient constructor never actually makes it to the Glance API. This is in contrast to the HTTPClient class, where get() is used instead of pop() [4]. This can be reproduced simply by creating a server in Nova from an image in Glance, which will attempt to create the Glance client instance using the global_request_id [5]. Passing the "X-Openstack-Request-Id" header during the initial API call for the server creation, makes it visible in Nova (using a suitable "logging_context_format_string" setting) but it's not visible in Glance. Using a Python debugger shows Glance generating a new local ID instead. 2. Glance interprets global ID as local one for Oslo Context objects While observing the Glance log file, I observed Glance always logging the global_request_id instead of a local one if it is available. Using "%(global_request_id)s" within "logging_context_format_string"[6] in the glance-api.conf will always print "None" in the logs whereas "%(request_id)s" will either be an ID generated by Glance if no global ID is available or the received global ID. Culprit seems to be the context middleware of Glance where the global ID in form of the "X-Openstack-Request-Id" header is parsed from the request and passed as "request_id" instead of "global_request_id" to the "glance.context.RequestContext.from_environ()" call [7]. This is in contrast to other services such as Nova or Neutron where the two variables actually print the values according to their name (request_id always being the local one, whereas global_request_id is the global one or None). [1] https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.h… [2] https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022… [3] https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d314… [4] https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022… [5] https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca6… [6] https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-vari… [7] https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f5645108… Thanks! Markus -- Markus Hentsch Team Leader secustack GmbH - Digital Sovereignty in the Cloud https://www.secustack.com Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden District Court Dresden, Register Number: HRB 38890

2 2

Re: [rdo-users] [rdo][ussuri][TripleO][nova][kvm] libvirt.libvirtError: internal error: process exited while connecting to monitor
by Arkady Shtempler 06 Jul '20

06 Jul '20

Hi all! I was able to analyze the attached log files and I hope that the results may help you understand what's going wrong with instance creation. You can find *Log_Tool's unique exported Error blocks* here: http://paste.openstack.org/show/795356/ *Some statistics and problematical messages:* ##### Statistics - Number of Errors/Warnings per Standard OSP log since: 2020-06-30 12:30:00 ##### Total_Number_Of_Errors --> 9 /home/ashtempl/Ruslanas/controller/neutron/server.log --> 1 /home/ashtempl/Ruslanas/compute/stdouts/ovn_controller.log --> 1 /home/ashtempl/Ruslanas/compute/nova/nova-compute.log --> 7 *nova-compute.log* *default default] Error launching a defined domain with XML: <domain type='kvm'>* 368-2020-06-30 12:30:10.815 7 *ERROR* nova.compute.manager [req-87bef18f-ad3d-4147-a1b3-196b5b64b688 7bdb8c3bf8004f98aae1b16d938ac09b 69134106b56941698e58c61... 70dc50f] Instance *failed* to spawn: *libvirt.libvirtError*: internal *error*: qemu unexpectedly closed the monitor: 2020-06-30T10:30:10.182675Z qemu-kvm: *error*: failed to set MSR 0... he monitor: 2020-06-30T10:30:10.182675Z *qemu-kvm: error: failed to set MSR 0x48e to 0xfff9fffe04006172* _msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' *failed*. [instance: 128f372c-cb2e-47d9-b1bf-ce17270dc50f] *Traceback* (most recent call last): 375-2020-06-30 12:30:10.815 7* ERROR* nova.compute.manager [instance: 128f372c-cb2e-47d9-b1bf-ce17270dc50f] File "/usr/lib/python3.6/site-packages/nova/vir... *server.log * 5821c815-d213-498d-9394-fe25c6849918', 'status': 'failed', *'code': 422} returned with failed status* *ovn_controller.log* 272-2020-06-30T12:30:10.126079625+02:00 stderr F 2020-06-30T10:30:10Z|00247|patch|WARN|*Bridge 'br-ex' not found for network 'datacentre'* Thanks! On Tue, Jun 30, 2020 at 2:13 PM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> wrote: > attaching logs here. let's see if it will work. > > On Tue, 30 Jun 2020 at 12:55, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > >> hi all, >> >> I am back, had some issues with MTU. >> Now looks good, at least deployment part. >> >> So I have installed back what I had, and still failing at same point as >> in first message. >> >> I have tried to use: LogTool, how to use it? well, I launched it, but it >> always say [0] detailed output: >> File "./PyTool.py", line 596, in <module> >> random_node=random.choice(overcloud_nodes) >> >> I do not get, how to make it work, should it get from stackrc ? as I see >> in >> overcloud_nodes = [] >> all_nodes = exec_command_line_command('source ' + source_rc_file_path >> + 'stackrc;openstack server list -f json')[ >> >> [0] http://paste.openstack.org/show/795345/ >> >> On Wed, 24 Jun 2020 at 20:02, Arkady Shtempler <ashtempl(a)redhat.com> >> wrote: >> >>> Hi Ruslanas! >>> >>> Is it possible to get all logs under /var/log/containers somehow? >>> >>> Thanks! >>> >>> On Wed, Jun 24, 2020 at 2:18 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>> wrote: >>> >>>> Hi Alfredo, >>>> >>>> Compute nodes are baremetal or virtualized?, I've seen similar bug >>>>>> reports when using nested virtualization in other OSes. >>>>>> >>>>> baremetal. Dell R630 if to be VERY precise. >>>> >>>> >>>> >>>>>> When using podman, the recommended way to restart containers is using >>>>> systemd: >>>>> >>>>> >>>>> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… >>>>> >>>> >>>> Thank you, I will try. I also modified a file, and it looked like it >>>> relaunched podman container once config was changed. Either way, if I >>>> understand Linux config correctly, the default value for user and group is >>>> root, if commented out: >>>> #user = "root" >>>> #group = "root" >>>> >>>> also in some logs, I saw, that it detected, that it is not AMD CPU :) >>>> and it is really not AMD CPU. >>>> >>>> >>>> Just for fun, it might be important, here is how my node info looks. >>>> ComputeS01Parameters: >>>> NovaReservedHostMemory: 16384 >>>> KernelArgs: "crashkernel=no rhgb" >>>> ComputeS01ExtraConfig: >>>> nova::cpu_allocation_ratio: 4.0 >>>> nova::compute::libvirt::rx_queue_size: 1024 >>>> nova::compute::libvirt::tx_queue_size: 1024 >>>> nova::compute::resume_guests_state_on_host_boot: true >>>> _______________________________________________ >>>> users mailing list >>>> users(a)lists.rdoproject.org >>>> http://lists.rdoproject.org/mailman/listinfo/users >>>> >>>> To unsubscribe: users-unsubscribe(a)lists.rdoproject.org >>>> >>> >> >> -- >> Ruslanas Gžibovskis >> +370 6030 7030 >> > > > -- > Ruslanas Gžibovskis > +370 6030 7030 >

4 10

[All][Neutron] Migrate old DB migration versions to init ops
by Rodolfo Alonso Hernandez 06 Jul '20

06 Jul '20

Hello all: Along this years we have increased the number of DB migrations each time we needed a new DB schema. This is good because that means the project is evolving and adding new features. Although this is not a problem per se, there are some inconvenients: - Every time a system is deployed (for example in the CI using devstack), the initial DB schema is created. Then, each migration is applied sequentially. - Some FT tests are still checking the sanity of some migrations [1] implemented a few releases ago. - We are still testing the contract DB migrations. Of course, this is something supported before and we still need to apply those revisions. - "TestWalkMigrationsMysql" and "TestModelsMigrationsMysql", both using MySQL backend, are still affected by LP#1687027. The proposal is to remove some DB migrations, starting from Liberty; of course, because all migrations must be applied in a specific order, we should begin from the initial revision, "kilo". The latest migration to be removed should be decided depending on the stable releases support. Apart from mitigating or solving some of the commented problems, we can "group" the DB model definition in one place. E.g.: "subnetpools" table is created in "other_extensions_init_ops". This file contains the first table. However is modified in at least two migrations: - 1b4c6e320f79_address_scope_support_in_subnetpool: added "address_scope_id" field - 13cfb89f881a_add_is_default_to_subnetpool: added "is_default" field Instead of having (at least) three places where the "subnetpools" DB schema is defined, we can remove the Mitaka migration and group this definition in just one place. One possible issue: some migrations add dependencies on other tables. That means the table the dependency is referring should be created in advance. That implies that, in some cases, the table creation order should be modified. That should never affect subsequent created tables or migrations. Do you see any inconvenience on this proposal? Am I missing something that I didn't consider? Thank you and regards. [1] https://github.com/openstack/neutron/blob/9fd60ffaac6b178de62dab169c826d52f…

4 6

[Telemetry] Propose Matthias Runge for Telemetry core
by Rong Zhu 03 Jul '20

03 Jul '20

Hello all, Matthias Runge have been very active in the repository with patches and reviews. So I would like to propose adding Matthias as core developer for the telemetry project. Please, feel free to add your votes into the thread. -- Thanks, Rong Zhu

3 3

[2020 Summit] Programming Committee Nominations Open
by Ashlee Ferguson 02 Jul '20

02 Jul '20

Programming Committee nominations for the 2020 Open Infrastructure Summit are open! Programming Committees for each Track will help build the Summit schedule, and are made up of individuals working in open infrastructure. Responsibilities include: • Help the Summit team put together the best possible content based on your subject matter expertise • Promote the individual Tracks within your networks • Review the submissions and Community voting results in your particular Track • Determine if there are any major content gaps in your Track, and if so, potentially solicit additional speakers directly to submit • Ensure diversity of speakers and companies represented in your Track • Avoid vendor sales pitches, focusing more on real-world user stories and technical, in-the-trenches experiences 2020 Summit Tracks: • 5G, NFV & Edge • AI, Machine Learning & HPC • CI/CD • Container Infrastructure • Getting Started • Hands-on Workshops • Open Development • Private & Hybrid Cloud • Public Cloud • Security If you’re interested in nominating yourself or someone else to be a member of the Summit Programming Committee for a specific Track, please fill out the nomination form[1]. Nominations will close on July 10, 2020. NOMINATION FORM[1] Programming Committee selections will occur before we open the Call for Presentations (CFP) to receive presentations so that the Committees can host office hours to consult on submissions, and help promote the event. The CFP will be open July 1 - August 4, 2020. Please email speakersupport(a)openstack.org with any questions or feedback. Cheers, Ashlee [1] https://openstackfoundation.formstack.com/forms/programmingcommitteenom_sum… Ashlee Ferguson Community & Events Coordinator OpenStack Foundation

1 1

[all][stable] Moving the stable/ocata to 'Unmaintained' phase and then EOL
by Ghanshyam Mann 02 Jul '20

02 Jul '20

Hello Everyone, Ocata is in the 'Extended Maintenance' phase[1] but its gate is broken and difficult to fix now. The main issue started for gate is about Tempest and its deps constraint is correctly used on Ocata or not. We fixed most of those issues when things started failing during py2 drop[2]. But now things are broken due to newer stestr is installed in Ocata jobs[2]. As Sean also mentioned in this backport[4] that stestr is not constraint in Ocata[5] as it was not in g-r that time so cannot be added in u-c. Tempest old tag cannot be fixed to have that with cap on the tempest side. The only option left to unblock the gate is to remove the integration testing from Ocata gate. But it will untested after that as we cannot say unit testing is enough to mark it tested. At least from QA perspective, I would not say it is tested if there is no integration job running there. Keeping it untested (no integration testing) and saying it is Maintained in the community can be very confusing things. I think its time to move it to the 'Unmaintained' phase and then EOL. Thought? [1] https://releases.openstack.org/ [2] https://review.opendev.org/#/q/topic:fix-stable-gate+(status:open+OR+status… [3] https://zuul.opendev.org/t/openstack/build/e477d0919ff14eea9cd8a6235ea00c3d… [4] https://review.opendev.org/#/c/726931/ [5] https://opendev.org/openstack/requirements/src/branch/stable/ocata/upper-co… -gmann

8 12

[tripleo][ci] status RED
by Wesley Hayutin 02 Jul '20

02 Jul '20

Greetings, Unfortunately both check and gate are RED atm due to [1]. The issue w/ CirrOS-5.1 was fixed / reverted over the weekend [2]. I expect the check and gate jobs to continue to be RED for the next few days as the investigation proceeds. I would encourage folks to only workflow patches that are critical as the chances you will actually merge anything is not great. [1] https://bugs.launchpad.net/tripleo/+bug/1885286 [2] https://review.opendev.org/#/c/738025/

1 2

[neutron][drivers] Propose Rodolfo Alonso Hernandez for Neutron drivers team
by Slawek Kaplonski 01 Jul '20

01 Jul '20

Hi, Rodolfo is very active Neutron contributor since long time. He has wide knowledge about all or almost all areas of the Neutron and Neutron stadium projects. He is an expert e.g. in ovs agent, pyroute and privsep module, openvswitch firewall, db layer, OVO and probably many others. He also has very good understanding about Neutron project in general, about it's design and direction of development. Rodolfo is also active on our drivers meetings already and I think that his feedback about many things there is very good and valuable for the team. For all these reasons I think that he will be great addition to our drivers team. I will keep this nomination open for a week waiting for Your feedback and votes. -- Slawek Kaplonski Senior software engineer Red Hat

6 6

[cinder] monthly video meeting poll
by Brian Rosmaita 30 Jun '20

30 Jun '20

https://rosmaita.wufoo.com/forms/monthly-video-meeting-proposal/ Please respond before 1200 UTC on Thursday 9 July. cheers, brian

1 0

[manila] IRC meeting this week has been canceled
by Goutham Pacha Ravi 30 Jun '20

30 Jun '20

Hello Zorillas, Per the conversation on freenode #openstack-manila, the next occurrence of the Manila community IRC meeting on Thursday, 2nd July 2020 has been cancelled as a number of the regular folks will be on vacation. Our next meeting will be on 9th July 2020 at 1500 UTC: https://wiki.openstack.org/wiki/Manila/Meetings#Next_meeting Please note that the specifications targeting the Victoria cycle must be submitted by 9th July 2020. Please pop into #openstack-manila if you have any concerns in the meantime. Thanks! Goutham

1 0

[all]OpenStack Saves the Day
by Apsey, Christopher 30 Jun '20

30 Jun '20

All, Ref: https://www.fifthdomain.com/dod/army/2020/06/25/pandemic-doesnt-slow-cyber-… The system referenced in the article was the one that I managed in my previous job and is an OpenStack/Ceph platform. It started as three servers that we took out of the trash and slapped on a desk in a classroom four years ago using MOS 8.0 and grew to hit ~20,000 cores and ~12PB of raw ceph storage a few months ago right before I left. We used a mix of UCA and RDO packages depending on the project as well as source compilation if needed. The system referenced in the article is a close fork of https://github.com/GeorgiaCyber/kinetic. Special thanks to the folks at Red Hat and Canonical as well as all the other companies and individual contributors that made that success possible. Everyone's contribution matters. Chris Apsey Director | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 https://www.gacybercenter.org

2 1

Re: [rdo-users] [rdo][ussuri][TripleO][nova][kvm] libvirt.libvirtError: internal error: process exited while connecting to monitor
by Ruslanas Gžibovskis 30 Jun '20

30 Jun '20

here are container logs, from compute and controller. https://drive.google.com/drive/folders/158HHigQmHAyg9NvXmiC45ztenFOCosUR?us…

2 2

[rdo][ussuri][TripleO][nova][kvm] libvirt.libvirtError: internal error: process exited while connecting to monitor
by Ruslanas Gžibovskis 30 Jun '20

30 Jun '20

Hi all. I have finally deployed Ussuri, it went quite good, except some my site specific issues and newton config files remake, but now I face issues when launching an instance. [4] Here are logs from compute: /var/log/containers/nova/nova-compute.log [5] from the controller - nova-conductor.log According to errors from compute, something is wrong with kvm? some older bugs refer to kvm exec permissions [2] a bit more recent [1] which refers to selinux issues, but I have setenforce 0 and tried to create instance, not much changed. Also, [3] very interesting kvm.conf file updates suggested, but I am not sure, how to restart service since config changed, as I am BRAND new to container concept with docker/podman, bit more used to uml :) so, how to reload service in container (when container is running as non-root user)? [1] https://bugzilla.redhat.com/show_bug.cgi?id=1551651 # related to RHOSP7 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1091710 # 6 year old. [3] https://ask.openstack.org/en/question/95198/could-not-access-kvm-kernel-mod… [4] http://paste.openstack.org/show/pBVlVbaQGNuoRLHozHGO/ [5] http://paste.openstack.org/show/OqvozreN5aW4HRXM4g4p/ -- Ruslanas Gžibovskis +370 6030 7030

3 6

[neutron] QoS meeting - Tuesday 30.06.2020
by Rodolfo Alonso Hernandez 30 Jun '20

30 Jun '20

Hello: Because the meeting agenda is very short (and nothing urgent) and overlaps the OpenDev sessions, let's cancel today's meeting. Regards.

1 0