June 2020 - openstack-discuss - lists.openstack.org

[ironic] 2nd Victoria meetup
by Dmitry Tantsur 08 Jul '20

08 Jul '20

Hi all, Since we're switching to 6 releases per year cadence, I think it makes sense to have short virtual meetups after every release. The goal will be to sync on priorities, exchange ideas and define plans for the upcoming 2 months of development. Fooling around is also welcome! Please vote for the best 2 hours slot next week: https://doodle.com/poll/3r9tbhmniattkty8. I tried to include more potential time zones, so apologies for so many options. Please cast your vote until Friday, 12pm UTC, so that I can announce the final time slot this week. Dmitry

2 4

[glance] Global Request ID issues in Glance
by Markus Hentsch 07 Jul '20

07 Jul '20

Hello everyone, while I was experimenting with the Global Request ID functionality of OpenStack [1], I identified two issues in Glance related to this topic. I have written my findings below and would appreciate it if you could take a look and confirm whether those are intended behaviors or indeed issues with the implementation. In case of the latter please advice me which bug tracker to report them to. 1. The Glance client does not correctly forward the global ID When the SessionClient class is used, the global_request_id is removed from kwargs in the constructor using pop() [2]. Directly after this, the parent constructor is called using super(), which in this case is Adapter from the keystoneauth1 library. Therein the global_request_id is set again [3] but since it has been removed from the kwargs, it defaults to None as specified in the Adapter's __init__() header. Thus, the global_request_id passed to the SessionClient constructor never actually makes it to the Glance API. This is in contrast to the HTTPClient class, where get() is used instead of pop() [4]. This can be reproduced simply by creating a server in Nova from an image in Glance, which will attempt to create the Glance client instance using the global_request_id [5]. Passing the "X-Openstack-Request-Id" header during the initial API call for the server creation, makes it visible in Nova (using a suitable "logging_context_format_string" setting) but it's not visible in Glance. Using a Python debugger shows Glance generating a new local ID instead. 2. Glance interprets global ID as local one for Oslo Context objects While observing the Glance log file, I observed Glance always logging the global_request_id instead of a local one if it is available. Using "%(global_request_id)s" within "logging_context_format_string"[6] in the glance-api.conf will always print "None" in the logs whereas "%(request_id)s" will either be an ID generated by Glance if no global ID is available or the received global ID. Culprit seems to be the context middleware of Glance where the global ID in form of the "X-Openstack-Request-Id" header is parsed from the request and passed as "request_id" instead of "global_request_id" to the "glance.context.RequestContext.from_environ()" call [7]. This is in contrast to other services such as Nova or Neutron where the two variables actually print the values according to their name (request_id always being the local one, whereas global_request_id is the global one or None). [1] https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.h… [2] https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022… [3] https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d314… [4] https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022… [5] https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca6… [6] https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-vari… [7] https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f5645108… Thanks! Markus -- Markus Hentsch Team Leader secustack GmbH - Digital Sovereignty in the Cloud https://www.secustack.com Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden District Court Dresden, Register Number: HRB 38890

2 2

Re: [rdo-users] [rdo][ussuri][TripleO][nova][kvm] libvirt.libvirtError: internal error: process exited while connecting to monitor
by Arkady Shtempler 06 Jul '20

06 Jul '20

Hi all! I was able to analyze the attached log files and I hope that the results may help you understand what's going wrong with instance creation. You can find *Log_Tool's unique exported Error blocks* here: http://paste.openstack.org/show/795356/ *Some statistics and problematical messages:* ##### Statistics - Number of Errors/Warnings per Standard OSP log since: 2020-06-30 12:30:00 ##### Total_Number_Of_Errors --> 9 /home/ashtempl/Ruslanas/controller/neutron/server.log --> 1 /home/ashtempl/Ruslanas/compute/stdouts/ovn_controller.log --> 1 /home/ashtempl/Ruslanas/compute/nova/nova-compute.log --> 7 *nova-compute.log* *default default] Error launching a defined domain with XML: <domain type='kvm'>* 368-2020-06-30 12:30:10.815 7 *ERROR* nova.compute.manager [req-87bef18f-ad3d-4147-a1b3-196b5b64b688 7bdb8c3bf8004f98aae1b16d938ac09b 69134106b56941698e58c61... 70dc50f] Instance *failed* to spawn: *libvirt.libvirtError*: internal *error*: qemu unexpectedly closed the monitor: 2020-06-30T10:30:10.182675Z qemu-kvm: *error*: failed to set MSR 0... he monitor: 2020-06-30T10:30:10.182675Z *qemu-kvm: error: failed to set MSR 0x48e to 0xfff9fffe04006172* _msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' *failed*. [instance: 128f372c-cb2e-47d9-b1bf-ce17270dc50f] *Traceback* (most recent call last): 375-2020-06-30 12:30:10.815 7* ERROR* nova.compute.manager [instance: 128f372c-cb2e-47d9-b1bf-ce17270dc50f] File "/usr/lib/python3.6/site-packages/nova/vir... *server.log * 5821c815-d213-498d-9394-fe25c6849918', 'status': 'failed', *'code': 422} returned with failed status* *ovn_controller.log* 272-2020-06-30T12:30:10.126079625+02:00 stderr F 2020-06-30T10:30:10Z|00247|patch|WARN|*Bridge 'br-ex' not found for network 'datacentre'* Thanks! On Tue, Jun 30, 2020 at 2:13 PM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> wrote: > attaching logs here. let's see if it will work. > > On Tue, 30 Jun 2020 at 12:55, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > >> hi all, >> >> I am back, had some issues with MTU. >> Now looks good, at least deployment part. >> >> So I have installed back what I had, and still failing at same point as >> in first message. >> >> I have tried to use: LogTool, how to use it? well, I launched it, but it >> always say [0] detailed output: >> File "./PyTool.py", line 596, in <module> >> random_node=random.choice(overcloud_nodes) >> >> I do not get, how to make it work, should it get from stackrc ? as I see >> in >> overcloud_nodes = [] >> all_nodes = exec_command_line_command('source ' + source_rc_file_path >> + 'stackrc;openstack server list -f json')[ >> >> [0] http://paste.openstack.org/show/795345/ >> >> On Wed, 24 Jun 2020 at 20:02, Arkady Shtempler <ashtempl(a)redhat.com> >> wrote: >> >>> Hi Ruslanas! >>> >>> Is it possible to get all logs under /var/log/containers somehow? >>> >>> Thanks! >>> >>> On Wed, Jun 24, 2020 at 2:18 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>> wrote: >>> >>>> Hi Alfredo, >>>> >>>> Compute nodes are baremetal or virtualized?, I've seen similar bug >>>>>> reports when using nested virtualization in other OSes. >>>>>> >>>>> baremetal. Dell R630 if to be VERY precise. >>>> >>>> >>>> >>>>>> When using podman, the recommended way to restart containers is using >>>>> systemd: >>>>> >>>>> >>>>> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… >>>>> >>>> >>>> Thank you, I will try. I also modified a file, and it looked like it >>>> relaunched podman container once config was changed. Either way, if I >>>> understand Linux config correctly, the default value for user and group is >>>> root, if commented out: >>>> #user = "root" >>>> #group = "root" >>>> >>>> also in some logs, I saw, that it detected, that it is not AMD CPU :) >>>> and it is really not AMD CPU. >>>> >>>> >>>> Just for fun, it might be important, here is how my node info looks. >>>> ComputeS01Parameters: >>>> NovaReservedHostMemory: 16384 >>>> KernelArgs: "crashkernel=no rhgb" >>>> ComputeS01ExtraConfig: >>>> nova::cpu_allocation_ratio: 4.0 >>>> nova::compute::libvirt::rx_queue_size: 1024 >>>> nova::compute::libvirt::tx_queue_size: 1024 >>>> nova::compute::resume_guests_state_on_host_boot: true >>>> _______________________________________________ >>>> users mailing list >>>> users(a)lists.rdoproject.org >>>> http://lists.rdoproject.org/mailman/listinfo/users >>>> >>>> To unsubscribe: users-unsubscribe(a)lists.rdoproject.org >>>> >>> >> >> -- >> Ruslanas Gžibovskis >> +370 6030 7030 >> > > > -- > Ruslanas Gžibovskis > +370 6030 7030 >

4 10

[All][Neutron] Migrate old DB migration versions to init ops
by Rodolfo Alonso Hernandez 06 Jul '20

06 Jul '20

Hello all: Along this years we have increased the number of DB migrations each time we needed a new DB schema. This is good because that means the project is evolving and adding new features. Although this is not a problem per se, there are some inconvenients: - Every time a system is deployed (for example in the CI using devstack), the initial DB schema is created. Then, each migration is applied sequentially. - Some FT tests are still checking the sanity of some migrations [1] implemented a few releases ago. - We are still testing the contract DB migrations. Of course, this is something supported before and we still need to apply those revisions. - "TestWalkMigrationsMysql" and "TestModelsMigrationsMysql", both using MySQL backend, are still affected by LP#1687027. The proposal is to remove some DB migrations, starting from Liberty; of course, because all migrations must be applied in a specific order, we should begin from the initial revision, "kilo". The latest migration to be removed should be decided depending on the stable releases support. Apart from mitigating or solving some of the commented problems, we can "group" the DB model definition in one place. E.g.: "subnetpools" table is created in "other_extensions_init_ops". This file contains the first table. However is modified in at least two migrations: - 1b4c6e320f79_address_scope_support_in_subnetpool: added "address_scope_id" field - 13cfb89f881a_add_is_default_to_subnetpool: added "is_default" field Instead of having (at least) three places where the "subnetpools" DB schema is defined, we can remove the Mitaka migration and group this definition in just one place. One possible issue: some migrations add dependencies on other tables. That means the table the dependency is referring should be created in advance. That implies that, in some cases, the table creation order should be modified. That should never affect subsequent created tables or migrations. Do you see any inconvenience on this proposal? Am I missing something that I didn't consider? Thank you and regards. [1] https://github.com/openstack/neutron/blob/9fd60ffaac6b178de62dab169c826d52f…

4 6

[Telemetry] Propose Matthias Runge for Telemetry core
by Rong Zhu 03 Jul '20

03 Jul '20

Hello all, Matthias Runge have been very active in the repository with patches and reviews. So I would like to propose adding Matthias as core developer for the telemetry project. Please, feel free to add your votes into the thread. -- Thanks, Rong Zhu

3 3

[2020 Summit] Programming Committee Nominations Open
by Ashlee Ferguson 02 Jul '20

02 Jul '20

Programming Committee nominations for the 2020 Open Infrastructure Summit are open! Programming Committees for each Track will help build the Summit schedule, and are made up of individuals working in open infrastructure. Responsibilities include: • Help the Summit team put together the best possible content based on your subject matter expertise • Promote the individual Tracks within your networks • Review the submissions and Community voting results in your particular Track • Determine if there are any major content gaps in your Track, and if so, potentially solicit additional speakers directly to submit • Ensure diversity of speakers and companies represented in your Track • Avoid vendor sales pitches, focusing more on real-world user stories and technical, in-the-trenches experiences 2020 Summit Tracks: • 5G, NFV & Edge • AI, Machine Learning & HPC • CI/CD • Container Infrastructure • Getting Started • Hands-on Workshops • Open Development • Private & Hybrid Cloud • Public Cloud • Security If you’re interested in nominating yourself or someone else to be a member of the Summit Programming Committee for a specific Track, please fill out the nomination form[1]. Nominations will close on July 10, 2020. NOMINATION FORM[1] Programming Committee selections will occur before we open the Call for Presentations (CFP) to receive presentations so that the Committees can host office hours to consult on submissions, and help promote the event. The CFP will be open July 1 - August 4, 2020. Please email speakersupport(a)openstack.org with any questions or feedback. Cheers, Ashlee [1] https://openstackfoundation.formstack.com/forms/programmingcommitteenom_sum… Ashlee Ferguson Community & Events Coordinator OpenStack Foundation

1 1

[all][stable] Moving the stable/ocata to 'Unmaintained' phase and then EOL
by Ghanshyam Mann 02 Jul '20

02 Jul '20

Hello Everyone, Ocata is in the 'Extended Maintenance' phase[1] but its gate is broken and difficult to fix now. The main issue started for gate is about Tempest and its deps constraint is correctly used on Ocata or not. We fixed most of those issues when things started failing during py2 drop[2]. But now things are broken due to newer stestr is installed in Ocata jobs[2]. As Sean also mentioned in this backport[4] that stestr is not constraint in Ocata[5] as it was not in g-r that time so cannot be added in u-c. Tempest old tag cannot be fixed to have that with cap on the tempest side. The only option left to unblock the gate is to remove the integration testing from Ocata gate. But it will untested after that as we cannot say unit testing is enough to mark it tested. At least from QA perspective, I would not say it is tested if there is no integration job running there. Keeping it untested (no integration testing) and saying it is Maintained in the community can be very confusing things. I think its time to move it to the 'Unmaintained' phase and then EOL. Thought? [1] https://releases.openstack.org/ [2] https://review.opendev.org/#/q/topic:fix-stable-gate+(status:open+OR+status… [3] https://zuul.opendev.org/t/openstack/build/e477d0919ff14eea9cd8a6235ea00c3d… [4] https://review.opendev.org/#/c/726931/ [5] https://opendev.org/openstack/requirements/src/branch/stable/ocata/upper-co… -gmann

8 12

[tripleo][ci] status RED
by Wesley Hayutin 02 Jul '20

02 Jul '20

Greetings, Unfortunately both check and gate are RED atm due to [1]. The issue w/ CirrOS-5.1 was fixed / reverted over the weekend [2]. I expect the check and gate jobs to continue to be RED for the next few days as the investigation proceeds. I would encourage folks to only workflow patches that are critical as the chances you will actually merge anything is not great. [1] https://bugs.launchpad.net/tripleo/+bug/1885286 [2] https://review.opendev.org/#/c/738025/

1 2

[neutron][drivers] Propose Rodolfo Alonso Hernandez for Neutron drivers team
by Slawek Kaplonski 01 Jul '20

01 Jul '20

Hi, Rodolfo is very active Neutron contributor since long time. He has wide knowledge about all or almost all areas of the Neutron and Neutron stadium projects. He is an expert e.g. in ovs agent, pyroute and privsep module, openvswitch firewall, db layer, OVO and probably many others. He also has very good understanding about Neutron project in general, about it's design and direction of development. Rodolfo is also active on our drivers meetings already and I think that his feedback about many things there is very good and valuable for the team. For all these reasons I think that he will be great addition to our drivers team. I will keep this nomination open for a week waiting for Your feedback and votes. -- Slawek Kaplonski Senior software engineer Red Hat

6 6

[cinder] monthly video meeting poll
by Brian Rosmaita 30 Jun '20

30 Jun '20

https://rosmaita.wufoo.com/forms/monthly-video-meeting-proposal/ Please respond before 1200 UTC on Thursday 9 July. cheers, brian

1 0