October 2020 - openstack-discuss - lists.openstack.org

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

[nova][dev] Revisiting qemu emulation where guest arch != host arch
by Apsey, Christopher 05 Jan '22

05 Jan '22

All, A few years ago I asked a question[1] about why nova, when given a hw_architecture property from glance for an image, would not end up using the correct qemu-system-xx binary when starting the guest process on a compute node if that compute nodes architecture did not match the proposed guest architecture. As an example, if we had all x86 hosts, but wanted to run an emulated ppc guest, we should be able to do that given that at least one compute node had qemu-system-ppc already installed and libvirt was successfully reporting that as a supported architecture to nova. It seemed like a heavy lift at the time, so it was put on the back burner. I am now in a position to fund a contract developer to make this happen, so the question is: would this be a useful blueprint that would potentially be accepted? Most of the time when people want to run an emulated guest they would just nest it inside of an already running guest of the native architecture, but that severely limits observability and the task of managing any more than a handful of instances in this manner quickly becomes a tangled nightmare of networking, etc. I see real benefit in allowing this scenario to run natively so all of the tooling that exists for fleet management 'just works'. This would also be a significant differentiator for OpenStack as a whole. Thoughts? [1] http://lists.openstack.org/pipermail/openstack-operators/2018-August/015653… Chris Apsey Director | Georgia Cyber Range GEORGIA CYBER CENTER 100 Grace Hopper Lane | Augusta, Georgia | 30901 https://www.gacybercenter.org

2 2

Help with eventlet 0.26.1 and dnspython >= 2
by Thomas Goirand 04 Oct '21

04 Oct '21

Hi, As you may know, eventlet is incompatible with dnspython >= 2.0.0.0rc1. See [1] for the details. However, Debian unstable has 2.0.0. Would there be some good soul willing to help me fix this situation? I would need a patch to fix this, but I'm really not sure how to start. Cheers, Thomas Goirand (zigo)

8 15

[simplification] Making ask.openstack.org read-only
by Thierry Carrez 27 May '21

27 May '21

Hi everyone, This has been discussed several times on this mailing list in the past, but we never got to actually pull the plug. Ask.openstack.org was launched in 2013. The reason for hosting our own setup was to be able to support multiple languages, while StackOverflow rejected our proposal to have our own openstack-branded StackExchange site. The Chinese ask.o.o side never really took off. The English side also never really worked perfectly (like email alerts are hopelessly broken), but we figured it would get better with time if a big community formed around it. Fast-forward to 2020 and the instance is lacking volunteers to help run it, while the code (and our customization of it) has become more complicated to maintain. It regularly fails one way or another, and questions there often go unanswered, making us look bad. Of the top 30 users, most have abandoned the platform since 2017, leaving only Bernd Bausch actively engaging and helping moderate questions lately. We have called for volunteers several times, but the offers for help never really materialized. At the same time, people are asking OpenStack questions on StackOverflow, and sometimes getting answers there[1]. The fragmentation of the "questions" space is not helping users getting good answers. I think it's time to pull the plug, make ask.openstack.org read-only (so that links to old answers are not lost) and redirect users to the mailing-list and the "OpenStack" tag on StackOverflow. I picked StackOverflow since it seems to have the most openstack questions (2,574 on SO, 76 on SuperUser and 430 on ServerFault). We discussed that option several times, but I now proposed a change to actually make it happen: https://review.opendev.org/#/c/746497/ It's always a difficult decision to make to kill a resource, but I feel like in this case, consolidation and simplification would help. Thoughts, comments? [1] https://stackoverflow.com/questions/tagged/openstack -- Thierry

10 18

[stein][neutron] gratuitous arp
by Ignazio Cassano 13 May '21

13 May '21

Hello, I have a problem on stein neutron. When a vm migrate from one node to another I cannot ping it for several minutes. If in the vm I put a script that ping the gateway continously, the live migration works fine and I can ping it. Why this happens ? I read something about gratuitous arp. Please, help me ? Any workaround , please ? Best Regards Ignazio

3 33

[ironic] Announcing deprecation of the iSCSI deploy interface
by Dmitry Tantsur 04 May '21

04 May '21

Hi all, Following up to the previous mailing list [1] and virtual meetup [2] discussions, I would like to announce the plans to deprecate the 'iscsi' deploy interface. This is the updated plan discussed on the virtual meetup: 1) In the Victoria cycle (i.e. right now): - Fill in the detected feature gaps [3]. - Switch off the iscsi deploy interface by default. - Change [agent]image_dowload_source to HTTP by default. - Give the direct deploy a higher priority, so that it's used by default unless disabled. - Mark it as deprecated in the code (causing warnings when enabled). - Release a major version of ironic to highlight the defaults changes. 2) In the W cycle: - Keep the iscsi deploy deprecated. - Listen to operators' feedback. 3) In the X cycle - Remove the iscsi deploy completely from ironic and IPA. - Remove support code from ironic-lib with a major version bump. Please let us know if you have any questions or concerns. Dmitry [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016681.h… [2] https://etherpad.opendev.org/p/Ironic-Victoria-midcycle [3] https://storyboard.openstack.org/#!/story/2008075 -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

1 1

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[largescale-sig] OpenStack DB Archiver
by Arnaud Morin 10 Feb '21

10 Feb '21

Hello large-scalers! TLDR: we opensource a tool to help reducing size of databases. See https://github.com/ovh/osarchiver/ Few months ago, we released a tool, name osarchiver, which we are using on our production environment (at OVH) to help reduce the size of our tables in mariadb (or mysql) In fact, some tables are well know to grow very quickly. We use it, for example, to clean the OpenStack mistral database from old tasks, actions and executions which are older than a year. Another use case could be to archive some data in another table (e.g. with _archived as suffix) if they are 6 months old, and delete this data after 1 year. The source code of this tool is available here: https://github.com/ovh/osarchiver/ We were wondering if some other users would be interested in using the tool, and maybe move it under the opendev governance? Feel free to contact us and/or answer this thread. Cheers, -- Arnaud, Pierre-Samuel and OVH team

12 18

[qa][tempest] Update language in tempest code base
by Arx Cruz 05 Jan '21

05 Jan '21

Hello, I would like to start a discussion regarding the topic. At this moment in time we have an opportunity to be a more open and inclusive project by eliminating outdated naming conventions from tempest codebase, such as blacklist, whitelist. We should take the opportunity and do our best to replace outdated terms with their more inclusive alternatives. As you can see in [1] the TripleO project is already working on this initiative, and I would like to work on this as well on the tempest side. Any thoughts? Shall I start with a sepc, adding deprecation warnings? [1] https://review.opendev.org/#/c/740013/1/specs/victoria/renaming_rules.rst Kind regards, -- Arx Cruz Software Engineer Red Hat EMEA <https://www.redhat.com> arxcruz(a)redhat.com @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

6 9

Re: [placement] Train upgrade warning
by Seth Tunstall 21 Dec '20

21 Dec '20

Hello, As another person has mentioned (Chanu Romain), I am attempting an upgrade from Queens to Train. I have successfully upgraded Keystone and Glance, but while trying to separate Placement's databases out from Nova, I run into the the same issue with placement-status upgrade check: # placement-status upgrade check +-----------------------------------------------------------------+ | Upgrade Check Results | +-----------------------------------------------------------------+ | Check: Missing Root Provider IDs | | Result: Success | | Details: None | +-----------------------------------------------------------------+ | Check: Incomplete Consumers | | Result: Warning | | Details: There are -3511 incomplete consumers table records for | | existing allocations. Run the "placement-manage db | | online_data_migrations" command. | +-----------------------------------------------------------------+ Likewise, running placement-manage db online_data_migrations does nothing. I am running this placement migration on a dev cluster which has had the databases from a Prod cluster restored to it. This has worked well for me previously as a way to run through upgrades without touching the live system. Thanks very much, Seth

3 10

[ptl][release][stable][EM] Extended Maintenance - Stein
by Előd Illés 26 Nov '20

26 Nov '20

Hi, As Victoria was released yesterday and we are in a less busy period, it is a good opportunity to call your attention to the following: In less than a month Stein is planned to enter into Extended Maintenance phase [1] (planned date: 2020-11-11). I have generated the list of *open* and *unreleased* changes in *stable/stein* for the follows-policy tagged repositories [2]. These lists could help the teams, who are planning to do a *final* release on Stein before moving stable/stein branches to Extended Maintenance. Feel free to edit and extend these lists to track your progress! * At the transition date the Release Team will tag the *latest* (Stein) releases of repositories with *stein-em* tag. * After the transition stable/stein will be still open for bugfixes, but there won't be any official releases. NOTE: teams, please focus on wrapping up your libraries first if there is any concern about the changes, in order to avoid broken releases! Thanks, Előd [1] https://releases.openstack.org [2] https://etherpad.openstack.org/p/stein-final-release-before-em

2 2

[kolla][ptg] Kolla Wallaby priorities
by Mark Goddard 23 Nov '20

23 Nov '20

Hi, Thanks to everyone who attended the Kolla PTG sessions, we covered a lot of ground. The Etherpad is available at [1]. I wrote an email [2] summarising the discussions. I have moved candidate work items across to a second Etherpad [2] for voting on. Now is the time to vote! Please add your name against a maximum of 12 items across the 3 deliverables (kolla, kolla-ansible, kayobe) in the priorities [2] Etherpad. Anyone in the community is welcome to vote. [1] https://etherpad.opendev.org/p/kolla-wallaby-ptg [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018445.… [3] https://etherpad.opendev.org/p/kolla-wallaby-priorities Thanks, Mark

1 3

[release] Meeting Time Poll
by Herve Beraud 19 Nov '20

19 Nov '20

Hello We have a few regular attendees of the Release Management meeting who have conflicts with the current meeting time. As a result, we would like to find a new time to hold the meeting. I've created a Doodle poll[1] for everyone to give their input on times. It's mostly limited to times that reasonably overlap the working day in the US and Europe since that's where most of our attendees are located. If you attend the Release Management meeting, please fill out the poll so we can hopefully find a time that works better for everyone. Thanks! [1] https://doodle.com/poll/7zgvfb8ne54bwhzw -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud -----BEGIN PGP SIGNATURE----- wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+ Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+ RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G 5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0 qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3 B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O v6rDpkeNksZ9fFSyoY2o =ECSj -----END PGP SIGNATURE-----

1 1

[SDK][OSC] Meeting Times
by Kendall Nelson 16 Nov '20

16 Nov '20

Hello! In the PTG session today we talked about doing some sort of regular meetings to kind of jump start things after the more recent lack of activity. We discussed beginning with more frequent meetings (a couple across the next few weeks) and then shifting to a more relaxed monthly cadence. I propose 15:00 UTC on the third thursday of each month as our monthly meeting time? If people agree, I will propose a patch :) As for the more frequent ones to keep the momentum from discussion from this week/last week. We could use that time (15 UTC) and date (Thursdays) and maybe just do them on Nov 12 and 19th to keep us moving? That gives us next week to catch up/ take a breath from the last two weeks, but not waiting till after the US Thanksgiving holiday. Let me know what you think! -Kendall Nelson (diablo_rojo)

4 6

CloudKitty deployment on Train and/or Ussuri
by Thode Jocelyn 15 Nov '20

15 Nov '20

Hi, I was looking at the possibilities to have Rating-as-A-Service in our Openstack (currently train but planning to move to Ussuri) and I stumbled upon CloudKitty. I'm currently facing multiple issues with cloudkitty itself and with openstack-ansible. (We are using a containerized deployment with LXC) * I noticed that openstack-ansible has no playbook yet to install a service like CloudKitty even though starting from Ussuri it was added in the ansible-role-requirements.yml. I have created a small patch to be able to install CloudKitty via openstack-ansible and if this is of interest I could potentially submit a PR with this. * The os_cloudkitty role seems to kinda work and only with a relatively old version of CloudKitty. For example, here : https://opendev.org/openstack/openstack-ansible-os_cloudkitty/src/branch/st… since CloudKitty v8.0.0 the section should be named "[fetcher_keystone]", but I found no information as to which CloudKitty version should be used with this role. Does someone have any recommendation as to which version should be used and if there is any interest in improving the role to support more recent versions of CloudKitty? * Even when tweaking the configuration and installing v7.0.0 of Cloudkitty it only fixes cloudkitty-processor, I was never able to make cloudkitty-api work I always get an error like "Fatal Python error: Py_Initialize: Unable to get the locale encoding ModuleNotFoundError: No module named 'encodings'". Any input on this would be greatly appreciated. Cheers Jocelyn

4 5

[neutron][stable]Proposing Rodolfo Alonso Hernandez as a stable branches core reviewer
by Slawek Kaplonski 06 Nov '20

06 Nov '20

Hi, I would like to propose Rodolfo Alonso Hernandez (ralonsoh) to be new member of the Neutron stable core team. Rodolfo works in Neutron since very long time. He is core reviewer in master branch already. During last few cycles he also proved his ability to help with stable branches. He is proposing many backports from master to our stable branches as well as doing reviews of other backports. He has knowledge about stable branches policies. I think that he will be great addition to our (not too big) stable core team. I will open this nomination open for a week to get feedback about it. -- Slawek Kaplonski Principal Software Engineer Red Hat

6 6

[openstack][ptg][InteropWG] : Conclusion of InteropWG session 10/26 - next one will be 10/30
by prakash RAMCHANDRAN 06 Nov '20

06 Nov '20

Hi all My apology on the delayed start and here is the items we went over on first session. Refer : https://etherpad.opendev.org/p/interop-wallaby-ptg (Monday 10/26 recorded the meeting for everyone to see, its cloud recording and will know from PTG hosts where they have that?) Summary:1. Plan to Complete Victoria guidelines specs for interop in 2-6 weeks depending on adding new [OpenStack Powered File System Branding as Add-on Program] - last Ussuri guidelines are at https://opendev.org/osf/interop/src/branch/master/2020.06.json New one will be 2020.10.json to be added - For existing integrated modules [ Core OpenStack services including identity, compute, networking, block storage, and object storage ] - for two existing add-ons [Orchestration (heat) , DNS (designate) "Not a core capability, add-on program is available" - This is for Manila Filesystem add-on https://etherpad.opendev.org/p/wallaby-ptg-manila-interop "Not a core capability, add-on program to be made available" We have one more opportunity on 10/30 slot at the end of PTG if BMaaS (Ironic) if Julia has any proposals, besides (Open Infra Kata + Docker) Container has additional proposal community is working on. Look forward to seeing you all on October 30 as what the Road-map will look like for 2020 and 2021 going forward. ThanksPrakash On Monday, October 26, 2020, 06:09:00 AM PDT, Goutham Pacha Ravi <gouthampravi(a)gmail.com> wrote: Hi Prakash, We're here: https://zoom.us/j/92649902134?pwd=a01aMXl6ZlNEZDlsMjJMTGNMVUp1UT09 On Mon, Oct 26, 2020 at 6:06 AM prakash RAMCHANDRAN <pramchan(a)yahoo.com> wrote: https://meetpad.opendev.org/Interop-WG-weekly-meeting

3 5

Editing libvirt xml file
by vladimir franciz blando 04 Nov '20

04 Nov '20

Hi, I edited the libvirt xml (virsh edit...) file of a running Windows server instance because I changed the video model type from cirrus to vmvga so it can support a higher resolution on a console (cirrus supports up to 1280 only). After editing, i "soft-rebooted" the instance and the new configuration sticks. That's awesome. But when you do a "hard-reboot" it will revert back to the original config. I also tried to do a "shut-down" then "start" again, but it still reads the original config. Not sure where it is reading the config from... Regards Vlad

3 5

[ptl] Victoria Release Community Meeting
by helena＠openstack.org 04 Nov '20

04 Nov '20

Hi Everyone, We are looking to do a community meeting following The Open Infrastructure Summit and PTG to discuss the Victoria Release. If you’re a PTL, please let me know if you’re interested in doing a prerecorded explanation of your project’s key features for the release. We will show a compilation of these recordings at the community meeting and follow it with a live Q&A session. Post community meeting we will have this recording live in the project navigator. Cheers, Helena

8 10

[tripleo][ci] Monday Nov 2
by Wesley Hayutin 04 Nov '20

04 Nov '20

Greetings, The tripleo ci team has identified a handful of patches that we'd like to land prior to Nov 2, the day docker.io goes away. We've hit some new bugs and have also tuned a few things to try and make sure we can get patches to merge. Our current focus is across master, victoria, ussuri and centos-8 train, and queens while reducing coverage in rocky and stein. A list of the prioritized gerrit reviews can be found here: https://hackmd.io/MlbZ_izSTEuZsCWTJvu_Kg?view The entire topic can be found here: https://review.opendev.org/#/q/topic:new-ci-job Thanks all.

4 4

[placement][nova][cinder][neutron][blazar] Placement governance switch(back)
by Stephen Finucane 03 Nov '20

03 Nov '20

Placement has been a separate project with its own governance since Stein [1]. Since then, the main drivers behind the separation have moved onto pastures new and with Tetsuro sadly declaring his non- candidacy for the PTL position for Wallaby [2], we're left in the unenviable position of potentially not having a PTL for the Wallaby cycle. As such, it's probably time to discuss the future of Placement governance. Assuming no one steps forward for the Placement PTL role, it would appear to me that we have two options. Either we look at transitioning Placement to a PTL-less project, or we move it back under nova governance. To be honest, given how important placement is to nova and other projects now, I'm uncomfortable with the idea of not having a point person who is ultimately responsible for things like cutting a release (yes, delegation is encouraged but someone needs to herd the cats). At the same time, I do realize that placement is used by more that nova now so nova cores and what's left of the separate placement core team shouldn't be the only ones making this decision. So, assuming the worst happens and placement is left without a PTL for Victoria, what do we want to do? Stephen PS: Apologies if I missed other projects with an interest in placement. I did try to catch them all /o\ [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002575… [2] http://lists.openstack.org/pipermail/openstack-discuss/2020-September/01749…

8 10

[tripleo] please avoid creating ansible playbooks that call modules for trivial tasks
by Bogdan Dobrelya 02 Nov '20

02 Nov '20

Hi, In today TripleO client development we invoke ansible playbooks instead of Mistral workflows. I believe this may only be justified for non-trivial things. Otherwise please just import python modules into the client command actions without introducing additional moving parts in the form of ansible modules (that import the same python modules as well). Ansible brings declarative DSL that sometimes can better and simpler and faster describes things to do and shape inputs data, than if the same would be done in python, in the client. What should be the trivial thing and non-trivial then? Well, I think that example [0] illustrates such a trivial thing. A new playbook [1] would only install a few packages then "returns" back into python and calls a new ansible module [2]. And that new module only wraps [3] the same code being removed from the client: from tripleo_common.image import build ... manager = build.ImageBuildManager(<args>) I see no point of adding a new playbook and a new module for that trivial example. Those 4 packages could (and should) be as well installed from the client caller code without any ansible involved in the middle IMO. [0] https://review.opendev.org/#/c/759954/ [1] https://review.opendev.org/#/c/759953/3/tripleo_ansible/playbooks/cli-overc… [2] https://review.opendev.org/#/c/759953/3/tripleo_ansible/ansible_plugins/mod… -- Best regards, Bogdan Dobrelya, Irc #bogdando

6 9

[tripleo] stable/victoria for tripleo repos
by Marios Andreou 02 Nov '20

02 Nov '20

Hello all, I got some pings about tripleo stable/victoria so here is an update. The tripleo-ci team is busy preparing and we are almost ready. Basically we've blocked branching until we have adequate CI coverage. This means check & gate jobs but also promotion jobs so that check & gate aren't running with stale content. tl;dr We are aiming to branch victoria tripleoclient & common today and everything else next week - with more details below but please note: PLEASE AVOID MERGING to the stable/victoria tripleoclient/common that will appear after [4] until we are confident that our ci coverage is working well. Status: Most of our 'core' (standalone/scenarios/undercloud etc) check/gate jobs match on branch like [1] so we expect those to run OK once there is a stable/victoria available. There is ongoing work for branchful, upgrade, 3rd party and ovb jobs but we will not block branching on those. The victoria integration pipeline [2] (which produces current-tripleo) is in good shape. It was blocked on a nit in the component pipeline but it is resolved with [3]. Once we have a victoria promotion then we should be all clear to branch stable/victoria. Actions: A). today: make stable/victoria for python-tripleoclient & tripleo-common. The change at [4] will do this once merged. We will use this to see our check/gate jobs run against victoria & fix issues. B). next week: once we are happy with A). AND we've had a promotion, we release and make stable/victoria for everything else. Hopefully B). will happen end of next week but that depends on how well the above goes (plus inevitable delays cos PTG), I'll repeat this cos it's important: PLEASE AVOID MERGING to stable/victoria tripleoclient/common for now until we are confident that our jobs are doing the right thing! As always grateful for a sanity check or any thoughts about any of the above - I'm pretty new to this (weshay has been a great help so far thank you weshay++), thanks, marios [1] https://opendev.org/openstack/tripleo-ci/src/commit/d5514028452f9d427949f5a… [2] https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integ… [3] https://review.rdoproject.org/r/#/c/30654/ [4] https://review.opendev.org/#/c/759449/

1 2

Python 3.9 is here in Debian Sid
by Thomas Goirand 02 Nov '20

02 Nov '20

Hi there! Matthias Klose decided that uploading Python 3.9 on the 14th of this month (yes, the same day of the OpenStack release) would be fun. I am now really having fun ... fixing bugs! :) What's not so funny for me, is that some key packages like Greenlet or lxml aren't built yet for Python 3.9, even though they should. As a consequence, I just need to wait to be able to upload to Unstable. So this will delay my usual announcement for the release GA in Debian (even though the unofficial Victoria repo for Buster is ready and I could spawn VMs on Buster + Victoria). Anyways, all of this will settle slowly. Though Python 3.9 is here to stay, and to soon reach Ubuntu as well (it's planned for the next release). It'd be nice if projects were starting to investigate gating on Python 3.9 as early as possible. As much as I can tell, there's not so many issues to fix. The first one I fixed is here: https://review.opendev.org/758237 but I don't expect much more, and hopefully, I'll be able to propose more patch as I find the issues. Must we wait until Python 3.9 reaches Ubuntu to enable gating in the OpenStack projects? FYI, I do expect Victoria to run over Python 3.9 in Bullseye. Please don't reply with the usual "Victoria doesn't support it", because that's irrelevant. I'm fine with backporting patches, as long as they also exist in Wallaby. Cheers, Thomas Goirand (zigo)

7 9

[magnum] hyperkube deprecation situation
by Ionut Biru 02 Nov '20

02 Nov '20

Hi guys, I was hoping that in Victoria there will be a decision regarding this topic. Currently, if I use vanilla magnum, I'm stuck on 1.18.6 being the latest version, even so 1.18.10 was released. We are kinda stuck on a version that maybe in the near future is pruned to security issues. How do other operators are using magnum (vanilla) and keeping their kubernetes up to date? Are operators using a forked version of https://review.opendev.org/#/c/752254/ ? -- Ionut Biru - https://fleio.com

2 2

Fwd: [MURANO] DevStack Murano "yaql_function" Error
by İzzettin Erdem 01 Nov '20

01 Nov '20

Thanks Andy. Regards. Forwarded Conversation Subject: [MURANO] DevStack Murano "yaql_function" Error ------------------------ Gönderen: Andy Botting <andy(a)andybotting.com> Date: 30 Nis 2020 Per, 14:13 To: İzzettin Erdem <root.mch(a)gmail.com> Cc: Rong Zhu <aaronzhu1121(a)gmail.com>, < openstack-discuss(a)lists.openstack.org> Hi İzzettin, Thanks for your interest. Complete log link is at below. It is happening > when i try to deploy or update an environment. > > http://paste.openstack.org/show/792444/ > I've encountered the error AttributeError: 'method' object has no attribute '__yaql_function__' in our environment recently. I believe it started after we upgraded to the Stein release. I haven't tracked it down yet, but calls to the YAQL function: std:Project.getEnvironmentOwner() in a Murano app seem to generate it. It's good to know that at least someone else has reproduced this. cheers, Andy ---------- Gönderen: İzzettin Erdem <root.mch(a)gmail.com> Date: 2 May 2020 Cmt, 16:08 To: Andy Botting <andy(a)andybotting.com> Hi Andy, I solved this problem, it is happening when you try to create or update an environment on existing network. If you change the network with "create new" parameter, it stuck on "creating -or updating- environment" state and after that throws connection error. So i searched Murano network errors and i found this: https://docs.oracle.com/cd/E73172_01/E73173/html/issue-21976631.html. I applied these steps and i created successfully an environment just only with "create new" network parameter. After all these, i realized my devstack environment has no internet connection and because of this murano-agent could not install. So i tried to install Murano on my OSA stable/train test environment and i came with success until last step. Murano-agent installed but this time agent could not connect to rabbitmq instance. I discussed this with OSA IRC channel and i think i have to struggle with this for a long time. Apologize for this long mail but i wanted to inform you. Thanks. Regards. ---------- Gönderen: Andy Botting <andy(a)andybotting.com> Date: 3 May 2020 Paz, 14:24 To: İzzettin Erdem <root.mch(a)gmail.com> Hi İzzettin, Glad you got it sorted. I think you've probably just worked around the issue. Using your new method, you're probably just avoiding the std:Project.getEnvironmentOwner YAQL call now. I've also worked around it in our environment, but I'd like to solve it properly. When I get some time to look into it properly, I'll let you know. cheers, Andy

3 3

[tc][all][ Wallaby Virtual PTG Summary
by Ghanshyam Mann 31 Oct '20

31 Oct '20

Hello Everyone, I am summarizing the TC PTG discussion and action items. Wallaby Leaderless assignment --------------------------------------- Four projects are still pending for leadership assignment. We discussed the next step. If you would like to help any of these or using in your cloud/distro, this is the right time to step up: 1. Karbor - Action item: diablo_rojo will start the retirement process. 2. Qinling - Action item: gmann to start the retirement process. 3. Searchlight -Action item: gmann to start the retirement process. 4. Placement - Action item: gibi will update TC about the final discussion and accordingly mnaser to propose the patch in governance. Victoria Retrospective: ----------------------------- There are a couple of things TC finished in last cycle, few of them are: * Reduced size of TC to speed up the things. * Updated policy to add projects faster. * Outlined Distributed Project Leadership Model as an alternate leadership solution of PTL. Oslo will be the first project to try this new model. * Normalized retirement processes. All the retired repos cleanup is finished. * Tag cleanups & additions This is not complete yet but started with tc-approved release tag in the Victoria cycle. * Merged UC with TC. This is one of the key things finished and the best strategy to make users-developers work more closely. TC Meeting time: ----------------- TC is going to try (re-try) the weekly meeting every Thursday 15:00 UTC. * Action Items: mnaser: propose patch. TC policy regarding OpenStack client: ----------------------------------------------- Forum Discussion: https://etherpad.opendev.org/p/vSummit2020_FeatureGap This is one of the important items and still no good progress on this. From TC perspective, we talked about how to ask or motivate projects to migrate to OSC. TC needs to be very clear on whether it's a strict policy or just guidelines. After long discussion on this we are going with the below strategy: These strategies will be documented as TC resolution. * All user docs should have `openstack` client commands. * Focus on user docs first and then ops docs later * All ci-tooling should have `openstack` client usage * Use the process to find what is missing so we can prioritize it. * Action items: - diablo_rojo to work on initial resolution proposal[1] Finalize the Wallaby cycle goal selection: --------------------------------------------------- There are two proposals for the Wallaby community-wide goals. First is 'oslo.rootwrap to oslo.privsep' which is already being discussed in the last cycle also and all set to be selected[2]. 2nd proposal came up during PTG itself. During policy-popup PTG[3], it came up that deprecating the JSON formatted of policy file will be a good advance step before the projects move to new RBAC policies. This will help operators to smoothly migrate to new policies. This does not involve much work and ok to select as 2nd goal for Wallaby cycle, TC agreed to have below goals for Wallaby cycle: * Migrate from oslo.rootwrap to oslo.privsep [2] * Migrate Default Policy Format from JSON to YAML [4] *Action: - TC to merge the patches which selects both goals for Wallaby cycle. TC tag cleanup: ------------------- In the Victoria cycle, we started the process to audit all the TC tags and start cleanup those. We removed the 'tc:approved-release' tag in the Victoria cycle[5]. In this PTG we discussed two more tags. 1. assert:supports-zero-downtime-upgrade: Currently, there is no project who has this tag and also no testing framework available. Testing for zero downtime is not so easy in upstream testing. We decided to remove this tag as it is advertising something we aren't doing. If anyone interested to spend time on this in the future then we can add it after projects start testing it and document it. 2. assert:supports-api-interoperability: This tag is whether project API is interoperable also or not and this is important from the interop trademark program also. We only have Nova having this tag. Our goal is to promote more projects to apply for this tag. During the discussion, we found that we need to clarify this tag more clearly. For example, this tag is not about implementing the microversion but any versioning schema which provides the feature (API changes) discoverability. And also it is about how we change the API not how our APIs are currently. As long as services have some versioning mechanism to discover the changes and follow the API SIG guidelines for interoperability, and test it via branchless testing way, that service is applicable to apply for this tag. TC will document this tag in a more clear way and encourage each project to start working on applying this tag. * Action: - Graham to put up a change to update the wording to allow for discovery mechanisms[6] - Should include the release in which they started to assert this tag. - also documents that it's about following the guidelines for future changes not existing one. - gmann - Should socialize this out to the projects after we get the documentation improvements landed. - gmann k8s Community Bridging : -------------------------------- This is one of the exciting discussion for everyone. We hosted this cross-community meeting with Kubernetes steering committee teams. Bob and dims from the Kubernetes steering committee joined us in this discussion. It was started with a quick introduction from both teams and then started the discussion on the below topics [11] * Sharing governance models: k8s governance hierarchy is LF->CNCF->k8s steering commitee->various SIG/working group. k8s Steering the committee consists of 7 elected members and doesn't actually influence the direction of the code instead leave it to SIG and arch committee. There is no chair in this committee and host biweekly private as well as public meetings. Each SIG (repos) team has approver and reviewer roles where reviewer review the code and approver are responsible for merging the code. Naser explained the OpenStack governance model. * What are your 3 biggest issues that are barriers to growth/sustainability as a community? as a project? - Going up the contribution ladder is hard. - Stale/emeritus membership in reviewer/approver level groups. - Mono repo makes it so that each SIG involved needs to review which can slow things down. * Area/Domain mapping vs service/repo centered teams - mono repo forces people to work together and it's a little more diverse who is reviewing - mono repo is bad because sometimes things need to be in multiple places like documentation - mono repo struggles with libraries especially on testing the external one. * Related Ecosystem projects comparisons The SIG lead has to sign off and accept that they are willing to take ownership + handle maintenance + releasing etc. It is general consensus that we try to distribute much work as possible to the subgroup and keep it out of k/k. There is general consensus across k8s leadership that work should be delegated out to subgroups. For API interoperability challenge in a distributed model, k8s has conformance tests which exercise the API performance and vendors try and upload conformance results every release or dot release. * Common FOSS topics release and CI part was discussed. and also on how COVID things are impacting community health. k8s community almost lost their independents or part-time contributors. also, the k8s community is doing 3 releases per year compared to 4. * Staying connected going forward To stay connected it will be a good idea if we extend an invite for K8s to join PTG. Upstream Investment Opportunities: ----------------------------------------------- We have three upstream opportunities defined for 2020[1] but there is no help on any of these, even in previous (2018, 2019) upstream opportunities also. We started the discussion if we need to continue this for 2021 years also or just stop defining it and decide the area to help when we have someone interested. Before deciding anything on this mnaser will discuss this with the board of directors and get their opinion. * Action Items: - mnaser to find ways to engage with members and talk with them. - mnaser to talk to board about infrastructure - mnaser to talk to board about platinum members contribution levels + what are they doing to drive engagement. Pop Up Team Check In: ----------------------------- Currently, we have two active popup teams 1. policy 2. Encryption. TC checked the progress on both teams. The policy team is very active and finished some work in Voctoria cycle (cyborg finished it and Barbican started) and also hosted forum[8], PTG sessions[9], and discussed Wallaby development plan. This team host biweekly meeting to discuss and review the progress. The encryption team is also active. Josephine explained the progress on this. Glance spec is merged. Both teams will continue in Wallaby cycle also. Completing the Gerrit breach audit : ---------------------------------------------- There are still 19 teams pending to finish this audit. Those are listed in etherpad - https://etherpad.opendev.org/p/code-audit-gerrit-breach-tracker We encourage all those pending teams to finish the audit, TC members will start following with those projects every week. * Action: - TC to follow up every week on progress Other topics: ----------------- We ran out of time and all the pending (below) topics will be discussed in TC regular meetings. TC will skip this month's meeting[10] but will have weekly meetings on 12th Nov onwards. *Better way to test defined testing runtime *Monitoring in OpenStack: Ceilometer + Telemetry + Gnocchi state *Stable core team maintainer and process to recruit the new members and adding core team members in stable core list. Detailed discussions are in this Etherpad: https://etherpad.opendev.org/p/tc-wallaby-ptg [1] https://review.opendev.org/759904 [2] https://review.opendev.org/#/c/755590/ [3] https://etherpad.opendev.org/p/policy-popup-wallaby-ptg [4] https://review.opendev.org/#/c/759881/ [5] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/017786.… [6] https://review.opendev.org/760562 [7] https://governance.openstack.org/tc/reference/upstream-investment-opportuni… [8] https://etherpad.opendev.org/p/consistent-and-secure-default-policies-walla… [9] https://etherpad.opendev.org/p/policy-popup-wallaby-ptg [10] http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018439.… [11] https://etherpad.opendev.org/p/kubernetes-cross-community-topics -gmann

1 0

devstack installation error - [qa-octavia]
by Ratnopam Chakrabarti 31 Oct '20

31 Oct '20

I tried installing devstack in ubuntu 20.04 with no luck. Are the instructions > at https://docs.openstack.org/devstack/latest/ and > https://docs.openstack.org/devstack/latest/guides/devstack-with-lbaas-v2.ht… > still valid? Could anyone please confirm?

2 3

Weird iptables problem
by frommel＠gmail.com 31 Oct '20

31 Oct '20

Hi, so we deployed a test cluster with 8 total nodes. Openstack ussuri, Ubuntu 20.04 , opevswitch, octavia. Everything works more or less as it should , however when deploying a loadbalancer (which works perfectly), instances on the same node as the amphora deployment loose all access via public up after open switch logs a message : modified security group “long uuid”, which is a sec group belonging to octavia. What effect does that have with the rest? Questions: Should ufw be on and started? How can we troubleshoot this more in depth? Why does the lb work perfectly whole instance with floating up loos all access from the internet but the lb works fine passing thru the web services. One more thing. Does anyone have a recommended sysctl.conf Settings list for Ubuntu/ openstack ussuri? Thanks already and have a nice weekend, //f

1 0

[magnum] Keystone trust credentials failing authentication (after previously working)
by Paul Browne 30 Oct '20

30 Oct '20

Hello Magnum users and devs, I was wondering if any may be able to provide some clues on a mystifying problem we've been seeing with some Magnum-deployed k8s clusters, of older deployment date (v1.17, deployed ~6 months ago) and Keystone trusts used to manage OpenStack cloud resources. I've attached a copy of the Magnum template for the k8s cluster to give an idea of its initial environment. The issue we've been seeing is that the Keystone trust generated at Magnum cluster creation time seems no longer be usable to successfully authenticate to the OpenStack APIs, and so operations such as creation of Manila shares used as k8s Persistent Volume Claims then get stuck in Pending state, forever. The strange thing is that this doesn't happen for newly created Magnum clusters of the same template (in attached file, I use trust credentials for a newly deployed and old cluster; OpenStack API calls for new credentials succeed, whereas old cluster trust fails). Unfortunately following debug-level Keystone logging for rejected trust auth attempts hasn't led me much further to understanding the root issue; the Magnum generated trust exists and has no expiry, but it still seems to be rejected. Keystone debug logging doesn't seem to give much indication as to *why* it is being rejected or what policy violations may be involved, but perhaps there's details hidden in there I'm not seeing yet. Does anyone have any clues as to why this may be happening, or any advice on how we may be able to replace or refresh the trust that the Magnum-created k8s cluster is expecting to use in calling OpenStack APIs? Many thanks, Paul -- ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk Tel: 0044-1223-746548 *******************

1 0

[kolla] Wallaby PTG summary
by Mark Goddard 30 Oct '20

30 Oct '20

Hi, Thanks to everyone who joined the virtual PTG sessions this week. We had some great discussions covering many different topics. I will try to provide a summary here. # Victoria cycle retrospective The Ubuntu Focal migration went smoothly. Core reviewer bandwidth was hurt by the pandemic, but we did pick up towards the end of the cycle and landed a good number of features. As usual, lots of CI fires to extinguish. # Review Victoria PTG Actions We looked back at the actions from the last PTG. Quite a few are still incomplete - we should do a better job of reviewing these. We agreed that the docs-related actions are still the most important. Any help with improving our documentation is appreciated. # General topics ## Ability to run CI jobs locally (without Zuul, but possibly with Ansible) This depends on a Zuul feature called Zuul Runner [1], which should be ready soon. yoctozepto signed up to test it out and document how to use it with Kolla. ## Future of Kolla Klub and Kall The Kolla Klub started well, with lots of involvement, and some good sessions. Over time, attendance has fallen, and the agenda still seems to be largely driven by me. Keeping up productive sessions every two weeks is starting to feel like a bit of a chore, and I think we need to change things up or stop. We agreed to try rebooting at a time that better suits contributors in APAC, possibly recycling some earlier material. There was a proposal at the summit to have monthly ops meetups, and maybe we would do well to link these related communities. We had a few ideas for how to improve the productivity of the more development oriented Kolla Kall, including multiple smaller groups, pair/mob programming, and fewer longer sessions. ## Keep or drop lower-constraints jobs? We didn't find many reasons to keep the lower-constraints jobs, and they do seem to need a bit of maintenance. We voted 2 in favour to remove them, with everyone else not having strong feelings. ## How can we integrate better with our user & contributor base outside of EU/US timezones? Kolla has a large user base outside of EU/US timezones, but the core team and surrounding community is not always that effective at involving these users. We agreed to try to improve the situation by reaching out to known community members in other regions, and offering some limited mentorship. # Kolla ## Infra images This feature has been in progress for a few cycles, but is proving difficult to land. The main blocker now is in CI. We need to split our image build CI jobs into several pipelines. They need to share a local docker registry. This pattern should open up a few other interesting features. ## New Docker Hub limits Docker hub is changing its policy for anonymous and free users. This affects retention of unused images, and limits the number of pulls in a 6 hour period (100 for anonymous users, limited by IP, 200 for logged in free users). This is likely to affect users running without a local registry. Fl1nt agreed to add some information to our documentation to help users. This is likely also to impact our CI jobs, which need to pull images. There is an application process to be listed as an open source project, which we need to ask the opendev infra team to apply for. # Kolla Ansible ## More fine grained skipping of tasks, e.g. allow to skip service registration priteau proposed making it easier to skip certain tasks during deployment, such as bootstrapping or service registration. There are two obvious ways to do this - a new 'action' (e.g. deploy-without-register), or add tags to the relevant tasks, which may be skipped. A third approach is to combine two commands - genconfig & deploy-containers. This is related to work currently going to allow configuration to be generated separately from container deployment. In the end, we considered adding tags, as well as the two command approach. ## Deprecate reconfigure command? The reconfigure command is now essentially the same as deploy. This is a source of confusion for users, so we agreed to deprecate reconfigure. We will keep reconfigure as an alias for deploy at the CLI level. ## Modernise the old skool Swift role The Swift role is the last (except Bifrost, which is generally weird), to use the old format, prior to the better-reconfigure blueprint. This has some limitations, so we should bring it into line. Nothing controversial here, just needs someone to pick it up. ## DNS-based endpoint naming rafaelweingartne has proposed a mini spec [2] for DNS-based endpoint naming. I think we now have a reasonable idea of his solution (which is to use Consul for service discovery, and avoid HAProxy for internal API communication), however the hard part is defining the problem that is being solved. ## Support to identity federation (OpenID Connect) configurations in Keystone and Horizon via Kolla-ansible There is a patch [3] to support this which has been open for some time. It requires some fairly specific knowledge to review, and the challenge will be in making core reviewers confident that it works. We agreed that CI testing and/or documentation of a simple development environment would help. ## podman Podman keeps coming up as a replacement for Docker. There are several issues here, including support on Debian/Ubuntu - we do not want to maintain support for two container engines. We agreed that this is likely to be a large feature, and that we should get started with a proof of concept in Wallaby, possibly with a spec to tease out the details. Fl1nt may have some time early 2021 for this, but anyone is welcome to pick it up. ## Finish Masakari integration We don't have support for the host monitor, which is arguably the most useful part of Masakari. With yoctozepto as Masakari PTL, we expect to see good progress in Wallaby. ## Performance improvements We made some good progress with performance improvement in Victoria. There is always more that can be done, and the next phase will probably involve splitting roles that target both control and compute hosts. Mitogen [4] is also an interesting option. As always, changes should be based on benchmarks [5]. ## Let's Encrypt integration - container running certbot, triggers certificate distribution yoctozepto added initial support for the ACME protocol in Ussuri, and the next step is to support deployment of a certbot container, and unattended certificate rotation. There are lots of fiddly details here, including how to bootstrap it. headphoneJames offered to write up a high level description of how the pieces fit together. ## Support for HAProxy reload (e.g. for TLS) Currently any configuration changes to HAProxy require a restart to be applied. This causes connections to be dropped. We discussed adding support for hitless reloads of HAProxy [6]. ## Native fluentd logging We didn't discuss this as kevko was busy, but I thought I'd add it here anyway. The idea is to use use logging.conf to configure services for native fluentd logging. This avoids all the regexes in fluentd, but may change our document schema in Elasticsearch. Patch [7] is up for review. # Kayobe ## Support multiple environments from a single kayobe configuration This feature has been on the cards for some time, and is an important step for Kayobe. priteau has proposed a patch [8] which is a good starting point. We spent a good hour going over some of the outstanding issues, and made some good progress. We agreed on the need to keep the meaning of KAYOBE_CONFIG_PATH intact, and to allow a branch to define the default environment to use. We agreed to write a high level design document to pull out the remaining uncertainties, and aim to get it over the line in Wallaby. ## Switching Docker storage from devicemapper to overlay2 We switched the default Docker storage driver from devicemapper to overlay2 in Victoria. For existing users of devicemapper, the plan is to try to ensure they are migrated to overlay2 during CentOS 8 migration. Further, we will deprecate devicemapper on stable branches, and require users to take some action (set a variable) to allow its use. This should deter new users from using devicemapper, without completely breaking existing users. # Switch to NetworkManager? jovial proposed this discussion, as he likes the checkpoint/restore feature of NetworkManager. There are various ways to interface with NM, including nm-cli, an ansible module, templated config files + nmcli con reload. ## Strip out Grafana post configure functionality and move it to Kolla-Ansible dougszu proposed this topic, to improve the separation of concerns between Kayobe and Kolla Ansible. It may also apply to setup we do for ironic. ## Create a reference custom playbook repo We have a stackhpc repo [9] with various useful playbooks and scripts. dougszu proposed making something more official. I suggested that Ansible collections could be used to improve the user experience with custom playbooks in external repos. ## Config gen on ansible control host This is something we've talked about for a while - offline generation of configuration, and potentially validating it in a CI pipeline. jovial noted the importance of being able to redact passwords. This is mostly dependent on ongoing changes to Kolla Ansible, but could be wrapped up nicely by Kayobe. ## Hashivault integration While we have integrated with Hashicorp vault for secret retrieval, the user experience could be better. We also lack support for pushing passwords generated by kolla up to Vault. This is likely to be a Kolla Ansible feature, plus some Kayobe integration. [1] https://review.opendev.org/681277 [2] https://review.opendev.org/759706 [3] https://review.opendev.org/695432 [4] https://github.com/dw/mitogen [5] https://github.com/stackhpc/ansible-scaling [6] https://www.haproxy.com/blog/hitless-reloads-with-haproxy-howto/ [7] https://review.opendev.org/755775 [8] https://review.opendev.org/734867 [9] https://github.com/stackhpc/kayobe-ops Cheers, Mark

1 0

Devstack installation failure
by Ratnopam Chakrabarti 30 Oct '20

30 Oct '20

Hi, Has anyone successfully installed devstack in last 2 weeks with stack.sh script?It was working well for me before that. I am seeing failures when I try to install on Ubuntu 18.04. Here's the trace - [Call Trace] ./stack.sh:1304:start_glance /home/stack/devstack/lib/glance:480:die [ERROR] /home/stack/devstack/lib/glance:480 g-api did not start Error on exit nova-compute: no process found neutron-dhcp-agent: no process found neutron-l3-agent: no process found neutron-metadata-agent: no process found neutron-openvswitch-agent: no process found git repo i am using to download devstack is https://git.openstack.org/openstack-dev/devstackI tried with with stable/victoria branch also, same issue. Any idea on how to solve this? Thanks

2 1

[tc] monthly meeting skip
by Mohammed Naser 30 Oct '20

30 Oct '20

Hello everyone, Our monthly meeting was scheduled for next Thursday, Nov 5th. Since the PTG took place this week, we will be skipping this month’s meeting and consider the PTG as our monthly meeting instead. Thanks, Regards, Mohammed -- Mohammed Naser VEXXHOST, Inc.

1 0

[ansible][tripleo][openstack-ansible-modules] Openstack Ansible modules PTG wrap up
by Sagi Shnaidman 30 Oct '20

30 Oct '20

Openstack Ansible modules PTG wrap up 1. We continue to align new features in SDK, OSC and modules a) need to complete switch of cloud layer to use ONLY proxy layer and not reimplement the APIs b) in Ansible modules we'd better to use cloud layer when it's available and not proxy layer 2. For exposing SDK logs when running modules, we need to find a right place to print them. Either it would be one of module returns or special plugin printing logs with more verbosity. The discussion will be in the maillist. 3. We will start developing unit tests for modules and run them with ansible-test. 4. Also need to rework our current test system and make it more atomic and add coverage. 5. We decided to align all *_info modules to one standard - <resource>_info, some of them named as <resource>s_info. 6. We'll need to generate documentation for modules and place it on docs.openstack.org if it's possible. Current documentation in Ansible covers the release that is built-in in Ansible and doesn't reflect last changes. 7. Effort of moving modules to use a single OpenstackModule class will continue. 8. Transition and adding new baremetal modules will continue as well. Thanks -- Best regards Sagi Shnaidman

1 0

Re: [Openstack][cinder] dell unity iscsi faulty devices
by Gorka Eguileor 30 Oct '20

30 Oct '20

On 30/10, Ignazio Cassano wrote: > Please, se the last email where I upgraded to the last openstack nova on > queens. > [root@compute-0 nova]# rpm -qa|grep queens > centos-release-openstack-queens-1-2.el7.centos.noarch > [root@compute-0 nova]# rpm -qa|grep nova > openstack-nova-compute-17.0.13-1.el7.noarch > openstack-nova-common-17.0.13-1.el7.noarch > python-nova-17.0.13-1.el7.noarch > python2-novaclient-10.1.0-1.el7.noarch > > I sent you the logs on the update release. > I am not so skilled for reading fine logs output about this issue. > Sorry > Ignazio > Hi, I missed that email and the attachment, sorry. The logs you sent me were missing most of the connect_volume call, and only the end of the call was present, but I think it doesn't matter as I see what the problem is. The problem is that some of the nodes and sessions are duplicated. An example of a duplicated node: tcp: [3] 10.102.189.156:3260,15 iqn.1992-04.com.emc:cx.ckm00200502005.b11 (non-flash) tcp: [4] 10.102.189.156:3260,15 iqn.1992-04.com.emc:cx.ckm00200502005.b11 (non-flash) An example of that node's duplicated session: tcp: [3] 10.102.189.156:3260,15 iqn.1992-04.com.emc:cx.ckm00200502005.b11 (non-flash) tcp: [4] 10.102.189.156:3260,15 iqn.1992-04.com.emc:cx.ckm00200502005.b11 (non-flash) And os-brick is not prepared to handle that, because it is programmed to reuse the nodes and sesssions. So on disconnect it get's the first of each to look for the volumes provided by it. In the example of the duplicated node-session aboveit sees that it provides /dev/sdd, but that is not one of the disks that belong to the multipath that we are disconnecting, so it gets ignored. The volume we are looking for it's probably on the second session. So from this point forward (where we have duplicated node-sessions) it will not work again. I recommend you clean up that system so that you don't have duplicated nodes and sessions before trying to do a VM migration with a single volume attached. If that works, then try to attach 2 volumes on instances on the same host and see if the nodes and sessions are duplicated. Cheers, Gorka. > Il giorno ven 30 ott 2020 alle ore 09:27 Gorka Eguileor <geguileo(a)redhat.com> > ha scritto: > > > On 30/10, Ignazio Cassano wrote: > > > Hello, these are versions we are using: > > > [root@podto2-kvm02 ansible]# rpm -qa|grep queens > > > centos-release-openstack-queens-1-2.el7.centos.noarch > > > [root@podto2-kvm02 ansible]# rpm -qa|grep nova > > > openstack-nova-common-17.0.11-1.el7.noarch > > > python2-novaclient-10.1.0-1.el7.noarch > > > openstack-nova-compute-17.0.11-1.el7.noarch > > > python-nova-17.0.11-1.el7.noarch > > > > > > Cheers,Gorka. > > > > Hi, > > > > That release has the Nova bug fix, so Nova should not be calling Cinder > > to do an initialize connection on the source on the post-migration step > > anymore. > > > > I recommend comparing the connection info passed to connect_volume when > > the volume is attached on the source host and when it's disconnected on > > the post-migration step on the source host. > > > > Cheers, > > Gorka. > > > > > > > > > > > > > [1]: https://bugs.launchpad.net/nova/+bug/1754716 > > > > [2]: > > > > > > https://github.com/openstack/nova/commit/013f421bca4067bd430a9fac1e3b290cf1… > > > > > > > > > > > > > > > > > > > Il giorno gio 29 ott 2020 alle ore 09:12 Gorka Eguileor < > > > > geguileo(a)redhat.com> > > > > > ha scritto: > > > > > > > > > > > On 28/10, Ignazio Cassano wrote: > > > > > > > Hello Gorka, I would like to know if with unity iscsi driver, I > > must > > > > > > > configure iscsi initiator on both compute and controller nodes. > > > > > > > At this time I installed and cinfigured iscsi initiator only on > > > > compute > > > > > > > nodes and I got a lot of faulty devices when volumes ate > > detached. > > > > > > > Thanks > > > > > > > Ignazio > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > Both compute and controller nodes are in the data path. Computes > > when > > > > > > instances use the volumes, and controllers when we create volume > > from > > > > > > images, do generic volume migrations, create or restore backups, > > etc. > > > > > > > > > > > > Unless your deployment isn't doing any of the Cinder operations > > that > > > > > > involve the data plane, you'll have to configure iSCSI on the > > > > controller > > > > > > as well. > > > > > > > > > > > > Having said that, whether you configure the iSCSI initiator or not > > on > > > > > > the controller will have no effect on the paths used by the > > compute. > > > > > > > > > > > > I've seen the iSCSI initiator going crazy when the iscsid and the > > > > > > iscsiadm are from different versions. I've seen this in > > containerized > > > > > > environments. > > > > > > > > > > > > Faulty paths on multipathing is a tricky business, because there > > are > > > > > > different checkers, some generic (readsector0, tur, directio) and > > some > > > > > > vendor specific (emc_clarrion, hp_wd, rdac), and each one behaves > > in a > > > > > > different way. > > > > > > > > > > > > If you have a multipath device with faulty paths, that you think > > should > > > > > > not be faulty, you should look into what's going on with those > > paths: > > > > > > > > > > > > - Confirm that the device is still in the system under /dev/XYZ > > > > > > - Confirm in your storage array's console/webconsole that the > > volume is > > > > > > still mapped on that target-portal to that host's iscsi initiator > > > > > > name. > > > > > > - Confirm you can read the faulty devices with dd on the host > > > > > > - Confirm that the WWN of the device is the same in all the paths > > > > (using > > > > > > /lib/udev/scsi_id) > > > > > > - Finally look into what checker is multipath using for your device > > > > > > (sometimes checkers have bugs). > > > > > > > > > > > > Cheers, > > > > > > Gorka. > > > > > > > > > > > > > > > > > > > > > > > > > > Il Mar 20 Ott 2020, 19:58 Gorka Eguileor <geguileo(a)redhat.com> > > ha > > > > > > scritto: > > > > > > > > > > > > > > > On 20/10, Ignazio Cassano wrote: > > > > > > > > > This is the entre log from when the migration started: > > > > > > > > > > > > > > > > > > http://paste.openstack.org/show/799199/ > > > > > > > > > > > > > > > > > > Ignazio > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > There are no os-brick calls in there. :-( > > > > > > > > > > > > > > > > You should look for the call to connect_volume that should have > > > > > > > > something like: > > > > > > > > > > > > > > > > ==> disconnect_volume: call "{'args': > > > > > > > > (<os_brick.initiator.connectors.iscsi > > > > > > > > > > > > > > > > And the second parameter to that call is a dictionary where you > > > > can see > > > > > > > > the target_lun, target_luns, target_portals, target_portal, > > > > target_iqn, > > > > > > > > target_iqns... This will allow us to check if we are actually > > > > > > connected > > > > > > > > to those targets-portals > > > > > > > > > > > > > > > > The third parameter should contain two things that are > > relevant, > > > > the > > > > > > > > scsi_wwn and the path. You can check if the path exists and if > > > > that > > > > > > > > path actually has that wwn using /lib/udev/scsi_id --page 0x83 > > > > > > > > --whitelisted $path > > > > > > > > > > > > > > > > Those are the things I would check, because the only reason I > > can > > > > think > > > > > > > > that os-brick is not disconnecting any volumes are that the > > > > connection > > > > > > > > info is not right, or that the volume is no longer connected. > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Gorka. > > > > > > > > > > > > > > > > > > > > > > > > > > Il giorno mar 20 ott 2020 alle ore 11:23 Gorka Eguileor < > > > > > > > > geguileo(a)redhat.com> > > > > > > > > > ha scritto: > > > > > > > > > > > > > > > > > > > On 20/10, Ignazio Cassano wrote: > > > > > > > > > > > Hello Gorka,this is what happens on nova compute with > > debug > > > > > > enabled, > > > > > > > > > > when I > > > > > > > > > > > migrate an instance with iscsi volumes ( note > > Disconnecting > > > > > > from[] > > > > > > > > should > > > > > > > > > > > be the issue): > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > The disconnect from [] is the right clue, not necessarily > > the > > > > > > issue. > > > > > > > > > > > > > > > > > > > > OS-Brick is saying that for the connection information > > that has > > > > > > been > > > > > > > > > > passed in the "disconnect_volume" call (which is not > > present > > > > in the > > > > > > > > > > emailed logs) there are no volumes present in the system. > > > > > > > > > > > > > > > > > > > > You should check the connection info that Nova is passing > > to > > > > > > > > > > disconnect_volume and confirm if that data is correct. For > > > > example > > > > > > > > > > checking if the path present in the connection info > > dictionary > > > > is > > > > > > the > > > > > > > > > > same as the one in the instance's XML dump, or if the LUN > > from > > > > the > > > > > > > > > > connection info dict is actually present in the system. > > > > > > > > > > > > > > > > > > > > There are multiple reasons why Nova could be passing the > > wrong > > > > > > > > > > connection info to os-brick. The ones that come to mind > > are: > > > > > > > > > > > > > > > > > > > > - There was a failed migration at some point, and Nova > > didn't > > > > > > rollback > > > > > > > > > > the connection info on the BDM table. > > > > > > > > > > - Nova is calling multiple times initialize_connection on > > > > Cinder > > > > > > for > > > > > > > > the > > > > > > > > > > same host and the driver being used is not idempotent. > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > Gorka. > > > > > > > > > > > > > > > > > > > > > stderr= _run_iscsiadm_bare > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py:1122 > > > > > > > > > > > 2020-10-20 09:52:33.066 132171 DEBUG > > > > > > > > os_brick.initiator.connectors.iscsi > > > > > > > > > > > [-] iscsi session list stdout=tcp: [10] > > 10.138.209.48:3260,9 > > > > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.a3 (non-flash) > > > > > > > > > > > tcp: [11] 10.138.215.17:3260,8 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.a2 > > > > > > > > > > > (non-flash) > > > > > > > > > > > tcp: [12] 10.138.215.17:3260,8 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.a2 > > > > > > > > > > > (non-flash) > > > > > > > > > > > tcp: [13] 10.138.215.18:3260,7 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.b2 > > > > > > > > > > > (non-flash) > > > > > > > > > > > tcp: [14] 10.138.215.18:3260,7 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.b2 > > > > > > > > > > > (non-flash) > > > > > > > > > > > tcp: [15] 10.138.209.47:3260,6 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.b3 > > > > > > > > > > > (non-flash) > > > > > > > > > > > tcp: [16] 10.138.209.47:3260,6 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.b3 > > > > > > > > > > > (non-flash) > > > > > > > > > > > tcp: [9] 10.138.209.48:3260,9 > > > > > > > > iqn.1992-04.com.emc:cx.ckm00184400687.a3 > > > > > > > > > > > (non-flash) > > > > > > > > > > > stderr= _run_iscsi_session > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py:1111 > > > > > > > > > > > 2020-10-20 09:52:33.078 132171 DEBUG > > > > > > > > os_brick.initiator.connectors.iscsi > > > > > > > > > > > [-] Resulting device map defaultdict(<function <lambda> > > at > > > > > > > > > > 0x7f4f1b1f7cf8>, > > > > > > > > > > > {(u'10.138.215.17:3260', > > > > > > > > u'iqn.1992-04.com.emc:cx.ckm00184400687.a2'): > > > > > > > > > > > (set([]), set([u'sdg', u'sdi'])), (u'10.138.209.47:3260 > > ', > > > > > > > > > > > u'iqn.1992-04.com.emc:cx.ckm00184400687.b3'): (set([]), > > > > > > set([u'sdo', > > > > > > > > > > > u'sdq'])), (u'10.138.209.48:3260', > > > > > > > > > > > u'iqn.1992-04.com.emc:cx.ckm00184400687.a3'): (set([]), > > > > > > set([u'sdd', > > > > > > > > > > > u'sdb'])), (u'10.138.215.18:3260', > > > > > > > > > > > u'iqn.1992-04.com.emc:cx.ckm00184400687.b2'): (set([]), > > > > > > set([u'sdm', > > > > > > > > > > > u'sdk']))}) _get_connection_devices > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py:844 > > > > > > > > > > > 2020-10-20 09:52:33.078 132171 DEBUG > > > > > > > > os_brick.initiator.connectors.iscsi > > > > > > > > > > > [-] Disconnecting from: [] _disconnect_connection > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/initiator/connectors/iscsi.py:1099 > > > > > > > > > > > 2020-10-20 09:52:33.079 132171 DEBUG > > > > oslo_concurrency.lockutils > > > > > > [-] > > > > > > > > Lock > > > > > > > > > > > "connect_volume" released by > > > > > > > > > > > "os_brick.initiator.connectors.iscsi.disconnect_volume" > > :: > > > > held > > > > > > > > 1.058s > > > > > > > > > > > inner > > > > > > > > > > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:339 > > > > > > > > > > > 2020-10-20 09:52:33.079 132171 DEBUG > > > > > > > > os_brick.initiator.connectors.iscsi > > > > > > > > > > > [-] <== disconnect_volume: return (1057ms) None > > > > > > trace_logging_wrapper > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/utils.py:170 > > > > > > > > > > > 2020-10-20 09:52:33.079 132171 DEBUG > > > > > > nova.virt.libvirt.volume.iscsi > > > > > > > > [-] > > > > > > > > > > > [instance: 0c846f66-f194-40de-b31e-d53652570fa7] > > Disconnected > > > > > > iSCSI > > > > > > > > > > Volume > > > > > > > > > > > disconnect_volume > > > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/nova/virt/libvirt/volume/iscsi.py:78 > > > > > > > > > > > 2020-10-20 09:52:33.080 132171 DEBUG os_brick.utils [-] > > ==> > > > > > > > > > > > get_connector_properties: call u"{'execute': None, > > 'my_ip': > > > > > > > > > > > '10.138.208.178', 'enforce_multipath': True, 'host': > > > > > > > > 'podiscsivc-kvm02', > > > > > > > > > > > 'root_helper': 'sudo nova-rootwrap > > /etc/nova/rootwrap.conf', > > > > > > > > 'multipath': > > > > > > > > > > > True}" trace_logging_wrapper > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/utils.py:146 > > > > > > > > > > > 2020-10-20 09:52:33.125 132171 DEBUG > > > > os_brick.initiator.linuxfc > > > > > > [-] > > > > > > > > No > > > > > > > > > > > Fibre Channel support detected on system. get_fc_hbas > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/initiator/linuxfc.py:157 > > > > > > > > > > > 2020-10-20 09:52:33.126 132171 DEBUG > > > > os_brick.initiator.linuxfc > > > > > > [-] > > > > > > > > No > > > > > > > > > > > Fibre Channel support detected on system. get_fc_hbas > > > > > > > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/initiator/linuxfc.py:157 > > > > > > > > > > > 2020-10-20 09:52:33.145 132171 DEBUG os_brick.utils [-] > > <== > > > > > > > > > > > get_connector_properties: return (61ms) {'initiator': > > > > > > > > > > > u'iqn.1994-05.com.redhat:fbfdc37eed4c', 'ip': > > > > u'10.138.208.178', > > > > > > > > 'system > > > > > > > > > > > uuid': u'4C4C4544-0051-4E10-8057-B6C04F425932', > > 'platform': > > > > > > > > u'x86_64', > > > > > > > > > > > 'host': u'podiscsivc-kvm02', 'do_local_attach': False, > > > > 'os_type': > > > > > > > > > > > u'linux2', 'multipath': True} trace_logging_wrapper > > > > > > > > > > > /usr/lib/python2.7/site-packages/os_brick/utils.py:170 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best regards > > > > > > > > > > > Ignazio > > > > > > > > > > > > > > > > > > > > > > Il giorno gio 15 ott 2020 alle ore 10:57 Gorka Eguileor < > > > > > > > > > > geguileo(a)redhat.com> > > > > > > > > > > > ha scritto: > > > > > > > > > > > > > > > > > > > > > > > On 14/10, Ignazio Cassano wrote: > > > > > > > > > > > > > Hello, thank you for the answer. > > > > > > > > > > > > > I am using os-brick 2.3.8 but I got same issues on > > stein > > > > with > > > > > > > > > > os.brick > > > > > > > > > > > > 2.8 > > > > > > > > > > > > > For explain better the situation I send you the > > output of > > > > > > > > multipath > > > > > > > > > > -ll > > > > > > > > > > > > on > > > > > > > > > > > > > a compute node: > > > > > > > > > > > > > root@podvc-kvm01 ansible]# multipath -ll > > > > > > > > > > > > > Oct 14 18:50:01 | sdbg: alua not supported > > > > > > > > > > > > > Oct 14 18:50:01 | sdbe: alua not supported > > > > > > > > > > > > > Oct 14 18:50:01 | sdbd: alua not supported > > > > > > > > > > > > > Oct 14 18:50:01 | sdbf: alua not supported > > > > > > > > > > > > > 360060160f0d049007ab7275f743d0286 dm-11 DGC > > ,VRAID > > > > > > > > > > > > > size=30G features='1 retain_attached_hw_handler' > > > > hwhandler='1 > > > > > > > > alua' > > > > > > > > > > wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=0 status=enabled > > > > > > > > > > > > > | |- 15:0:0:71 sdbg 67:160 failed faulty running > > > > > > > > > > > > > | `- 12:0:0:71 sdbe 67:128 failed faulty running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=0 status=enabled > > > > > > > > > > > > > |- 11:0:0:71 sdbd 67:112 failed faulty running > > > > > > > > > > > > > `- 13:0:0:71 sdbf 67:144 failed faulty running > > > > > > > > > > > > > 360060160f0d049004cdb615f52343fdb dm-8 DGC ,VRAID > > > > > > > > > > > > > size=80G features='2 queue_if_no_path > > > > > > retain_attached_hw_handler' > > > > > > > > > > > > > hwhandler='1 alua' wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=50 status=active > > > > > > > > > > > > > | |- 15:0:0:210 sdau 66:224 active ready running > > > > > > > > > > > > > | `- 12:0:0:210 sdas 66:192 active ready running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=10 status=enabled > > > > > > > > > > > > > |- 11:0:0:210 sdar 66:176 active ready running > > > > > > > > > > > > > `- 13:0:0:210 sdat 66:208 active ready running > > > > > > > > > > > > > 360060160f0d0490034aa645fe52265eb dm-12 DGC > > ,VRAID > > > > > > > > > > > > > size=100G features='2 queue_if_no_path > > > > > > > > retain_attached_hw_handler' > > > > > > > > > > > > > hwhandler='1 alua' wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=50 status=active > > > > > > > > > > > > > | |- 12:0:0:177 sdbi 67:192 active ready running > > > > > > > > > > > > > | `- 15:0:0:177 sdbk 67:224 active ready running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=10 status=enabled > > > > > > > > > > > > > |- 11:0:0:177 sdbh 67:176 active ready running > > > > > > > > > > > > > `- 13:0:0:177 sdbj 67:208 active ready running > > > > > > > > > > > > > 360060160f0d04900159f225fd6126db9 dm-6 DGC ,VRAID > > > > > > > > > > > > > size=40G features='2 queue_if_no_path > > > > > > retain_attached_hw_handler' > > > > > > > > > > > > > hwhandler='1 alua' wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=50 status=active > > > > > > > > > > > > > | |- 11:0:0:26 sdaf 65:240 active ready running > > > > > > > > > > > > > | `- 13:0:0:26 sdah 66:16 active ready running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=10 status=enabled > > > > > > > > > > > > > |- 12:0:0:26 sdag 66:0 active ready running > > > > > > > > > > > > > `- 15:0:0:26 sdai 66:32 active ready running > > > > > > > > > > > > > Oct 14 18:50:01 | sdba: alua not supported > > > > > > > > > > > > > Oct 14 18:50:01 | sdbc: alua not supported > > > > > > > > > > > > > Oct 14 18:50:01 | sdaz: alua not supported > > > > > > > > > > > > > Oct 14 18:50:01 | sdbb: alua not supported > > > > > > > > > > > > > 360060160f0d049007eb7275f93937511 dm-10 DGC > > ,VRAID > > > > > > > > > > > > > size=40G features='1 retain_attached_hw_handler' > > > > hwhandler='1 > > > > > > > > alua' > > > > > > > > > > wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=0 status=enabled > > > > > > > > > > > > > | |- 12:0:0:242 sdba 67:64 failed faulty running > > > > > > > > > > > > > | `- 15:0:0:242 sdbc 67:96 failed faulty running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=0 status=enabled > > > > > > > > > > > > > |- 11:0:0:242 sdaz 67:48 failed faulty running > > > > > > > > > > > > > `- 13:0:0:242 sdbb 67:80 failed faulty running > > > > > > > > > > > > > 360060160f0d049003a567c5fb72201e8 dm-7 DGC ,VRAID > > > > > > > > > > > > > size=40G features='2 queue_if_no_path > > > > > > retain_attached_hw_handler' > > > > > > > > > > > > > hwhandler='1 alua' wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=50 status=active > > > > > > > > > > > > > | |- 12:0:0:57 sdbq 68:64 active ready running > > > > > > > > > > > > > | `- 15:0:0:57 sdbs 68:96 active ready running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=10 status=enabled > > > > > > > > > > > > > |- 11:0:0:57 sdbp 68:48 active ready running > > > > > > > > > > > > > `- 13:0:0:57 sdbr 68:80 active ready running > > > > > > > > > > > > > 360060160f0d04900c120625f802ea1fa dm-9 DGC ,VRAID > > > > > > > > > > > > > size=25G features='2 queue_if_no_path > > > > > > retain_attached_hw_handler' > > > > > > > > > > > > > hwhandler='1 alua' wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=50 status=active > > > > > > > > > > > > > | |- 11:0:0:234 sdav 66:240 active ready running > > > > > > > > > > > > > | `- 13:0:0:234 sdax 67:16 active ready running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=10 status=enabled > > > > > > > > > > > > > |- 15:0:0:234 sday 67:32 active ready running > > > > > > > > > > > > > `- 12:0:0:234 sdaw 67:0 active ready running > > > > > > > > > > > > > 360060160f0d04900b8b0615fb14ef1bd dm-3 DGC ,VRAID > > > > > > > > > > > > > size=50G features='2 queue_if_no_path > > > > > > retain_attached_hw_handler' > > > > > > > > > > > > > hwhandler='1 alua' wp=rw > > > > > > > > > > > > > |-+- policy='round-robin 0' prio=50 status=active > > > > > > > > > > > > > | |- 11:0:0:11 sdan 66:112 active ready running > > > > > > > > > > > > > | `- 13:0:0:11 sdap 66:144 active ready running > > > > > > > > > > > > > `-+- policy='round-robin 0' prio=10 status=enabled > > > > > > > > > > > > > |- 12:0:0:11 sdao 66:128 active ready running > > > > > > > > > > > > > `- 15:0:0:11 sdaq 66:160 active ready running > > > > > > > > > > > > > > > > > > > > > > > > > > The active running are related to running virtual > > > > machines. > > > > > > > > > > > > > The faulty are related to virtual macnines migrated > > on > > > > other > > > > > > kvm > > > > > > > > > > nodes. > > > > > > > > > > > > > Every volume has 4 path because iscsi on unity needs > > two > > > > > > > > different > > > > > > > > > > vlans, > > > > > > > > > > > > > each one with 2 addresses. > > > > > > > > > > > > > I think this issue can be related to os-brick because > > > > when I > > > > > > > > migrate > > > > > > > > > > a > > > > > > > > > > > > > virtual machine from host A host B in the cova > > compute > > > > log on > > > > > > > > host A > > > > > > > > > > I > > > > > > > > > > > > read: > > > > > > > > > > > > > 2020-10-13 10:31:02.769 118727 DEBUG > > > > > > > > > > os_brick.initiator.connectors.iscsi > > > > > > > > > > > > > [req-771ede8c-6e1b-4f3f-ad4a-1f6ed820a55c > > > > > > > > > > > > 66adb965bef64eaaab2af93ade87e2ca > > > > > > > > > > > > > 85cace94dcc7484c85ff9337eb1d0c4c - default default] > > > > > > > > *Disconnecting > > > > > > > > > > from: > > > > > > > > > > > > []* > > > > > > > > > > > > > > > > > > > > > > > > > > Ignazio > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > That's definitely the right clue!! Though I don't > > fully > > > > agree > > > > > > with > > > > > > > > > > this > > > > > > > > > > > > being an os-brick issue just yet. ;-) > > > > > > > > > > > > > > > > > > > > > > > > Like I mentioned before, RCA is usually non-trivial, > > and > > > > > > > > explaining how > > > > > > > > > > > > to debug these issues over email is close to > > impossible, > > > > but if > > > > > > > > this > > > > > > > > > > > > were my system, and assuming you have tested normal > > > > > > attach/detach > > > > > > > > > > > > procedure and is working fine, this is what I would do: > > > > > > > > > > > > > > > > > > > > > > > > - Enable DEBUG logs on Nova compute node (I believe you > > > > already > > > > > > > > have) > > > > > > > > > > > > - Attach a new device to an instance on that node with > > > > --debug > > > > > > to > > > > > > > > get > > > > > > > > > > > > the request id > > > > > > > > > > > > - Get the connection information dictionary that > > os-brick > > > > > > receives > > > > > > > > on > > > > > > > > > > > > the call to connect_volume for that request, and the > > data > > > > > > that > > > > > > > > > > > > os-brick returns to Nova on that method call > > completion. > > > > > > > > > > > > - Check if the returned data to Nova is a multipathed > > > > device or > > > > > > > > not (in > > > > > > > > > > > > 'path'), and whether we have the wwn or not (in > > > > > > 'scsi_wwn'). It > > > > > > > > > > > > should be a multipath device, and then I would check > > its > > > > > > status > > > > > > > > in > > > > > > > > > > the > > > > > > > > > > > > multipath daemon. > > > > > > > > > > > > - Now do the live migration (with --debug to get the > > > > request > > > > > > id) > > > > > > > > and > > > > > > > > > > see > > > > > > > > > > > > what information Nova passes in that request to > > > > os-brick's > > > > > > > > > > > > disconnect_volume. > > > > > > > > > > > > - Is it the same? Then it's likely an os-brick issue, > > > > and I > > > > > > can > > > > > > > > have > > > > > > > > > > a > > > > > > > > > > > > look at the logs if you put the logs for that > > os-brick > > > > > > detach > > > > > > > > > > > > process in a pastebin [1]. > > > > > > > > > > > > - Is it different? Then it's either a Nova bug or a > > > > Cinder > > > > > > driver > > > > > > > > > > > > specific bug. > > > > > > > > > > > > - Is there a call from Nova to Cinder, in the > > migration > > > > > > > > request, > > > > > > > > > > for > > > > > > > > > > > > that same volume to initialize_connection > > passing the > > > > > > source > > > > > > > > host > > > > > > > > > > > > connector info (info from the host that is > > currently > > > > > > > > attached)? > > > > > > > > > > > > If there is a call, check if the returned data is > > > > > > different > > > > > > > > from > > > > > > > > > > > > the one we used to do the attach, if that's the > > case > > > > then > > > > > > > > it's a > > > > > > > > > > > > Nova and Cinder driver bug that was solved on the > > > > Nova > > > > > > side > > > > > > > > in > > > > > > > > > > > > 17.0.10 [2]. > > > > > > > > > > > > - If there's no call to Cinder's > > > > initialize_connection, the > > > > > > > > it's > > > > > > > > > > > > most likely a Nova bug. Try to find out if this > > > > > > connection > > > > > > > > info > > > > > > > > > > > > makes any sense for that host (LUN, target, > > etc.) or > > > > if > > > > > > this > > > > > > > > is > > > > > > > > > > > > the one from the destination volume. > > > > > > > > > > > > > > > > > > > > > > > > I hope this somehow helps. > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Gorka. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1]: http://paste.openstack.org/ > > > > > > > > > > > > [2]: https://review.opendev.org/#/c/637827/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Il giorno mer 14 ott 2020 alle ore 13:41 Gorka > > Eguileor < > > > > > > > > > > > > geguileo(a)redhat.com> > > > > > > > > > > > > > ha scritto: > > > > > > > > > > > > > > > > > > > > > > > > > > > On 09/10, Ignazio Cassano wrote: > > > > > > > > > > > > > > > Hello Stackers, I am using dell emc iscsi driver > > on > > > > my > > > > > > > > centos 7 > > > > > > > > > > > > queens > > > > > > > > > > > > > > > openstack. It works and instances work as well > > but on > > > > > > compute > > > > > > > > > > nodes I > > > > > > > > > > > > > > got a > > > > > > > > > > > > > > > lot a faulty device reported by multipath il > > comand. > > > > > > > > > > > > > > > I do know why this happens, probably attacching > > and > > > > > > detaching > > > > > > > > > > > > volumes and > > > > > > > > > > > > > > > live migrating instances do not close something > > well. > > > > > > > > > > > > > > > I read this can cause serious performances > > problems > > > > on > > > > > > > > compute > > > > > > > > > > nodes. > > > > > > > > > > > > > > > Please, any workaround and/or patch is suggested > > ? > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > Ignazio > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > There are many, many, many things that could be > > > > happening > > > > > > > > there, > > > > > > > > > > and > > > > > > > > > > > > > > it's not usually trivial doing the RCA, so the > > > > following > > > > > > > > questions > > > > > > > > > > are > > > > > > > > > > > > > > just me hoping this is something "easy" to find > > out. > > > > > > > > > > > > > > > > > > > > > > > > > > > > What os-brick version from Queens are you running? > > > > Latest > > > > > > > > > > (2.3.9), or > > > > > > > > > > > > > > maybe one older than 2.3.3? > > > > > > > > > > > > > > > > > > > > > > > > > > > > When you say you have faulty devices reported, are > > > > these > > > > > > faulty > > > > > > > > > > devices > > > > > > > > > > > > > > alone in the multipath DM? Or do you have some > > faulty > > > > ones > > > > > > with > > > > > > > > > > some > > > > > > > > > > > > > > that are ok? > > > > > > > > > > > > > > > > > > > > > > > > > > > > If there are some OK and some that aren't, are they > > > > > > consecutive > > > > > > > > > > > > devices? > > > > > > > > > > > > > > (as in /dev/sda /dev/sdb etc). > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > Gorka. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >

2 2

Re: [kolla] Setting up eth1 on VM for AIO
by Braden, Albert 30 Oct '20

30 Oct '20

From: Mark Goddard <mark(a)stackhpc.com> Sent: Friday, October 30, 2020 4:37 AM To: Radosław Piliszek <radoslaw.piliszek(a)gmail.com> Cc: Braden, Albert <C-Albert.Braden(a)charter.com>; OpenStack Discuss <openstack-discuss(a)lists.openstack.org> Subject: Re: [kolla] Setting up eth1 on VM for AIO On Thu, 29 Oct 2020 at 16:20, Radosław Piliszek <radoslaw.piliszek(a)gmail.com<mailto:radoslaw.piliszek@gmail.com>> wrote: Hi Albert! Sorry for top posting but I don't really see which part to quote would be best so they all lose. :D Indeed, it's tricky to handle this if you are not a fan of linux networking. I guess we should document an example way.... Anyhow, the way e.g. Kayobe handles it for users (as it deals with baremetal details) is to use bridges and veths. Basically, you create a bridge, attach it to the expected interface, set any networking you still want on that network for this host on this bridge. Then, you create a veth pair and connect one end to it and the second one is given to neutron. >Alternatively, if this is just for development, you can use a dummy interface. > >ip link add link eth1 type dummy > >It's not persistent, but good enough for testing. If you need external network access from it you could add a MASQUERADE iptables rule and enable the IP >forwarding sysctl. That seems to be working. After googling a bit I was able to make it persistent: [root@adjutant-poc ~]# cat /etc/modules-load.d/dummy.conf dummy [root@adjutant-poc ~]# cat /etc/modprobe.d/dummy.conf install dummy /sbin/modprobe --ignore-install dummy; /sbin/ip link set name eth1 dev dummy0 [root@adjutant-poc ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 NAME=eth1 BOOTPROTO=none DEVICE=eth1 ONBOOT=yes TYPE=Ethernet NM_CONTROLLED=no [root@adjutant-poc ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8950 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:22:c4:e2 brd ff:ff:ff:ff:ff:ff inet 192.168.1.83/22 brd 192.168.3.255 scope global dynamic eth0 valid_lft 85821sec preferred_lft 85821sec inet6 fe80::f816:3eff:fe22:c4e2/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 7e:e1:fe:31:c9:2e brd ff:ff:ff:ff:ff:ff inet6 fe80::7ce1:feff:fe31:c92e/64 scope link valid_lft forever preferred_lft forever -yoctozepto On Thu, Oct 29, 2020 at 12:49 PM Braden, Albert <C-Albert.Braden(a)charter.com<mailto:C-Albert.Braden@charter.com>> wrote: > > I’m trying to setup an allinone kolla-ansible Train POC on a VM, and I’m following the instructions here: > > > > https://docs.openstack.org/kolla-ansible/train/user/quickstart.html > > > > Under “Prepare initial configuration” where it talks about globals.yml, it says I need 2 network interfaces: > > > > network_interface: "eth0" > > neutron_external_interface: "eth1" > > > > My VM has a private IP and a floating IP that I use to ssh to it: > > > > | ID | Name | Status | Networks | Image | Flavor | > > | 41f6782a-8ca8-4ab4-8fdc-776584139ba0 | adjutant-poc | ACTIVE | private=<primary IP>, <floating IP> | | s1-standard-08 | > > > > I tried various options for eth1 but I can’t seem to get it to work: > > > > [root@adjutant-poc network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 > > BOOTPROTO=dhcp > > DEVICE=eth1 > > ONBOOT=yes > > TYPE=Ethernet > > NM_CONTROLLED=no > > > > I tried creating another port and adding it but that didn’t seem to make a difference. What am I doing wrong? Do I really need 2 network interfaces on the VM? If so, what’s the correct way to add one? > > > > [root@adjutant-poc network-scripts]# service network restart > > Restarting network (via systemctl): Job for network.service failed because the control process exited with error code. See "systemctl status network.service" and "journalctl -xe" for details. > > [FAILED] > > > > Log: > > > > http://paste.debian.net/1168982/ > > > > I apologize for the nonsense below. So far I have not been able to stop it from being attached to my external emails. I'm working on it. > > > > The contents of this e-mail message and > any attachments are intended solely for the > addressee(s) and may contain confidential > and/or legally privileged information. If you > are not the intended recipient of this message > or if this message has been addressed to you > in error, please immediately alert the sender > by reply e-mail and then delete this message > and any attachments. If you are not the > intended recipient, you are notified that > any use, dissemination, distribution, copying, > or storage of this message or any attachment > is strictly prohibited. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

1 0

[ironic] SPUC winter season
by Dmitry Tantsur 30 Oct '20

30 Oct '20

Boo! It's SPUC time again! Ironicers, their pet AIs, friendly stackers, as well as significant others and fluffy overladies and overlords, are welcome! For those who don't know, Sanity Preserving Un-Conference is a series of Friday calls for keeping (the remainings) of our sanity in these difficult times. It's goal is to let people virtually socialize and help brighten up lives for each other. Work achievements? A new hobby? A hilarious story? Funny things your kid did at 4am? The first words from your pizza dough that you forgot in the fridge during the previous lockdown? Anything that makes you and others happier is on-topic! Due to an increasing number of APAC contributors, I would like two pick slots: one in APAC evening/Europe morning, the other in Europe morning/US afternoon. Please vote in the doodle: https://doodle.com/poll/tw72utagidz2nac3 Cheers, Dmitry -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

4 6

[openstack][nova] urgent compute in evacuate state
by Ignazio Cassano 30 Oct '20

30 Oct '20

Hello Guys, I have a node in evacuate state....how exit from this state ? Thanks Ignazio

2 6

[kolla] Setting up eth1 on VM for AIO
by Braden, Albert 30 Oct '20

30 Oct '20

I'm trying to setup an allinone kolla-ansible Train POC on a VM, and I'm following the instructions here: https://docs.openstack.org/kolla-ansible/train/user/quickstart.html Under "Prepare initial configuration" where it talks about globals.yml, it says I need 2 network interfaces: network_interface: "eth0" neutron_external_interface: "eth1" My VM has a private IP and a floating IP that I use to ssh to it: | ID | Name | Status | Networks | Image | Flavor | | 41f6782a-8ca8-4ab4-8fdc-776584139ba0 | adjutant-poc | ACTIVE | private=<primary IP>, <floating IP> | | s1-standard-08 | I tried various options for eth1 but I can't seem to get it to work: [root@adjutant-poc network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 BOOTPROTO=dhcp DEVICE=eth1 ONBOOT=yes TYPE=Ethernet NM_CONTROLLED=no I tried creating another port and adding it but that didn't seem to make a difference. What am I doing wrong? Do I really need 2 network interfaces on the VM? If so, what's the correct way to add one? [root@adjutant-poc network-scripts]# service network restart Restarting network (via systemctl): Job for network.service failed because the control process exited with error code. See "systemctl status network.service" and "journalctl -xe" for details. [FAILED] Log: http://paste.debian.net/1168982/ I apologize for the nonsense below. So far I have not been able to stop it from being attached to my external emails. I'm working on it. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

3 2

[Openstack][cinder] dell unity iscsi faulty devices
by Ignazio Cassano 30 Oct '20

30 Oct '20

Hello Stackers, I am using dell emc iscsi driver on my centos 7 queens openstack. It works and instances work as well but on compute nodes I got a lot a faulty device reported by multipath il comand. I do know why this happens, probably attacching and detaching volumes and live migrating instances do not close something well. I read this can cause serious performances problems on compute nodes. Please, any workaround and/or patch is suggested ? Regards Ignazio

2 18

cannot ping my guest's floating ip
by R Kendal 29 Oct '20

29 Oct '20

I am struggling with the fact that I cannot ping my guest's floating ip. I have one Controller node and one Compute node brought up with devstack. Any help would be appreciated! Kendal Details follow... Controller node... [1 picasso@controller ~]$ openstack server list +--------------------------------------+----------+--------+-------------------------------+----------------+----------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+----------+--------+-------------------------------+----------------+----------+ | c5a7fd0a-61a1-4698-9c12-45b44cb20e54 | myserver | ACTIVE | private=10.0.1.12, 10.0.0.196 | picasso-server | m1.small | +--------------------------------------+----------+--------+-------------------------------+----------------+----------+ [1 picasso@controller ~]$ openstack floating ip list +--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+ | ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project | +--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+ | 055d825f-9398-4a11-bf5e-ff3d1a9fc8be | 10.0.0.196 | 10.0.1.12 | 9d036770-e1ba-4d5f-8f16-780a066416fd | 32631db5-7798-47df-8af6-7ee8fca011e4 | a0e078d4bac34db89105a2eb0ba4d896 | +--------------------------------------+---------------------+------------------+--------------------------------------+--------------------------------------+----------------------------------+ [0 picasso@controller ~]$ ping -c 1 -W 1 10.0.1.1 PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data. 64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.066 ms --- 10.0.1.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.066/0.066/0.066/0.000 ms [0 picasso@controller ~]$ ping -c 1 -W 1 10.0.1.12 PING 10.0.1.12 (10.0.1.12) 56(84) bytes of data. --- 10.0.1.12 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [1 picasso@controller ~]$ ping -c 1 -W 1 10.0.0.196 PING 10.0.0.196 (10.0.0.196) 56(84) bytes of data. --- 10.0.0.196 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [0 picasso@controller ~]$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000 link/ether 08:00:27:72:b2:e5 brd ff:ff:ff:ff:ff:ff inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3 valid_lft 77664sec preferred_lft 77664sec inet6 fe80::a00:27ff:fe72:b2e5/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000 link/ether 02:00:68:ed:7c:08 brd ff:ff:ff:ff:ff:ff inet 192.168.1.6/24 brd 192.168.1.255 scope global enp0s8 valid_lft forever preferred_lft forever inet6 fe80::68ff:feed:7c08/64 scope link valid_lft forever preferred_lft forever 4: enp0s9: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UNKNOWN group default qlen 1000 link/ether 02:00:b6:d5:fe:c5 brd ff:ff:ff:ff:ff:ff inet6 fe80::b6ff:fed5:fec5/64 scope link valid_lft forever preferred_lft forever 5: enp0s10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 02:00:72:74:73:d0 brd ff:ff:ff:ff:ff:ff inet 10.0.1.1/24 brd 10.0.1.255 scope global enp0s10 valid_lft forever preferred_lft forever inet6 fe80::72ff:fe74:73d0/64 scope link valid_lft forever preferred_lft forever 6: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether e6:fd:7a:97:a1:32 brd ff:ff:ff:ff:ff:ff 7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether f6:38:eb:d2:ce:41 brd ff:ff:ff:ff:ff:ff 8: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether ba:d2:e4:c1:7f:48 brd ff:ff:ff:ff:ff:ff inet 10.0.0.8/24 brd 10.0.0.255 scope global br-ex valid_lft forever preferred_lft forever inet6 fe80::b8d2:e4ff:fec1:7f48/64 scope link valid_lft forever preferred_lft forever 9: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether ca:5b:f2:bd:65:40 brd ff:ff:ff:ff:ff:ff 13: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000 link/ether ca:7f:ae:3b:5a:14 brd ff:ff:ff:ff:ff:ff inet6 fe80::c87f:aeff:fe3b:5a14/64 scope link valid_lft forever preferred_lft forever [0 picasso@controller ~]$ ip r default via 10.0.2.2 dev enp0s3 proto dhcp src 10.0.2.15 metric 100 10.0.0.0/24 dev br-ex proto kernel scope link src 10.0.0.8 10.0.1.0/24 dev enp0s10 proto kernel scope link src 10.0.1.1 10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15 10.0.2.2 dev enp0s3 proto dhcp scope link src 10.0.2.15 metric 100 192.168.1.0/24 dev enp0s8 proto kernel scope link src 192.168.1.6 [0 picasso@controller ~]$ neutron net-show "public" +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | nova | | created_at | 2020-10-29T18:21:07Z | | description | | | id | 32631db5-7798-47df-8af6-7ee8fca011e4 | | ipv4_address_scope | | | ipv6_address_scope | | | is_default | False | | mtu | 1500 | | name | public | | port_security_enabled | True | | project_id | a0e078d4bac34db89105a2eb0ba4d896 | | provider:network_type | flat | | provider:physical_network | public | | provider:segmentation_id | | | revision_number | 2 | | router:external | True | | shared | True | | status | ACTIVE | | subnets | daab0afa-9adc-4978-9e00-65669b89fec5 | | tags | | | tenant_id | a0e078d4bac34db89105a2eb0ba4d896 | | updated_at | 2020-10-29T18:21:14Z | +---------------------------+--------------------------------------+ [0 picasso@controller ~]$ neutron subnet-show public-subnet +-------------------+----------------------------------------------+ | Field | Value | +-------------------+----------------------------------------------+ | allocation_pools | {"start": "10.0.0.128", "end": "10.0.0.254"} | | cidr | 10.0.0.0/24 | | created_at | 2020-10-29T18:21:14Z | | description | | | dns_nameservers | 10.0.0.1 | | enable_dhcp | False | | gateway_ip | 10.0.0.1 | | host_routes | | | id | daab0afa-9adc-4978-9e00-65669b89fec5 | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | public-subnet | | network_id | 32631db5-7798-47df-8af6-7ee8fca011e4 | | project_id | a0e078d4bac34db89105a2eb0ba4d896 | | revision_number | 0 | | service_types | | | subnetpool_id | | | tags | | | tenant_id | a0e078d4bac34db89105a2eb0ba4d896 | | updated_at | 2020-10-29T18:21:14Z | +-------------------+----------------------------------------------+ [1 picasso@controller ~]$ neutron net-show private +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | nova | | created_at | 2020-10-29T18:20:56Z | | description | | | id | b440a536-a69c-42c8-b05d-5f8323f22cc0 | | ipv4_address_scope | | | ipv6_address_scope | | | mtu | 1450 | | name | private | | port_security_enabled | True | | project_id | 682549fcfe3b4645a21b59290e62e453 | | provider:network_type | vxlan | | provider:physical_network | | | provider:segmentation_id | 1 | | revision_number | 2 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | 35a88fb9-e1e6-4195-9c6b-95d96545d4c4 | | tags | | | tenant_id | 682549fcfe3b4645a21b59290e62e453 | | updated_at | 2020-10-29T18:21:02Z | +---------------------------+--------------------------------------+ [0 picasso@controller ~]$ neutron subnet-show private-subnet +-------------------+---------------------------------------------+ | Field | Value | +-------------------+---------------------------------------------+ | allocation_pools | {"start": "10.0.1.10", "end": "10.0.1.254"} | | cidr | 10.0.1.0/24 | | created_at | 2020-10-29T18:21:02Z | | description | | | dns_nameservers | 10.0.0.1 | | enable_dhcp | True | | gateway_ip | 10.0.1.1 | | host_routes | | | id | 35a88fb9-e1e6-4195-9c6b-95d96545d4c4 | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | private-subnet | | network_id | b440a536-a69c-42c8-b05d-5f8323f22cc0 | | project_id | 682549fcfe3b4645a21b59290e62e453 | | revision_number | 0 | | service_types | | | subnetpool_id | | | tags | | | tenant_id | 682549fcfe3b4645a21b59290e62e453 | | updated_at | 2020-10-29T18:21:02Z | +-------------------+---------------------------------------------+ [0 picasso@controller ~]$ neutron router-show public +-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | nova | | created_at | 2020-10-29T18:21:20Z | | description | | | distributed | False | | external_gateway_info | {"network_id": "32631db5-7798-47df-8af6-7ee8fca011e4", "external_fixed_ips": [{"subnet_id": "daab0afa-9adc-4978-9e00-65669b89fec5", "ip_address": "10.0.0.169"}], "enable_snat": true} | | flavor_id | | | ha | False | | id | d424d63e-cc93-4d02-afad-5851bc7f96cd | | name | public | | project_id | a0e078d4bac34db89105a2eb0ba4d896 | | revision_number | 4 | | routes | | | status | ACTIVE | | tags | | | tenant_id | a0e078d4bac34db89105a2eb0ba4d896 | | updated_at | 2020-10-29T18:21:28Z | +-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [1 picasso@controller ~]$ sudo ovs-vsctl show f2c58de6-9a33-4222-94f5-da1e4a2cfdbb Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port br-tun Interface br-tun type: internal Port "vxlan-c0a801c8" Interface "vxlan-c0a801c8" type: vxlan options: {df_default="true", egress_pkt_mark="0", in_key=flow, local_ip="192.168.1.6", out_key=flow, remote_ip="192.168.1.200"} Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-ex Interface br-ex type: internal Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Port "enp0s9" Interface "enp0s9" Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port "tap456085c0-61" tag: 1 Interface "tap456085c0-61" type: internal Port "qr-0434594d-ee" tag: 1 Interface "qr-0434594d-ee" type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} Port br-int Interface br-int type: internal Port "qg-77ed5bcd-25" tag: 2 Interface "qg-77ed5bcd-25" type: internal ovs_version: "2.12.1" router_name=public ROUTER_ID=$(openstack router show $router_name -f value -c id) QROUTER="qrouter-$ROUTER_ID" [0 picasso@controller ~]$ sudo ip netns exec $QROUTER ifconfig lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 45 bytes 4424 (4.4 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 45 bytes 4424 (4.4 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 qg-77ed5bcd-25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.0.169 netmask 255.255.255.0 broadcast 10.0.0.255 inet6 fe80::f816:3eff:febc:990c prefixlen 64 scopeid 0x20<link> ether fa:16:3e:bc:99:0c txqueuelen 1000 (Ethernet) RX packets 944 bytes 146787 (146.7 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 164 bytes 16468 (16.4 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 qr-0434594d-ee: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450 inet 10.0.1.1 netmask 255.255.255.0 broadcast 10.0.1.255 inet6 fe80::f816:3eff:fe5c:9a7d prefixlen 64 scopeid 0x20<link> ether fa:16:3e:5c:9a:7d txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 169 bytes 7550 (7.5 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [0 picasso@controller ~]$ sudo ip netns exec $QROUTER ping -c 1 -W 1 10.0.1.1 PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data. 64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.034 ms --- 10.0.1.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.034/0.034/0.034/0.000 ms [0 picasso@controller ~]$ sudo ip netns exec $QROUTER ping -c 1 -W 1 10.0.1.196 PING 10.0.1.196 (10.0.1.196) 56(84) bytes of data. --- 10.0.1.196 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [1 picasso@controller ~]$ sudo ip netns exec $QROUTER ping -c 1 -W 1 10.0.1.169 PING 10.0.1.169 (10.0.1.169) 56(84) bytes of data. --- 10.0.1.169 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms ---------- Compute node... [1 picasso@compute ~]$ ping -c 1 -W 1 10.0.1.1 PING 10.0.1.1 (10.0.1.1) 56(84) bytes of data. 64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.425 ms --- 10.0.1.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.425/0.425/0.425/0.000 ms [0 picasso@compute ~]$ ping -c 1 -W 1 10.0.1.12 PING 10.0.1.12 (10.0.1.12) 56(84) bytes of data. --- 10.0.1.12 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [1 picasso@compute ~]$ ping -c 1 -W 1 10.0.0.196 PING 10.0.0.196 (10.0.0.196) 56(84) bytes of data. --- 10.0.0.196 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms [0 picasso@compute ~]$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000 link/ether 08:00:27:72:b2:e5 brd ff:ff:ff:ff:ff:ff inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3 valid_lft 79425sec preferred_lft 79425sec inet6 fe80::a00:27ff:fe72:b2e5/64 scope link valid_lft forever preferred_lft forever 3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000 link/ether 02:00:73:9d:41:7d brd ff:ff:ff:ff:ff:ff inet 192.168.1.200/24 brd 192.168.1.255 scope global enp0s8 valid_lft forever preferred_lft forever inet6 fe80::73ff:fe9d:417d/64 scope link valid_lft forever preferred_lft forever 4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 02:00:08:c1:84:db brd ff:ff:ff:ff:ff:ff inet 10.0.1.31/24 brd 10.0.1.255 scope global enp0s9 valid_lft forever preferred_lft forever inet6 fe80::8ff:fec1:84db/64 scope link valid_lft forever preferred_lft forever 5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 52:54:00:ee:20:86 brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 valid_lft forever preferred_lft forever 6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN group default qlen 1000 link/ether 52:54:00:ee:20:86 brd ff:ff:ff:ff:ff:ff 7: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 9e:52:8b:6f:f1:47 brd ff:ff:ff:ff:ff:ff 8: br-int: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000 link/ether 76:e6:90:cc:01:48 brd ff:ff:ff:ff:ff:ff 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether aa:09:9f:d9:1d:4f brd ff:ff:ff:ff:ff:ff inet6 fe80::a809:9fff:fed9:1d4f/64 scope link valid_lft forever preferred_lft forever 10: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 26:76:22:af:06:40 brd ff:ff:ff:ff:ff:ff 11: vxlan_sys_4789: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000 link/ether 8a:87:fb:68:ec:8c brd ff:ff:ff:ff:ff:ff inet6 fe80::8887:fbff:fe68:ec8c/64 scope link valid_lft forever preferred_lft forever 12: tap9d036770-e1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel master ovs-system state UNKNOWN group default qlen 1000 link/ether fe:16:3e:f5:f5:c0 brd ff:ff:ff:ff:ff:ff inet6 fe80::fc16:3eff:fef5:f5c0/64 scope link valid_lft forever preferred_lft forever [0 picasso@compute ~]$ ip r default via 10.0.2.2 dev enp0s3 proto dhcp src 10.0.2.15 metric 100 10.0.1.0/24 dev enp0s9 proto kernel scope link src 10.0.1.31 10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15 10.0.2.2 dev enp0s3 proto dhcp scope link src 10.0.2.15 metric 100 192.168.1.0/24 dev enp0s8 proto kernel scope link src 192.168.1.200 192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown [130 picasso@compute ~]$ sudo ovs-vsctl show a75bdb8e-9266-48f4-bab5-e119cf07c6af Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-tun Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-tun Interface br-tun type: internal Port "vxlan-c0a80106" Interface "vxlan-c0a80106" type: vxlan options: {df_default="true", egress_pkt_mark="0", in_key=flow, local_ip="192.168.1.200", out_key=flow, remote_ip="192.168.1.6"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-int Interface br-int type: internal Port "tap9d036770-e1" tag: 1 Interface "tap9d036770-e1" Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Port br-ex Interface br-ex type: internal ovs_version: "2.12.1"

2 1

Nova missing instance
by Grant Morley 29 Oct '20

29 Oct '20

Hi all, A customer had contacted us stating their instance was not working and was unresponsive. They attempted to reboot it and it failed to reboot and went into an error status. I have had a look at the nova logs and nova seems to think the instance has "disappeared" and comes up with an error stating: "Instance cannot be found" I have attached the full logs in the pastebin link. Has anyone come across this before? https://pastebin.com/LWzvsc8u Regards, -- Grant Morley Cloud Engineer, Civo Ltd Unit H-K, Gateway 1000, Whittle Way Stevenage, Herts, SG1 2FP, UK Visit us at www.civo.com <https://www.civo.com/> Signup for an account now > <https://www.civo.com/signup>

2 1

[tripleo][ci] Docker.io Service Announcement: Pull Rate Limit Enforcement, Changes to Image Retention Policies Begin
by Wesley Hayutin 29 Oct '20

29 Oct '20

Greetings, FYI... docker.io will be unavailable to tripleo jobs on Monday Nov 2 2020. Documentation is available here: https://hackmd.io/YH9xtBmOQbSVqPu7fjgoog Reviews are welcome and appreciated https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merg… Thanks! ------ from docker.io ------ Hello: You are receiving this email because of a policy change to Docker products and services you use. On Monday, November 2, 2020 at 9am Pacific Standard Time, Docker will begin enforcing rate limits on container pulls for Anonymous and Free users. Anonymous (unauthenticated) users will be limited to 100 container image pulls every six hours, and Free (authenticated) users will be limited to 200 container image pulls every six hours, when enforcement is fully implemented. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction, as long as the quantities are not excessive or abusive. In addition, we are pausing enforcement of the changes to our image-retention policies until mid-2021, when we anticipate incorporating them into usage-based pricing. Two months ago, we announced an update to Docker image-retention policies. As originally stated, this change, which was set to take effect on November 1, 2020, would result in the deletion of images for free Docker account users after six months of inactivity. Today’s announcement means Docker will not enforce image expiration on November 1, 2020. For more details, please read the blog post <http://email.notify.docker.com/c/eJxNjsFuhSAURL9Gdxi46EMWLNq89j-ucFVSBCMY49…> summarizing the changes or review the documentation for rate limits <http://email.notify.docker.com/c/eJxNjkEOgyAURE8jOwl8kOqCRRvbe3z8WEhRG8WY3r…> . Details about Docker subscription levels and differentiators are available on the Docker Pricing Page <http://email.notify.docker.com/c/eJxNjcsOgyAURL9GlgTu5VEWLNrY_gfBq5KqGKUh_n…> . Thank you for using Docker. Jean-Laurent de Morlhon VP, Engineering Docker

1 0

Windows client can't connect to internet
by liu 29 Oct '20

29 Oct '20

Dear OpenStack community, I just launch a windows client in my OpenStack platform, but it can't connect to the internet with the float IP. Before that use a ubuntu client I can access the internet. Any ideas about why this happened. Thank you for your help. Liu

2 2

[tripleo][ci] Fwd: Service Announcement: Pull Rate Limit Enforcement, Changes to Image Retention Policies Begin November 2, 2020
by Wes Hayutin 29 Oct '20

29 Oct '20

FYI... Reviews are welcome and appreciated https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merg… Thanks! ---------- Forwarded message --------- From: Docker <noreply(a)notify.docker.com> Date: Wed, Oct 28, 2020 at 10:08 PM Subject: Service Announcement: Pull Rate Limit Enforcement, Changes to Image Retention Policies Begin November 2, 2020 To: <weshayutin(a)gmail.com> [image: Docker] <http://email.notify.docker.com/c/eJxNjUEOgyAQAF-DR7LsosCBQ43tPxChkKo0imn8fe…> Hello: You are receiving this email because of a policy change to Docker products and services you use. On Monday, November 2, 2020 at 9am Pacific Standard Time, Docker will begin enforcing rate limits on container pulls for Anonymous and Free users. Anonymous (unauthenticated) users will be limited to 100 container image pulls every six hours, and Free (authenticated) users will be limited to 200 container image pulls every six hours, when enforcement is fully implemented. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction, as long as the quantities are not excessive or abusive. In addition, we are pausing enforcement of the changes to our image-retention policies until mid-2021, when we anticipate incorporating them into usage-based pricing. Two months ago, we announced an update to Docker image-retention policies. As originally stated, this change, which was set to take effect on November 1, 2020, would result in the deletion of images for free Docker account users after six months of inactivity. Today’s announcement means Docker will not enforce image expiration on November 1, 2020. For more details, please read the blog post <http://email.notify.docker.com/c/eJxNjsFuhSAURL9Gdxi46EMWLNq89j-ucFVSBCMY49…> summarizing the changes or review the documentation for rate limits <http://email.notify.docker.com/c/eJxNjkEOgyAURE8jOwl8kOqCRRvbe3z8WEhRG8WY3r…>. Details about Docker subscription levels and differentiators are available on the Docker Pricing Page <http://email.notify.docker.com/c/eJxNjcsOgyAURL9GlgTu5VEWLNrY_gfBq5KqGKUh_n…>. Thank you for using Docker. Jean-Laurent de Morlhon VP, Engineering Docker © Docker, Inc., 2020 318 Cambridge Ave Palo Alto, CA 94306 Docker Privacy Policy <http://email.notify.docker.com/c/eJxNjssKgzAURL_G7AzxxiS6yKLF9j_y1ND4QG1D_r…>

1 0

Nova iSCSI oddness
by Grant Morley 29 Oct '20

29 Oct '20

Hi all, We are seeing some oddness on a couple of our compute hosts that seems to be related to iSCSI. On a couple of our hosts I am seeing this error in the nova compute logs: 2020-10-27 18:56:14.814 31490 WARNING os_brick.initiator.connectors.iscsi [req-8613ae69-1661-49cf-8bdc-6fec875d01ba - - - - -] Couldn't find iscsi sessions because iscsiadm err: iscsiadm: could not read session targetname: 5 iscsiadm: could not find session info for session1707 That seems to also stop any instance on the compute host from being able to reboot. Reboots seem to get accepted but the instance never completes and gets stuck in the reboot state: 2020-10-27 19:11:58.891 48612 INFO nova.compute.manager [-] [instance: c8079e85-4777-4615-9d5a-3d1151e11984] During sync_power_state the instance has a pending task (reboot_started_hard). Skip. 2020-10-27 19:11:58.891 48612 INFO nova.compute.manager [-] [instance: 31128f26-910d-411f-98e0-c95dd36f4f0f] During sync_power_state the instance has a pending task (reboot_started_hard). Skip. Does anyone know of a way to resolve this without rebooting the entire compute host? I can't see any other issues other than the fact there is this iSCSI error which in turn seems to stop nova from processing anything for any instance. Any advice would be much appreciated. Regards, -- Grant Morley Cloud Engineer, Civo Ltd Unit H-K, Gateway 1000, Whittle Way Stevenage, Herts, SG1 2FP, UK Visit us at www.civo.com <https://www.civo.com/> Signup for an account now > <https://www.civo.com/signup>

4 9

[tripleo] TripleO at Wallaby PTG wrapup
by Marios Andreou 29 Oct '20

29 Oct '20

Hello all, thanks to everyone that participated and especially thanks to all who prepared and lead our PTG sessions. The number of participants was variable however the maximum attendees (that I noticed at least) per day were Monday:43, Tuesday:48, Wednesday:39. We had three full days with 18 sessions in total. I've included a condensed listing below [3] for convenience and you can find the full schedule at [1] with session descriptions. All sessions were recorded - hopefully we will get access to those soon (I don't have any info about that yet). If you have problems accessing any of the etherpads reach out to me I made backups. Our "team photo" from Monday is available at [2] :) Thanks again TripleO for a great PTG! regards, marios [1] https://etherpad.opendev.org/p/tripleo-ptg-wallaby [2] https://i.imgur.com/apqiYhk.png [3] DAY 1: retrospective & community topics Etherpad: https://etherpad.opendev.org/p/tripleo-ptg-retrospective Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-community-process (Deployment) Composable Network v2 (sans Heat) Proposer: Emilien, Harald & Rabi Etherpad link: https://etherpad.opendev.org/p/composable-networks-v2 (Edge) Topic: State of DCN Proposer: James Slagle Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-edge (PIDONE) Updates from PIDONE Proposer: dciabrin/bandini Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-pidone (Ceph) Topic: review the cephadm integration specs https://review.opendev.org/#/c/757644/ and https://review.opendev.org/#/c/723108/ and https://review.opendev.org/759395 Etherpad link: https://etherpad.opendev.org/p/tripleo-ceph Presenter: [fmount, gfidente, fultonj] DAY 2: (Upgrades) Lessons learned upgrading from Queens to Train, then looking forward at Train to Wallaby Etherpad link: https://etherpad.opendev.org/p/tripleo-PTG-wallaby-upgrades Presenter: Jesse (odyssey4me), Lukas (lbezdick), Jose (jfrancoa), Sergii (holser) (CI) Topic: Dependency pipeline, CentOS Stream and CentOS 9 / Fedora 33 Slides: https://filedn.com/lHMSkrniM7gFSkf96rR711z/TripleODependencyPipeline.pdf Presenter: (rlandy) (Deployment) Topic: Drop heat-monolith local package requirement for undercloud Proposer: Alex Schultz (mwhahaha) Slides: https://docs.google.com/presentation/d/13ds5h9foCHEy3duElGs6mZ3mQwKcaggX6Vj… Blueprint: https://blueprints.launchpad.net/tripleo/+spec/switch-to-heat-all-container (Security) Topic: Updates for Security Etherpad Link: https://etherpad.opendev.org/p/tripleo-PTG-wallaby-security Presenter: Ade (alee) (Upgrades) Topic: Update should only be role inclusion, module and action plugins but we can't do the refactor without any upstream development framework or.... some broader coverage CI. Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-refactor-update (CI) Topic: 0/ so long docker.io Etherpad link: https://hackmd.io/YH9xtBmOQbSVqPu7fjgoog?both Presenter: sshnaidm, chkumar, weshay, marios, DAY 3: (Deployment) Topic: Move to Ansible 2.10.x Proposer: Kevin Carter Etherpad link: https://etherpad.opendev.org/p/tripleo-ansible-210 (Upgrades) Topics: Run all roles in parallel during update by default for massive speed improvment. Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-all-roles-in-parallel (Deployment) Topic: config-download Ansible Collection Etherpad link: https://etherpad.opendev.org/p/ptg-tripleo-config-download-collection Proposer: Luke Short (ekultails) (Deployment) Topic: Removal of Swift from the Undercloud Proposer: Kevin Carter (cloudnull) Etherpad link: https://etherpad.opendev.org/p/tripleo-swift-removal-undercloud (Ceph) Integration Discussion - Continued Etherpad link: https://etherpad.opendev.org/p/tripleo-ceph Presenter: [fmount, gfidente, fultonj] (Networking) Topic: BGP Integration https://review.opendev.org/#/c/758249/ "Spec for installing and configuring FRRouter" Proposer: Dan Sneddon (Cloudops) Topic: Moving deployment logic into ansible roles Proposer: efoley, all (discussion)

1 0

[operators][telemetry] Quick Telemetry survey
by Matthias Runge 29 Oct '20

29 Oct '20

Hi there, you may have seen, not much has been happening lately in OpenStack Telemetry. However, there have been interesting talks at the recent OpenInfra Summit. I would like to gather some data on current Telemetry usage in order to be able to focus a bit better[1]. There is not much value in keeping projects alive, if nobody uses them. At the same time, we may want to increase focus/contribution on often used projects. In order to help with the data, please fill in the form[1] and also feel free to forward this form to interested parties. Thank you, Matthias [1] https://forms.gle/EmncAxggRcG8r31b9

3 3

[devstack][cinder failure]:internal server error(HTTP 500)
by James 29 Oct '20

29 Oct '20

Hi:    I am new to openstack installing on one VM. The environment includes Win10,virtualbox,Ubuntu 18.04; The version of  devstack and openstack  is  Train;  the VM  includes 2vCPU,4G ram, 60G disk .    The process was interrupted by cinder error.[internal server error]        The stack.log shows as stack.log; the c-api log shows as c-api log. stack.log +[38;5;242mlib/cinder:create_volume_types:531 (B[m openstack --os-region-name=RegionOne volume type create --property volume_backend_name=lvmdriver-1 lvmdriver-1 2020-10-29 07:57:11.863 | Internal Server Error (HTTP 500) 2020-10-29 07:57:12.182 | +[38;5;242mlib/cinder:create_volume_types:1 (B[m exit_trap 2020-10-29 07:57:12.232 | +[38;5;242m./stack.sh:exit_trap:536 (B[m local r=1 2020-10-29 07:57:12.259 | ++[38;5;242m./stack.sh:exit_trap:537 (B[m jobs -p 2020-10-29 07:57:12.316 | +[38;5;242m./stack.sh:exit_trap:537 (B[m jobs= 2020-10-29 07:57:12.353 | +[38;5;242m./stack.sh:exit_trap:540 (B[m [[ -n '' ]] 2020-10-29 07:57:12.385 | +[38;5;242m./stack.sh:exit_trap:546 (B[m '[' -f /tmp/tmp.8Y7a6hpwLX ']' 2020-10-29 07:57:12.408 | +[38;5;242m./stack.sh:exit_trap:547 (B[m rm /tmp/tmp.8Y7a6hpwLX 2020-10-29 07:57:12.468 | +[38;5;242m./stack.sh:exit_trap:551 (B[m kill_spinner 2020-10-29 07:57:12.499 | +[38;5;242m./stack.sh:kill_spinner:446 (B[m '[' '!' -z '' ']' 2020-10-29 07:57:12.536 | +[38;5;242m./stack.sh:exit_trap:553 (B[m [[ 1 -ne 0 ]] 2020-10-29 07:57:12.581 | +[38;5;242m./stack.sh:exit_trap:554 (B[m echo 'Error on exit' 2020-10-29 07:57:12.581 | Error on exit 2020-10-29 07:57:12.615 | +[38;5;242m./stack.sh:exit_trap:556 (B[m type -p generate-subunit 2020-10-29 07:57:12.635 | +[38;5;242m./stack.sh:exit_trap:557 (B[m generate-subunit 1603957090 1142 fail 2020-10-29 07:57:14.084 | +[38;5;242m./stack.sh:exit_trap:559 (B[m [[ -z /opt/stack/logs ]] 2020-10-29 07:57:14.110 | +[38;5;242m./stack.sh:exit_trap:562 (B[m /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs 2020-10-29 07:57:17.223 | nova-compute: no process found 2020-10-29 07:57:17.264 | neutron-dhcp-agent: no process found 2020-10-29 07:57:17.338 | neutron-l3-agent: no process found 2020-10-29 07:57:17.409 | neutron-linuxbridge-agent: no process found 2020-10-29 07:57:17.471 | neutron-metadata-agent: no process found 2020-10-29 07:57:17.603 | +[38;5;242m./stack.sh:exit_trap:571 C-api logSTRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=12249) _check_effective_sql_mode /usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py:307}} 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault [req-7d67a008-3478-4da0-8b29-259571c1fb94 req-d823c701-e76a-489b-8ec9-fad9f9504c7b admin admin] Caught error: <class 'oslo_db.exception.DBConnectionError'> (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: (Background on this error at: http://sqlalche.me/e/e3q8) DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: (Background on this error at: http://sqlalche.me/e/e3q8) 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault Traceback (most recent call last): 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/middleware/fault.py", line 85, in __call__ 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault return req.get_response(self.application)Thanks for your help,firtly.   发自我的iPhone

1 0

[infra][ci][all] Devstack error
by Slawek Kaplonski 29 Oct '20

29 Oct '20

Hi, Recently I see pretty often devstack based jobs failing with error "ERROR] / opt/stack/devstack/functions-common:230 Failure retrieving default route device". I reported it at [1] as Devstack bug but maybe it's something related to some changes in Ubuntu used on the gate. Did You saw something similar also? Maybe You already know why it's like that and have any ideas about how to fix it? Thx in advance for any info. [1] https://bugs.launchpad.net/devstack/+bug/1902002 -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[heat] propose to move meeting time to Wednesday at 1400 UTC in #heat
by Rico Lin 29 Oct '20

29 Oct '20

Hi all as our current members are changed, I would like to propose we move our meeting time [1] one hour later. >From Weekly on Wednesday at *1300* UTC in #heat to Weekly on Wednesday at *1400* UTC in #heat Which I believe will be easier for US forks to join. [1] http://eavesdrop.openstack.org/#Heat_(Orchestration)_Team_Meeting -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

2 1

[nova] Weekly meeting canceled
by Balázs Gibizer 29 Oct '20

29 Oct '20

Hi, Today's meeting is canceled in favor of the PTG session. See you there. Cheers, gibi

1 0

[Openstack][Queens] iscsi unity driver
by Ignazio Cassano 28 Oct '20

28 Oct '20

Hello, I installed queens on centos7 with cinder unity iscsi driver. When volume are detached for example for a live migration, faulty devices are left on compute nodes. Must I configure iscsi initiator on both compute and controllers nodes? At this time I configured it only on compute nodes. Faulty could be an os brick module issue ? Thanks Ignazio

1 0

[ironic] Reviewing changes post-gerrit admin account compromise
by Julia Kreger 28 Oct '20

28 Oct '20

Greetings folks, Per the opendev notice that was published on the gerrit outage and compromise[0], I believe it is only prudent to sift through the changes published and verify if their content looks okay and that they were published as we expect. As we have a number of repositories, branches, and ultimately change sets to check, I've created a spreadsheet to help us track and coordinate the work[1]. In the interest of transparency, it is publicly viewable, but I've only shared editing ability with the members of ironic-core. Arne, unfortunately I don't see a google account for yourself, if you have an account you want me to add, let me know! Ironic-cores: If someone wishes to help, I personally think that is okay, but if you feel the need to update editing privileges on the document, please just reply to the mailing list so we have it on the record that this is the case. If I missed a branch that needs to be checked, just add it to the list. Same for any project repositories. And same goes for tabs. Copy/paste is likely easiest. Thanks everyone! -Julia [0]: http://lists.opendev.org/pipermail/service-announce/2020-October/000011.html [1]: https://docs.google.com/spreadsheets/d/1-_on0DOpS4gRriAnpdoLf2B7K98RZN7hNAP…

1 1

[policy] Wallaby Virtual Forum & PTG Summary
by Ghanshyam Mann 28 Oct '20

28 Oct '20

Hello Everyone, 'policy' popup team hosted 2 sessions in Wallaby Virtual PTG to discuss the progress and next steps. Forum: ===== https://etherpad.opendev.org/p/consistent-and-secure-default-policies-walla… Progress: * Three projects completed till now - Keystone, Nova and, Cyborg * One in progress - Barbican * Planning in Wallaby - Neutron, Manila We discussed the current and possible future challenges while migrating to the new policy. policy file in JSON format is one know and we talked about a current workaround and long term plan. Deprecation warnings are still an issue and a lot of warnings are logged. This is logged as a bug in nova[1]. Also, the HTML version of the policy doc does not have the deprecation rule and reason. Example: https://docs.openstack.org/nova/latest/configuration/policy.html We need to add it in these docs too. A clear step by step document about how to use system scope in cloud.yaml as well as in general with all migration steps are much needed. We also asked if any deployment is migrated to new policy but there is none yet. PTG: ==== https://etherpad.opendev.org/p/policy-popup-wallaby-ptg We carried the Forum sessions discussion in PTG with few extra topics. Migrate Default Policy Format from JSON to YAML (All projects) -------------------------------------------------------------------------------- We talked about it and decided that it will be great to do this in advance before projects start moving towards the new policy. and having this as a community goal in wallaby will help this effort to move faster. I have proposed this as a goal in TC PTG and it was agreed to select it for wallaby. - Goal proposal: https://review.opendev.org/#/c/759881/2 We also need to update devstack for the neutron policy file which is policy.json. Deprecation warning on-demand based on config option? ----------------------------------------------------------------------- There is no best solution for deprecation warnings when it is a lot in case of new policy work. We cannot stop logging warnings completely. We discussed an option to provide a config option to disable the warnings (enable by default) and only for default value change, not the policy name change. The policy name change is a little critical even in policy overridden case also that is why we should not make it switchable. This way operators can disable warnings after seeing it for the first time and if it is too noisy. New policy in Horizon ---------------------------- It is challenging to adopt new policy in Horizon when some projects have new policies and some not. We left this for now and will continue brainstorming once we do some investigation on the usage of system token for new policy and old one. For now, amotoki proposed a workaround to keep the policy file with the deprecated rule as the overridden rules. - https://review.opendev.org/#/c/750134/ What are the expectations from project teams? ----------------------------------------------------------- In the end, we discussed what all things to be targeted as part of new policy work and what all as a separate effort. Below is the list and I will be documenting it on wiki. Things to do as part of new policy effort: - Migrating JSON to YAML (in advance though) - Policy Tests coverage - System scoped administrator policy support and "reader" role support - upgrade-check support when changing the default policy As separate effort: - Remove hard-coded admin checks in the API/DB/"Views" - Standardizing policy names (depends on projects) [1] https://bugs.launchpad.net/nova/+bug/1900451 -gmann

1 0

[qa] Wallaby Virtual PTG Summary
by Ghanshyam Mann 28 Oct '20

28 Oct '20

Hello Everyone, I am summarizing the discussion QA team had in PTG and action items for quick ref. Victoria Retrospective: ----------------------------- We talked about the retrospective of the last cycle, one main challenge we discussed is the slow review in QA due to fewer contributors. Below are the action items that came up from improving things: * Action: - microversion need more work also when you try more combination or latest (gmann). - Triage bugs in the office hour as 30 min -> update agenda - Speed up the review. Office hour schedule: --------------------------- With daylight ending in Nov, we decided to shift the QA office hour by an hour late which is at 14 UTC. Move back the single horizon test in tempest tree from tempest-horizon: ------------------------------------------------------------------------------------------- Tempest and Horizon team decided to bring back the single horizon test to Tempest tree and retire the tempest-horizon plugins which was too much maintenance for single tests. It is unlikely to happen but if Tempest team in future plan to remove this test then it needs to be consulted with Horizon team first. * Action: - Move test back to Tempest - gmann - Deprecate and retire tempest-horizon - amotoki - Check with the release team if any other release needed. (amotoki + gmann) - modify the 'horizon-dsvm-tempest-plugin' job to direct to tests' new location Add run_validation as a centralized point for Tempest scenario tests: ------------------------------------------------------------------------------------- For now, validation resources automation and skip ssh part via run_validation is done in API test only and this proposal is to extend it to scenario tests also. There is no reason not to do that which will help to run tempest scenario tests on images where ssh is not allowed. There is a most possible situation where complete tests need to be skipped but we will see case by case and at least skipping automatically will help tester to avoid explicitly excluding the scenario tests to the skip list. *Action: - Implement it in scenario tests - paras333 - Refactor scenario manager's create_server method to consume common/compute.py - paras333 Patrole stable release: ---------------------------- We are lacking the maintainer in Patrole project which is a big challenge to release a stable version of it. Sonia and doug (already helping) will try to spend more bandwidth in Patrole. Another thing to reduce the Patrole tests execution time. We discussed a few options like policy engine to return the API based on the flag or use fake driver etc but we did not conclude any option yet as they need more investigation. * Action: - Keep eyes on gate and how tests are running and based on that we can make a stable release for Wallaby. - gmann - Keep exploring the options for making patrole lightweight. - gmann, doug Grenade: extending the testing for zero downtime upgrade testing: ------------------------------------------------------------------------------------ This is just a thought. There is TC tag 'assert:supports-zero-downtime-upgrade' which is not used by any of the projects and also we do not have any testing framework which can verify this tag. We talked about it if we can do such testing in grenade or not. Before moving forward on discussion and investigation we checked if anyone can volunteer for this work. As of now, no volunteer. Use different guest image for gate jobs to run tempest tests: --------------------------------------------------------------------------- We only test cirros guest image to create the test VM and all proposal from Paras is to try more images to enhance the different configuration scenarios in upstream testing. This will help to able to catch more failure/scenario at the upstream gate itself compare to the current situation where most of them are reported from downstream testing. * Action: - Add a job as experimental - paras333 Tempest Cred Manager proposed changes: ----------------------------------------------------- 'primary' and 'alt_promary' credentials in Tempest are hardcoded to non-admin and they are assigned configured 'tempest_role'. There is no way we can assign a different role to both of these creds. The idea here is to make 'primary' and 'alt_promary' credentials configurable so that different deployment can configure these with different roles in various combinations. We will be adding two config option similar to 'tempest_role' and default to an empty list so that we continue the default behavior with what we have currently. So that there is no backward-incompatible change instead it is an additional new thing provided. * Action: - Doug needs to propose the patch. Tempest Scenario manager stable version brainstorming: ----------------------------------------------------------------------- We talked about a few method changes to make the scenario manager stable. If any method is used among plugins only and not in Tempest then we do not actually need to move it to Tempest and they can stay on the side of the plugin itself. We ran out of time here and will continue the brainstorming in office hours or so. (this etherpad - https://etherpad.opendev.org/p/tempest-scenario-manager) * Action: - Sonia and Martin will continue working on this. Wallaby Priority & Planning: ------------------------------------ We did not prioritize the things as such but listed all working items in below etherpad along with Victoria cycle backlogs. - https://etherpad.opendev.org/p/qa-wallaby-ptg Detailed discussions are in this Etherpad: https://etherpad.opendev.org/p/qa-wallaby-ptg -gmann

1 0

[openstack-ansible] Gerrit breach and auditing all commits since Oct 01
by Dmitriy Rabotyagov 28 Oct '20

28 Oct '20

Hi! Most of you already know that Gerrit has been compromised a while ago [1]. Infra team has dealed with it very efficiently and I can assure you that this issue hasn't touched openstack-ansible project. I have looked through all commits that were merged in period from Oct 1 - Oct 21 (and we've got more than 200 of them during this timeframe, wow!), and all of them look valid and were properly approved. I have also double checked our releases and didn't found any extra or malicious ones. So we are pretty safe and can sleep calm [1] http://lists.opendev.org/pipermail/service-announce/2020-October/000011.html -- Kind Regards, Dmitriy Rabotyagov

1 0

[gate] many jobs failing for nova-compute libvirt driver error
by melanie witt 28 Oct '20

28 Oct '20

Hey all, This is just a FYI that we've got another gate failure where nova-compute is failing with the following error raised from the libvirt driver [1]: "TypeError: Parameterized generics cannot be used with class" which was raised from the eventlet.tpool code. (I don't yet understand what this error means exactly.) Because of this, we suspect this is related to the eventlet package version bump from 0.26.1 to 0.28.0 that happened in the upper-constraints change to openstack/requirements that merged earlier today: https://review.opendev.org/#/c/750084/47/upper-constraints.txt@150 But we're not certain yet exactly what is happening and whether this ^ is indeed the cause. I am currently investigating to find the root cause of failure and how to fix it/confirm which package we should pin for now. I am trying out a DNM nova patch to pin eventlet to 0.26.1 to see what happens, while I look into more depth at what the error means and whether it's something we should fix in the nova code. If anyone can help, it would be appreciated. Best, -melanie [1] https://zuul.opendev.org/t/openstack/build/20f1d309663347c28112b711b82b9c03…

4 7

[qa] Wallaby PTG Wrap-up and office hour was changed to Tue 1400 UTC
by Masayuki Igawa 28 Oct '20

28 Oct '20

Hi, Thank you for attending our sessions! We had discussed many things during the virtual PTG. Topics are the following. And as we discussed our office hour office time 1 hour earlier than the current time. So, the office hour starts at 1400 UTC on every Tuesday. ========= Topics: Mon: @ Juno Greetings and ordering the topics Victoria Retrospective Move back the single horizon test in tempest tree from tempest-horizon Add run_validation as a centralised point for Tempest scenario tests Tue: @Juno Patrole stable release Grenade: extending the testing for zero downtime upgrade testing Use different guest image for gate jobs to run tempest tests Tempest Cred Manager proposed changes Tempest Scenario manager stable version brainstorming Bug triaging -> moved to the office hour Wallaby priority items https://etherpad.opendev.org/p/qa-wallaby-priority Etherpad: https://etherpad.opendev.org/p/qa-wallaby-ptg You can find the recording video files on the etherpad. ========= -- Masayuki Igawa

2 1

[Neutron] no L3 meeting this week (2020-June-01)
by LIU Yulong 28 Oct '20

28 Oct '20

Hi there, Due to the PTG meetings this week, we will cancel the L3 meeting this week. See you guys online next week. Regards, LIU Yulong

1 2

[masakari][ptg] I cannot attend the 2nd Masakari PTG session
by Radosław Piliszek 28 Oct '20

28 Oct '20

Dears, I regret to inform that, due to personal matters (augmented by the pandemic and the current situation in my country), I will not be able to attend the 2nd Masakari PTG session: Thursday October 29, 2020 13:00 - 15:00 (UTC) The good news is we don't have that many topics to go through and I am generally able to get more time from later slots if we need to have it. -yoctozepto

1 0

[nova-scheduler] always the same compute was selected?
by Tony Liu 27 Oct '20

27 Oct '20

Hi, I have a Ussuri setup with 5 compute nodes (same resource spec). When launch 5 Cirros VMs in a batch, they are all launched on the same compute node. All 5 nodes have sufficient resource. AWIK, Nova scheduler picks a compute randomly after going through filtering. But 5 VMs all on the same compute is quite coincidental. Could it be something missing? Is there any control to such random pick? Any way to make a better balancing? >From the below log, could anyone let me know how compute-4 was selected? I tried a few more launches, it's always compute-4 was selected. ========================================= 2020-10-27 13:56:45.484 24 DEBUG nova.scheduler.filter_scheduler [req-e2c6bf4b-7512-4ff0-9899-2a24bffa9131 113ee63a9ed0466794e24d069efc302c 32be191b3ec9497eaec108a22220890b - default default] Filtered [(compute-4, compute-4) ram: 131698MB disk: 14251008MB io_ops: 0 instances: 17, (compute-5, compute-5) ram: 232050MB disk: 14251008MB io_ops: 0 instances: 2, (compute-2, compute-2) ram: 367785MB disk: 14252032MB io_ops: 0 instances: 5, (compute-1, compute-1) ram: 298665MB disk: 14251008MB io_ops: 0 instances: 9, (compute-3, compute-3) ram: 259753MB disk: 14251008MB io_ops: 0 instances: 19] _get_sorted_hosts /usr/lib/python3.6/site-packages/nova/scheduler/filter_scheduler.py:443 2020-10-27 13:56:45.485 24 DEBUG nova.scheduler.filter_scheduler [req-e2c6bf4b-7512-4ff0-9899-2a24bffa9131 113ee63a9ed0466794e24d069efc302c 32be191b3ec9497eaec108a22220890b - default default] Weighed [WeighedHost [host: (compute-4, compute-4) ram: 131698MB disk: 14251008MB io_ops: 0 instances: 17, weight: 2.315526126088031], WeighedHost [host: (compute-2, compute-2) ram: 367785MB disk: 14252032MB io_ops: 0 instances: 5, weight: -999997.0], WeighedHost [host: (compute-1, compute-1) ram: 298665MB disk: 14251008MB io_ops: 0 instances: 9, weight: -999997.2171186721], WeighedHost [host: (compute-3, compute-3) ram: 259753MB disk: 14251008MB io_ops: 0 instances: 19, weight: -999997.3362949106], WeighedHost [host: (compute-5, compute-5) ram: 232050MB disk: 14251008MB io_ops: 0 instances: 2, weight: -999997.371492924]] _get_sorted_hosts /usr/lib/python3.6/site-packages/nova/scheduler/filter_scheduler.py:462 2020-10-27 13:56:45.486 24 DEBUG nova.scheduler.utils [req-e2c6bf4b-7512-4ff0-9899-2a24bffa9131 113ee63a9ed0466794e24d069efc302c 32be191b3ec9497eaec108a22220890b - default default] Attempting to claim resources in the placement API for instance ac10e5df-7e55-4c88-a682-df25cb6dd535 claim_resources /usr/lib/python3.6/site-packages/nova/scheduler/utils.py:1175 2020-10-27 13:56:45.657 24 DEBUG nova.scheduler.filter_scheduler [req-e2c6bf4b-7512-4ff0-9899-2a24bffa9131 113ee63a9ed0466794e24d069efc302c 32be191b3ec9497eaec108a22220890b - default default] [instance: ac10e5df-7e55-4c88-a682-df25cb6dd535] Selected host: (compute-4, compute-4) ram: 131698MB disk: 14251008MB io_ops: 0 instances: 17 _consume_selected_host /usr/lib/python3.6/site-packages/nova/scheduler/filter_scheduler.py:354 ========================================= Thanks! Tony

2 2

[tc][all] Wallaby Cycle Community Goals
by Graham Hayes 27 Oct '20

27 Oct '20

Hi All It is that time of year / release again - and we need to choose the community goals for Wallaby. Myself and Nate looked over the list of goals [1][2][3], and we are suggesting one of the following: - Finish moving legacy python-*client CLIs to python-openstackclient - Move from oslo.rootwrap to oslo.privsep - Implement the API reference guide changes - All API to provide a /healthcheck URL like Keystone (and others) provide Some of these goals have champions signed up already, but we need to make sure they are still available to do them. If you are interested in helping drive any of the goals, please speak up! We need to select goals in time for the new release cycle - so please reply if there is goals you think should be included in this list, or not included. Next steps after this will be helping people write a proposed goal and then the TC selecting the ones we will pursue during Wallaby. Additionally, we have traditionally selected 2 goals per cycle - however with the people available to do the work across projects Nate and I briefly discussed reducing that to one for this cycle. What does the community think about this? Thanks, Graham 1 - https://etherpad.opendev.org/p/community-goals 2 - https://governance.openstack.org/tc/goals/proposed/index.html 3 - https://etherpad.opendev.org/p/community-w-series-goals 4 - https://governance.openstack.org/tc/goals/index.html#goal-selection-schedule

7 16

[ops][cinder] ceph version survey for operators
by Brian Rosmaita 27 Oct '20

27 Oct '20

Hello Cinder operators, tl;dr - if you use Ceph with your Cinder installation, please fill out this short (4 question) survey before 18:00 UTC on Wednesday 28 October 2020: https://rosmaita.wufoo.com/forms/ceph-version-survey/ Cinder currently contains an RBD driver that supports a Ceph backend, but it does not restrict Ceph to a particular range of versions. We'd like to take advantage of recent developments in Ceph to make some improvements to the RBD driver, but first want to gather data on what versions of Ceph are actually being used with Cinder so we can figure out backward compatibility issues we may face. (Basically, keeping backward compatibility for Ceph versions that don't have the new features will add complexity to the driver code, and more complex code is more likely to have bugs, and nobody likes bugs. Hence, we don't want to carry compatibility code for Ceph versions that no one is using.) If you manage multiple clouds, please fill out a separate survey for each cloud. We plan to discuss this issue at the PTG next week, so please respond before 18:00 UTC on Wednesday 28 October 2020. Here's the survey: https://rosmaita.wufoo.com/forms/ceph-version-survey/ Thank you!

1 1

[PTG] [Diversity] Extra session for Divisive Language discussions
by Amy Marrich 27 Oct '20

27 Oct '20

During the Monday session there were some questions in regards to technical writing and how things should be worded. I have invited Joanne O'Flynn, a Technical Writer from Red Hat to join us on Thursday at 17:00 UTC to help address these questions. This is not a normal PTG time slot but it was the only option due to time zones and already booked sessions for some of the participants from Monday. Thanks, Amy Marrich (spotz)

1 1

[TripleO] [ptg] deployment flow
by Kanevsky, Arkady 27 Oct '20

27 Oct '20

After today's PTG meeting want to make sure I have the right understanding of deployment flow. After deploying undercloud, are these the steps need to be done that are partially outside TripleO (https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/provisi… ): 1. Ironic discovery to discover all nodes. (networking for ironic must be in place for all nodes) 2. Metalsmith will provision baremetal machines - https://docs.openstack.org/metalsmith/latest/ 3. Metalsmith provisions OS on provisioned nodes 4. Something(???) optionally provisions Ceph (based on appropriate roles) 5. Something(???) optionally provisions other 3rd party services, like switch provisioning (based on appropriate roles) 6. Metalsmith optionally provisions Glance (may require Ceph or other 3rd party storage) 7. Metalsmith optionally provisions Neutron (may require switch provisioning) 8. TripleO deploys overcloud Is this the correct flow? Do we still need "openstack overcloud node provision" call or metalsmith already handles it? Or this call uses metalsmith under the cover? Thanks, Arkady

2 3

[requirements] Gerrit breach and commit audit
by Thierry Carrez 27 Oct '20

27 Oct '20

Following the recent gerrit breach, I manually audited all recent changes to the openstack/requirements repository, and could not find suspicious activity. Cheers, -- Thierry Carrez (ttx)

1 0

[cinder] qemu-img info is called as root even if nas_secure_file_operations is set to true
by Jonas Schäfer 27 Oct '20

27 Oct '20

Hi list, We’ve got a curious issue here. We’re trying to configure Cinder for use with NetApp NFS. The NetApp has been configured to disallow superuser access and cinder has been set up with nas_secure_file_operations set to true. While operations like creating volumes work, attaching a volume to an instance fails with the following traceback: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 4369, in _connection_create conn_info = self.driver.initialize_connection(volume, connector) File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nfs.py", line 134, in initialize_connection volume['name']) File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/nfs.py", line 543, in _qemu_img_info run_as_root=True) File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/remotefs.py", line 765, in _qemu_img_info_base run_as_root=run_as_root) File "/usr/lib/python2.7/dist-packages/cinder/image/image_utils.py", line 111, in qemu_img_info prlimit=QEMU_IMG_LIMITS) File "/usr/lib/python2.7/dist-packages/cinder/utils.py", line 126, in execute return processutils.execute(*cmd, **kwargs) File "/usr/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 425, in execute cmd=sanitized_cmd) ProcessExecutionError: Unexpected error while running command. Command: /usr/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --force-share /var/lib/cinder/mnt/1be0f17801f5c315c80d99a59518e26e/volume- ae4bba9b-2ab3-4234-8f86-352f1a20e086 Exit code: 1 Stdout: u'' Stderr: u"qemu-img: Could not open '/var/lib/cinder/mnt/ 1be0f17801f5c315c80d99a59518e26e/volume-ae4bba9b-2ab3-4234-8f86-352f1a20e086': Could not open '/var/lib/cinder/mnt/1be0f17801f5c315c80d99a59518e26e/volume- ae4bba9b-2ab3-4234-8f86-352f1a20e086': Permission denied\n" It seems as if cinder executes qemu-img as root. This is confirmed by looking at the code [3]. Looking at the superclass [4], we see that the superclass would’ve decided based on the setting of nas_secure_file_operations if [3] would not pass execute_as_root as True. To us, it makes no sense whatsoever. While [1] mentions something about incorrect permissions on volumes, it does not seem to consider the case of running with NFS Security enabled at all. However, it also seems as if this change went through completely uncontested. Hence, we assume that we are in the wrong here and wanted to ask on the mailing list instead of filing a bug. Does anyone have insight on what we’re doing wrong / missing here? kind regards, Jonas [1]: https://review.opendev.org/#/c/461134/ [2]: https://github.com/openstack/cinder/commit/dbc9c7ca [3]: https://github.com/openstack/cinder/blob/stable/queens/cinder/volume/ drivers/nfs.py#L537-L543 [4]: https://github.com/openstack/cinder/blob/stable/queens/cinder/volume/ drivers/remotefs.py#L753-L761 [5]: https://github.com/openstack/cinder/blob/stable/queens/cinder/volume/ drivers/nfs.py#L431-L432 -- Jonas Schäfer DevOps Engineer Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer(a)cloudandheat.com | www.cloudandheat.com New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/ Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp

1 0

[masakari] PTG starting Thursday
by Radosław Piliszek 27 Oct '20

27 Oct '20

Hello Everyone! A kind reminder that Masakari has PTG scheduled for the upcoming Thursday and Friday. First session: Thursday October 29, 2020 06:00 - 08:00 (UTC) Feel free to get acquainted with the etherpad [1] in the meantime. [1] https://etherpad.opendev.org/p/masakari-wallaby-vptg Cheers! -yoctozepto

1 0

Overcloud fatal error TASK [Ensure system is NTP time synced]
by dangerzone ar 27 Oct '20

27 Oct '20

Hi, I'm facing overcloud failure at the same line related with 'Ensure system is NTP time synced' *The error below refer:-* *TASK [Ensure system is NTP time synced] ****************************************fatal: [overcloud-controller-0]: FAILED! => {"changed": true, "cmd": ["chronyc", "waitsync", "20"], "delta":......* *controller timedatectl:-* *[heat-admin@overcloud-controller-0 ~]$ timedatectl Local time: Tue 2020-10-27 08:53:19 UTC Universal time: Tue 2020-10-27 08:53:19 UTC RTC time: Tue 2020-10-27 08:53:19 Time zone: UTC (UTC, +0000) NTP enabled: yesNTP synchronized: no RTC in local TZ: no DST active: n/a* *undercloud timedatectl:-* * Local time: Tue 2020-10-27 17:36:11 +08 Universal time: Tue 2020-10-27 09:36:11 UTC RTC time: Tue 2020-10-27 09:36:11 Time zone: Asia/Singapore (+08, +0800) NTP enabled: yesNTP synchronized: yes RTC in local TZ: no DST active: n/a* Tripleo deployment for 1compute and 1controller. I have run few times and end-up with the same error. Seems it's all related to chrony. No more ntp in rocky release and all change to chrony to handle time synchronization. Based on my reading the issue should be resolved in rocky but clearly my overcloud deployment failed at this stage. Please advise how to resolve this problem. Thank you all.

1 0

[cyborg] [ptg] cyborg Wallaby PTG schedule and topics
by yumeng bao 26 Oct '20

26 Oct '20

Hi all, Please check the cyborg PTG schedule below, and here are all the topics[0] that we've proposed, can't wait to see you tomorrow! And we plan to have a team photo on the first day at our internal meeting, come and join! ^^ Tuesday (27th ) 6:00 UTC - 8:00 UTC, Cyborg internal discussion Tuesday (27th) 13:00 - ?, Edge group discussion Wednesday (28th ) 6:00 UTC - 8:00 UTC, Cyborg internal Wednesday (28th): 13:00 - 14:00: Nova-Cyborg cross project session Tuesday (29th ) 6:00 UTC - 8:00 UTC, Cyborg internal [0]https://etherpad.opendev.org/p/cyborg-wallaby-goals Regards， Yumeng

2 1

[tacker] PTG planning
by Yasufumi Ogawa 26 Oct '20

26 Oct '20

Hi Tacker team, I've updated our vPTG etherpad for topics you've proposed [1]. We will start it from 27 Oct 6am UTC. So, I would like to skip the next IRC meeting this week. You can find Tacker's room on Zoom from schedule tracks here [2]. I'd also like to ask you to join in taking team photo on our 2nd day, although it's not mandatory. Thanks, Yasufumi [1] https://etherpad.opendev.org/p/Tacker-PTG-Wallaby [2] http://ptg.openstack.org/ptg.html

2 2

[ptg][swift][ops] Operator feedback session 28 Oct 21:00-22:00 UTC
by Tim Burke 26 Oct '20

26 Oct '20

Later this week, the Swift community will have an operator feedback session during our PTG slot [0]. If you're running a swift cluster, large or small, we'd love to see you there! As we've done in the past [1], we'll have an etherpad going [2] -- feel free to start filling it in and adding discussion items today! Even if you won't make the meeting, we'd appreciate your feedback. Looking forward to hearing from you! Tim [0] 28 Oct at 2100 UTC in https://www.openstack.org/ptg/rooms/kilo [1] https://wiki.openstack.org/wiki/Swift/Etherpads#List_of_Ops_Feedback_Etherp… [2] https://etherpad.opendev.org/p/swift-wallaby-ops-feedback

1 0

[keystone][ptg] Keystone vPTG Planning
by Kristi Nikolla 26 Oct '20

26 Oct '20

Hi all, I have started a doodle poll for scheduling our sessions during the vPTG. Please fill it out by September 10. [0] The etherpad that we are using for brainstorming the topics is [1]. Please fill it out with anything you would like us to schedule time for. More information on the vPTG and links to register can be found at [2]. [0]. https://doodle.com/poll/czgyuwepxk348vqq [1]. https://etherpad.opendev.org/p/keystone-wallaby-ptg [2]. https://www.openstack.org/ptg/

2 1

Help with meetpad needed
by Slawek Kaplonski 26 Oct '20

26 Oct '20

Hi, In Neutron sessions we are successfully using meetpad. But I have one question about it, maybe You will be able to help me somehow. When I click "open shared document" in the meetpad, etherpad is displayed but only until someone will start talking or someone new will join the call. Is it possible somehow to force display of the etherpad always? Thx in advance for any help with that. -- Slawek Kaplonski Principal Software Engineer Red Hat

2 1

How to use condition for resource creation via heat template
by Md. Hejbul Tawhid MUNNA 26 Oct '20

26 Oct '20

~~~ heat_template_version: 2017-02-24 ...... ...... ...... conditions: port_security_enabled: equals: - { get_param: port_security_enabled } - "true" resources: port: type: OS::Neutron::Port properties: admin_state_up: true network_id: { get_param: internal_net } port_security_enabled: { get_param: port_security_enabled } security_groups: { get_param: security_group } I want to create a condition , so if I set "port_security_enabled = false" security group will not apply in port.properties . If I set "port_security_enabled = true" then security group will applied. Please help me

2 1

[tripleo][overcloud] Deployment templates and command
by dangerzone ar 26 Oct '20

26 Oct '20

Hi, after exploring openstack via tripleo deployment, I encountered several issues especially during overcloud deployment. Most of the time I'm referring to openstack official web guideline and some other blogs resources. From my experiences I face problems identifying the right templates and the right command for my deployment. I have seen templates created in guideline web and some other examples. Even the command line to deploy overcloud is also a bit different from one example to another. I understand it is because of different requirements and setup. So many templates and several ways of deployment always made me puzzle and endup with deployment failure. I would like some advice to know what are the common or important templates/yaml files and overcloud command to use if I just want to deploy 3 controllers (ha) and several computes only. I also have plan to get dpdk in my deployment but...this will be later as I would like to successfully establish simple deployment with controllers and computes for now. Appreciate your good advice and support. Thank you all. This is some web link I use to refer:- ** https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/feature… ** https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13… ** https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/ Regards, daz

2 1

Help about openstack Storlets
by ghasemir 26 Oct '20

26 Oct '20

hi I am working on Openstack Storlets. I ran Swift in a Docker(container) and I want to put it in the Storlet midleware. How can that be? Can you help? My main question: Is there a place or activity group about Storlet to share comments and get helps? Thanks a lot Best regards Reza Ghasemi Department of Computer Engineering,Sharif University of Technology

2 1

[ptg][kolla] Please stop translating Etherpads
by Mark Goddard 26 Oct '20

26 Oct '20

Hi, Over the weekend, the Kolla Etherpad [1] has had various sections replaced with (what I think is) a Chinese translation, twice. This replaces the original English text for everyone. I guess that this is some browser based translator that does not realise it is modifying the page content for everyone. The export/import procedure I used to restore the etherpad loses some formatting, attributions, and may have lost content if changes were made in between. Please, we need to find another solution for translation. If you need to make translations, please copy text to another window, or at least provide translations for specific text *next to the English*. If this happens again we will need to move to another tool, such as Google docs. I would much prefer not to do this, and rather use an open source tool that is widely accessible, but we do need to be able to collaborate this week without disruption. I look forward to working with you this week. I hope this does not discourage non-native English speakers, of which there is generally a majority. If I can provide any assistance to make the sessions more accessible, please get in touch. Cheers, Mark [1] https://etherpad.opendev.org/p/kolla-wallaby-ptg

4 6

[First Contact] [SIG] [PTG] Meeting Reminder
by Kendall Nelson 26 Oct '20

26 Oct '20

Hello Everyone! Just wanted to remind you all that we have time scheduled at the PTG for Thursday October 29th at 23:00 UTC. Also, feel free to add agenda items here[1] in the meantime. Hope to see you there! -Kendall Nelson (diablo_rojo) [1] https://etherpad.opendev.org/p/wallaby-ptg-first-contact

1 0

[octavia] Weekly IRC meeting is cancelled this week due to the PTG
by Michael Johnson 26 Oct '20

26 Oct '20

Hello Octavia community, As we will be meeting for the PTG this week, I am cancelling the weekly IRC meeting on October 28th. Looking forward to seeing you all at the PTG! Michael

1 0

[neutron] bug deputy report October 19th - 25th
by Miguel Lavalle 26 Oct '20

26 Oct '20

Hi Neutrinos, Here's the bug report for the week of October 19th - 25th. It was a very quiet week High ==== https://bugs.launchpad.net/neutron/+bug/1900763 [OVN Octavia Provider] OVN provider status update failures can leave orphaned resources. It has an owner already (haleyb) RFE === https://bugs.launchpad.net/neutron/+bug/1900934 [RFE][DHCP][OVS] flow based DHCP. Will be discussed during PTG

1 0

[Glance] create image by downloading/importing from UI vs. CLI
by Tony Liu 26 Oct '20

26 Oct '20

Hi, The initial issue I saw was "The Store URI was malformed." from Horizon when creating image by URL [1]. It works fine with Glance CLI as the following. ======== glance image-create --container-format bare --disk-format qcow2 --name cirros-test glance image-import --import-method web-download --uri https://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img cirros-test ======== When I look into it, I see that UI and CLI work differently, they are handled by different workflow in Glance. Why is that? Why not UI follows the same workflow as CLI? The root cause is that, [1] redirect to [2] and redirect to [3]. One issue here is that, query is not supported in [4]. pieces.query is dropped. When I attached query parameters, I got 403. And that's because, the workflow for UI is checking quota policy when adding location. It needs to get the size of image by HEAD request. But the signature in [3] is for GET request. So that HEAD request is forbidden by S3. I don't think we can get an URL with signature for HEAD request in such case. So I go for an ugly workaround to skip the size check. Any ideas to make it work better? Is quota policy involved in CLI workflow? Again, can we make UI work the same way as CLI? [1] https://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img [2] https://github.com/cirros-dev/cirros/releases/download/0.5.1/cirros-0.5.1-x… [3] https://github-production-release-asset-2e65be.s3.amazonaws.com/219785102/6… [4] https://github.com/openstack/glance_store/blob/488d3e2fe31e4fc7a461e9de8b20… Thanks! Tony

3 2

[heat] PTG on this wednesday 13-15 UTC
by Rico Lin 26 Oct '20

26 Oct '20

Hi all Heat PTG room reserved on Wednesday 13-15 UTC at Juno room Feel free to update our PTG etherpad [1] [1] https://etherpad.opendev.org/p/wallaby-ptg-heat -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

[multi-arch SIG] PTG plan this week
by Rico Lin 26 Oct '20

26 Oct '20

Hi all, Multi-arch SIG will have our PTG on 10/27 Tuesday 07-08 UTC, and 15-16 UTG This week at room Juno Feel free to update our etherpad [1] [1] https://etherpad.opendev.org/p/wallaby-ptg-multiarch-sig -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

1 0

linux.conf.au 2021 - Call for Sessions and Miniconfs Open
by Joshua Hesketh 26 Oct '20

26 Oct '20

The best F/OSS conference in the southern hemisphere is back again - this time virtualised and without a 20+ hour flight! -, January 23-25. The CFP is open until November 6. Submit early, submit often! -------- Forwarded Message -------- Subject: [lca-announce] linux.conf.au 2021 - Call for Sessions and Miniconfs Open Date: Thu, 15 Oct 2020 20:43:24 +1000 From: linux.conf.au Announcements <lca-announce(a)lists.linux.org.au> <lca-announce(a)lists.linux.org.au> Reply-To: lca-announce(a)lists.linux.org.au To: lca-announce(a)lists.linux.org.au We're excited! The linux.conf.au 2021 Call for Sessions and Call for Miniconfs are now open. They will stay open until 6th November 2020 Anywhere on Earth (AoE) (https://en.wikipedia.org/wiki/Anywhere_on_Earth) This is only 3 weeks away - so don't delay, get your talks in early! Our theme is "So what's next?". We all know we're living through unprecedented change and uncertain times. How can open source play a role in creating, helping and adapting to this ongoing change? What new developments in software and coding can we look forward to in 2021 and beyond? If you have ideas or developments you'd like to share with the open source community at linux.conf.au, we'd love to hear from you. Call for Sessions The main conference runs on Sunday 24 and Monday 25 January, with multiple streams catering for a wide range of interest areas. We invite you to submit a session (https://linux.conf.au/programme/sessions/) proposal for a talk. Talks are generally 35-45 minute presentations on a single topic presented in lecture format. Call for Miniconfs Miniconfs are dedicated day-long streams focusing on single topics, creating a more immersive experience for delegates than a session. We encourage you to get creative with how you could deliver your Miniconf virtually! Running a Miniconf (https://linux.conf.au/programme/miniconfs/) is a great way to gain experience, provide exposure for your project or topic, and raise your professional profile. They're a crowd favourite and an awesome way to kick off the conference. Miniconfs will run on Saturday 23 January, before the main conference commences on Sunday. No need to book flights or hotels Don't forget: the conference will be a fully online, virtual experience. This means our speakers will be beaming in from their own homes or workplaces. The organising team will be able to help speakers with their tech set-up. Each accepted presenter will have a tech check prior to the event to smooth out any difficulties and there will be an option to pre-record presentations. Have we piqued your interest? You can find out how to submit your session or miniconf proposals at https://linux.conf.au/programme/proposals/. If you have any other questions you can contact us via email at contact(a)lca2021.linux.org.au. Timeline 16th October: Call for sessions opens 6th November: Call for sessions closes January 23 2021: Miniconfs! January 24-25 2021: Main conference presentations Tickets will go on sale in the coming weeks. We'll keep you posted. We're looking forward to reading your submissions. linux.conf.au 2021 Organising Team About linux.conf.au 2021 Running since 1999, linux.conf.au is the largest linux and open source conference in the Asia-Pacific region. The conference provides deeply technical presentations from industry leaders and experts on a wide array of subjects relating to open source projects, data and open government and community engagement.

1 0

[oslo] PTG Update
by Ben Nemec 26 Oct '20

26 Oct '20

Hi, Based on the lack of topic proposals in the etherpad[0], we are currently planning to skip the Oslo PTG session for this cycle. If you have something to discuss please contact the team ASAP so we have time to set something up. Thanks. -Ben 0: https://etherpad.opendev.org/p/oslo-wallaby-topics

3 3

[largescale-sig] Two PTG meetings
by Thierry Carrez 26 Oct '20

26 Oct '20

Hi everyone, Following a well-attended forum session last week on sharing your scaling story[1], the Large Scale SIG have *two* meetings this week during the PTG: Wednesday Oct 28, 7:00 UTC Wednesday Oct 28, 16:00 UTC To join either of those, please register for the PTG[2] and click on "largescale-sig" on the schedule[3]. I hope to see you there! [1] https://etherpad.opendev.org/p/vSummit2020_OpenStackScalingStory [2] https://www.openstack.org/ptg [3] http://ptg.openstack.org/ptg.html -- Thierry Carrez (ttx)

1 0

[docs][Zaqar]Zaqar API reference is missing from doc.openstack.org
by hao wang 26 Oct '20

26 Oct '20

Hi， I'm Hao Wang, the PTL of Zaqar project. I found that Zaqar API reference is missing from https://docs.openstack.org/victoria/api/. Is there something wrong or what I can help to fix it, please let me know. Thank you.

2 1

[tripleo] proposed schedule TripleO meetup @ Wallaby PTG
by Marios Andreou 25 Oct '20

25 Oct '20

Hello tripleOs o/ I would like to share a proposed schedule for the TripleO meetup at the coming PTG. The proposal is at [1]. First I'd like to give a huge thank you to everybody for taking the time to volunteer and lead those sessions. We are lucky to have folks bringing a wide array of topics including deployment, edge, upgrades, high-availability, ci, storage and networking. The schedule has 5 fourty minute sessions per day (we have 4 hours/day 1300-1700 UTC) with 10 mins breaks between them. If you would like to propose an alternative structure or have suggestions/improvements etc then let's discuss it here. If you want to make changes to your session time please reach out directly to the person running the time slot you wish to change with. If there are no objections then please go ahead and swap the sessions in the schedule (please let me know too). If you can't reach that person or there are any other problems please message me and we will work it out together. If you have added (or plan to add) a new discussion topic in [2] and you don't see it in [1] then please let me know so I can add it. If you are leading a session then please remember to prepare the session etherpad over the next few days and link it at [1]. Of course, make sure you are registered for the PTG :) at [3] thank you for reading and for sharing any thoughts/suggestions/improvement to the proposed schedule [1], regards (+ happy friday !) marios [1] https://etherpad.opendev.org/p/tripleo-ptg-wallaby [2] https://etherpad.opendev.org/p/tripleo-wallaby-topics [3] https://october2020ptg.eventbrite.com/

1 2

[cinder][grenade][neutron] Gate failure in grenade jons
by Slawek Kaplonski 25 Oct '20

25 Oct '20

Hi Cinder team, I saw in Neutron CI in last few days that all our grenade jobs (multinode if that is important) are failing with error which seems to be related to Cinder. I opened bug [1] for that. Please take a look at it as it seems pretty serious issue because our gate is now broken because of that. Sorry if there is already another bug reported for the same problem and I created duplicate :) If bug is somehow hard to fix or there is nobody who can take a look, I will be also greatful for such info so we can temporary mark those grenade jobs as non-voting to unblock our gate :) Thx a lot in advance for Your help. [1] https://bugs.launchpad.net/cinder/+bug/1901384 -- Slawek Kaplonski Principal Software Engineer Red Hat

2 2

[scientific-sig] Scientific SIG at the PTG
by Stig Telfer 24 Oct '20

24 Oct '20

Hi All - The Scientific SIG will be at the PTG this coming week, and everyone is welcome. We have two sessions on Monday: - 15:00-17:00 UTC to discuss use cases and requirements for SIG members. - 21:00-23:00 UTC to include lightning talks from SIG members and further discussions. We would love to have participants contribute lightning talks for the second session. It’s easy to do, particularly if you’ve just presented on something at the summit, or have new material to try out before SC :-) Our PTG Etherpad is available here: https://etherpad.opendev.org/p/wallaby-ptg-scientific-sig Remember to register for PTG attendance here: https://october2020ptg.eventbrite.com <https://october2020ptg.eventbrite.com/> Cheers, Stig

1 0

[nova] PTG schedule
by Balázs Gibizer 24 Oct '20

24 Oct '20

Hi, I did a bit of reordering of the topics in the PTL etherpad[1] to move similar topics closer to each other. I did not try to assign specific timing to each topic. We will use the ptgbot during the PTG to notify the audience what we are talk about and what is the next topic. See you next week! Cheers, gibi [1]https://etherpad.opendev.org/p/nova-wallaby-ptg

2 3

[cinder] PTG schedule
by Brian Rosmaita 23 Oct '20

23 Oct '20

Hello Cinder folks and fellow travelers, I've updated the official Cinder Wallaby PTG etherpad with discussion topics: https://etherpad.opendev.org/p/wallaby-ptg-cinder It's organized by days to give you a rough idea of what will be discussed when. As always, we will be flexible about moving things around if we have to. Friday has a little bit of slack built in, so I anticipate that we'll be able to discuss everything on the schedule. Please take a look at the schedule as soon as you can to identify the discussions you are leading and let me know right away if you anticipate any conflicts. Info for connecting to the Cinder sessions is on the etherpad. The plan is that we'll have discussions from 1300-1600 UTC Tuesday-Friday, and have optional "hallway time" from 1700-1800 UTC Tuesday-Friday to try to reproduce the useful informal encounters we'd be having at a face-to-face event. (This time is not ideal for APAC developers; we could additionally have "hallway time" from 1200-1300 UTC if there is interest. Let me know.) Don't forget that we decided to have Happy Hour the *first* day of meeting (Tuesday) so that people can get to know each other a bit. Hopefully this will make everyone more comfortable for participating in the discussions. Happy Hour is at 1500 UTC on Tuesday 27 October. Monday is open for you to attend non-cinder events; some are noted on the etherpad. Please add other sessions that Cinder people might find of interest. Looking forward to seeing everyone next week! brian

1 0

[openstack-ansible][osa] Ubuntu based deployments issue
by Dmitriy Rabotyagov 23 Oct '20

23 Oct '20

Hi, We have to announce, that due to Ubuntu has removed base images for all non-current point releases (ie 18.04.3) [1], all current or new OSA deployments will fail during base image retrievement process, which is part of the lxc_hosts role (playbook containers-lxc-host.yml which is included in setup-hosts.yml). This mean, that attempt of the new deployment, minor and major upgrade or container re-creation will fail. This won't influence already running containers in any manner. This affects ubuntu 16.04, 18.04 and 20.04 deployments for all OSA releases. At the moment we are actively working on the proper solution to prevent this from happening in the future (or at least minimize risk as much as we can). You can keep track about the progress in bug reports [2] and [3] In the meanwhile there are several workarounds, that you may easily apply locally to make your deployments working: 1. Set proper URL to the relevant Ubuntu images by setting variable `lxc_hosts_container_image_url` in user_variables.yml Relevant URL at the moment are folowing: - Ubuntu 20.04: http://cdimage.ubuntu.com/ubuntu-base/releases/20.04/release/ubuntu-base-20… - Ubuntu 18.04: http://cdimage.ubuntu.com/ubuntu-base/releases/18.04/release/ubuntu-base-18… - Ubuntu 16.04: http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16… 2. Use images provided by LXC team. To do this you will need to set `lxc_hosts_container_image_download_legacy: true` in your user_variables.yml Once we will have a reliable solution, we will backport it back to all supported releases and make new OSA release right afterwards. Right after that I will write an update to this notification with information about stable release number that fixes this issue. [1] https://lists.ubuntu.com/archives/ubuntu-release/2020-October/005113.html [2] https://bugs.launchpad.net/openstack-ansible/+bug/1900884 [3] https://bugs.launchpad.net/openstack-ansible/+bug/1900975 -- Kind Regards, Dmitriy Rabotyagov

1 0

[swift] Gerrit breach and commit audit
by Tim Burke 23 Oct '20

23 Oct '20

By now, I'm sure you've sen a lot of emails like these, but I've manually verified that all commits and tags across all branches are correct and proper for * swift, * python-swiftclient, and * liberasurecode following the recent gerrit security incident. PyEClib and swift-bench saw no activity during that time, so had nothing to verify. I'd just like to thank the infra team for their quick response and transparency throughout all of this; the notifications during the incident and final write-up [0] were really top-notch. Tim [0] http://lists.opendev.org/pipermail/service-announce/2020-October/000011.html

1 0

[horizon][glance] "The Store URI was malformed."
by Tony Liu 23 Oct '20

23 Oct '20

Hi, With Ussuri, when create image on Horizon with specified URL, I got error "The Store URI was malformed.". Here is the URL I put in "Location" box. https://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img What I am missing here? Thanks! Tony

1 1

[neutron] [neutron-dynamic-routing] Gerrit breach and commit audit for neutron-dynamic-routing
by Tobias Urdin 23 Oct '20

23 Oct '20

Hello, Could not find a neutron thread on this, anyways, as per the announcement in [1] I have audited the neutron-dynamic-routing where I'm a reviewer. Best regards [1] http://lists.opendev.org/pipermail/service-announce/2020-October/000011.html

2 1

[Open Infrastructure Foundation] [InteropWG] Planning meeting at 10 AM PDT in an hr
by prakash RAMCHANDRAN 23 Oct '20

23 Oct '20

Hi all, Please joinInterop Working Group - Weekly Friday 10-11 AM PDT Lnink: https://meetpad.opendev.org/Interop-WG-weekly-meeting We have scheduled 4 Slots at PTG. We will have today's 10/23 10AM PDT / 12 Noon CDT / 1.00 PM EST / 18 UTC/22.30 IST / 10/24 1 AM BJT Main focus will bePrep for PTG Request add your suggestions to https://etherpad.opendev.org/p/interop-wallaby-ptg ----------------------------------------------Session 1,2 Monday 23rd OCThttp://ptg.openstack.org/ptg.html ---------------------------------------------Session 3,4 Friday 30th OTC http://ptg.openstack.org/ptg.html --------------------------------------------- With hectic engagements please do start with InteropWG and end PTG with InteropWG.We see with kubernetes as defacto interop for Cloud Native Infrastructure we have more opportunities leverage Hybrid Cloud in Open Infrastructure PTG ThanksPrakashFor Interop WG

2 2

[nova][gate] openstack-tox-docs job broken
by melanie witt 23 Oct '20

23 Oct '20

Hey all, The pygments package version was bumped from 2.6.1 to 2.7.1 in the following openstack/requirements upper-constraints change that merged earlier today: https://review.opendev.org/750084 And it broke our openstack-tox-docs job as pygments 2.7.x validates docs for valid JSON as of: https://github.com/pygments/pygments/commit/9514e794e0c2a5c7c048df97fcfef4a… The fix is proposed here: https://review.opendev.org/759466 Please hold your rechecks until ^ merges. Cheers, -melanie

1 0