From pramchan at yahoo.com Sat Aug 1 02:43:36 2020 From: pramchan at yahoo.com (prakash RAMCHANDRAN) Date: Sat, 1 Aug 2020 02:43:36 +0000 (UTC) Subject: [Openstack-Interop] Friday 31 - Updates and Questions for future of Interop? References: <424140393.10386729.1596249816702.ref@mail.yahoo.com> Message-ID: <424140393.10386729.1596249816702@mail.yahoo.com> Hi all, We looked at OSF and Open-Infra initiatives and for sustaining and enhancing the Interop activity need your support. The current meeting notes are available on the site and will be meeting alternate Fridays Aug 14, 28,... 10 AM PST / 17 UTC - going forward on meetpad: Lnink: https://meetpad.opendev.org/Interop-WG-weekly-meeting Details in therpadhttps://etherpad.opendev.org/p/interop Do you have suggestions for Branding efforts? Few Question: 1. Interop WG being an OSF or Board Driven, how can Interop work with Projects to ensure Branding Integrated Logo Programs and other Cloud Providers can leverage OpenStack Logo Program? a. We do  have role to guide & reviewing refstack reports for Branding. We have decided to decouple Marketplace  & Branding programs.Seeking feedback from community on proposed add-on programs for Bare metal(Ironic, MaaS,...) & "Kubernetes-ready OpenStack"  b. Refstack hosting / Refstack-client are currently not maintained due to lack of volunteers to support. (seeking volunteers for updates - please reach out to @gmann in TC. c. Await user & operator survey annual reports, but if you have any Branding innovation like to propose or contribute please brainstorm your ideas here and bring it to midweek community meetings next week.  ThanksFor Interop WGPrakash -------------- next part -------------- An HTML attachment was scrubbed... URL: From anilj.mailing at gmail.com Sat Aug 1 21:36:37 2020 From: anilj.mailing at gmail.com (Anil Jangam) Date: Sat, 1 Aug 2020 14:36:37 -0700 Subject: Two subnets under same network context. Message-ID: Hi, I have observed that one can create two subnets under the same network scope. See below an example of the use case. [image: Screen Shot 2020-08-01 at 2.22.15 PM.png] Upon checking the data structures, I saw that the segment type (vlan) and segment id (55) is associated with the "network" object and not with the "subnet" (I was under impression that the segment type (vlan) and segment id (55) would be allocated to the "subnet"). When I create the VM instances, they always pick the IP address from the SUBNET1-2 IP range. If the segment (vlan 55) is associated with "network" then what is the reason two "subnets" are allowed under it? Does it mean that VM instances from both these subnets would be configured under the same VLAN? /anil. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2020-08-01 at 2.22.15 PM.png Type: image/png Size: 65802 bytes Desc: not available URL: From skaplons at redhat.com Sun Aug 2 11:27:58 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Sun, 2 Aug 2020 13:27:58 +0200 Subject: Two subnets under same network context. In-Reply-To: References: Message-ID: <4991a915-210a-ad24-8497-fab029c1a050@redhat.com> Hi, This is "normal". You can have many subnets (both IPv4 and IPv6 in the one network). By default Neutron will associate to the port IP address only from one subnet of one type (IPv4/IPv6) but You can change it and tell Neutron to allocate for the port IP adresses from more than one subnet. If You have both IPv4 and IPv6 subnets in the network, Neutron will by default allocate one IPv4 and one IPv6 to each port. But again, You can manually tell Neutron to use e.g. only IPv6 address for specific port. Please check [1] and [2] for more details. [1] https://docs.openstack.org/neutron/latest/admin/intro-os-networking.html [2] https://docs.openstack.org/api-ref/network/v2/ W dniu 01.08.2020 o 23:36, Anil Jangam pisze: > Hi, > > I have observed that one can create two subnets under the same network > scope. See below an example of the use case. > > [image: Screen Shot 2020-08-01 at 2.22.15 PM.png] > Upon checking the data structures, I saw that the segment type (vlan) and > segment id (55) is associated with the "network" object and not with the > "subnet" (I was under impression that the segment type (vlan) and segment > id (55) would be allocated to the "subnet"). > > When I create the VM instances, they always pick the IP address from the > SUBNET1-2 IP range. If the segment (vlan 55) is associated with "network" > then what is the reason two "subnets" are allowed under it? > > Does it mean that VM instances from both these subnets would be configured > under the same VLAN? > > /anil. > -- Slawek Kaplonski Principal software engineer Red Hat From romain.chanu at univ-lyon1.fr Sun Aug 2 10:27:12 2020 From: romain.chanu at univ-lyon1.fr (CHANU ROMAIN) Date: Sun, 2 Aug 2020 10:27:12 +0000 Subject: Two subnets under same network context. In-Reply-To: References: Message-ID: <1596364031962.69223@univ-lyon1.fr> Hello, Network object is an isolation layer, it's defined by the cloud administrator: isolation type (VLAN, VXLAN...), physical NIC.. The subnet is a free value to cloud users, this mechanism allows multiples users to use same L3 networks (overlapping). So the network is used by admin to isolate the client and subnet is used by client to "isolate" his instances (webfont / db ...). Isolation works only on layer3 because all subnets will use the same layer2 (defined by admin). It's very easy to verify: boot one instance on each subnet then capture the traffic: you will see ARP trames. I dont know why Neutron drains all IP from last network but anyway the best practice is to create port then allocate to instance. Does it mean that VM instances from both these subnets would be configured under the same VLAN? > yes Best regards, Romain ________________________________ From: Anil Jangam Sent: Saturday, August 1, 2020 11:36 PM To: openstack-discuss Subject: Two subnets under same network context. Hi, I have observed that one can create two subnets under the same network scope. See below an example of the use case. [Screen Shot 2020-08-01 at 2.22.15 PM.png] Upon checking the data structures, I saw that the segment type (vlan) and segment id (55) is associated with the "network" object and not with the "subnet" (I was under impression that the segment type (vlan) and segment id (55) would be allocated to the "subnet"). When I create the VM instances, they always pick the IP address from the SUBNET1-2 IP range. If the segment (vlan 55) is associated with "network" then what is the reason two "subnets" are allowed under it? Does it mean that VM instances from both these subnets would be configured under the same VLAN? /anil. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2020-08-01 at 2.22.15 PM.png Type: image/png Size: 65802 bytes Desc: Screen Shot 2020-08-01 at 2.22.15 PM.png URL: From gmann at ghanshyammann.com Mon Aug 3 00:10:36 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sun, 02 Aug 2020 19:10:36 -0500 Subject: [all][tc][goals] Migrate CI/CD jobs to new Ubuntu LTS Focal: Week R-11 Update Message-ID: <173b1a7e32b.adf0e67a81133.1529455632674185621@ghanshyammann.com> Hello Everyone, Please find the week R-11 updates on 'Ubuntu Focal migration' community goal. Tracking: https://storyboard.openstack.org/#!/story/2007865 Progress: ======= * We passed the first deadline which I planned initially but looking at the failure happing it will definitely will take more time. My first goal is "zero downtime in gate", so if we finish it little late (with all repos tested) is ok. * ~80 repos gate have been tested/fixed till now. ** https://review.opendev.org/#/q/topic:migrate-to-focal+(status:abandoned+OR+status:merged) * 115 repos are under test and failing. Debugging and fixing are in progress (If you would like to help, please check your project repos if I am late to fix them): ** https://review.opendev.org/#/q/topic:migrate-to-focal+status:open * Patches ready to merge: ** https://review.opendev.org/#/q/topic:migrate-to-focal+status:open+label%3AVerified%3E%3D1%2Czuul+NOT+label%3AWorkflow%3C%3D-1 Bugs Report: ========== Summary: Total 4 (1 fixed, 3 in-progress). 1. Bug#1882521. (IN-PROGRESS) There is open bug for nova/cinder where three tempest tests are failing for volume detach operation. There is no clear root cause found yet -https://bugs.launchpad.net/cinder/+bug/1882521 We have skipped the tests in tempest base patch to proceed with the other projects testing but this is blocking things for the migration. 2. We encountered the nodeset name conflict with x/tobiko. (FIXED) nodeset conflict is resolved now and devstack provides all focal nodes now. 3. Bug#1886296. (IN-PROGRESS) pyflakes till 2.1.0 is not compatible with python 3.8 which is the default python version on ubuntu focal[1]. With pep8 job running on focal faces the issue and fail. We need to bump the pyflakes to 2.1.1 as min version to run pep8 jobs on py3.8. As of now, many projects are using old hacking version so I am explicitly adding pyflakes>=2.1.1 on the project side[2] but for the long term easy maintenance, I am doing it in 'hacking' requirements.txt[3] nd will release a new hacking version. After that project can move to new hacking and do not need to maintain pyflakes version compatibility. 4. Bug#1886298. (IN-PROGRESS) 'Markupsafe' 1.0 is not compatible with the latest version of setuptools[4], We need to bump the lower-constraint for Markupsafe to 1.1.1 to make it work. There are a few more issues[5] with lower-constraint jobs which I am debugging. What work to be done on the project side: ================================ This goal is more of testing the jobs on focal and fixing bugs if any otherwise migrate jobs by switching the nodeset to focal node sets defined in devstack. 1. Start a patch in your repo by making depends-on on either of below: devstack base patch if you are using only devstack base jobs not tempest: Depends-on: https://review.opendev.org/#/c/731207/ OR tempest base patch if you are using the tempest base job (like devstack-tempest): Depends-on: https://review.opendev.org/#/c/734700/ Both have depends-on on the series where I am moving unit/functional/doc/cover/nodejs tox jobs to focal. So you can test the complete gate jobs(unit/functional/doc/integration) together. This and its base patches - https://review.opendev.org/#/c/738328/ Example: https://review.opendev.org/#/c/738126/ 2. If none of your project jobs override the nodeset then above patch will be testing patch(do not merge) otherwise change the nodeset to focal. Example: https://review.opendev.org/#/c/737370/ 3. If the jobs are defined in branchless repo and override the nodeset then you need to override the branches variant to adjust the nodeset so that those jobs run on Focal on victoria onwards only. If no nodeset is overridden then devstack being branched and stable base job using bionic/xenial will take care of this. Example: https://review.opendev.org/#/c/744056/2 4. If no updates need you can abandon the testing patch (https://review.opendev.org/#/c/744341/). If it need updates then modify the same patch with proper commit msg, once it pass the gate then remove the Depends-On so that you can merge your patch before base jobs are switched to focal. This way we make sure no gate downtime in this migration. Example: https://review.opendev.org/#/c/744056/1..2//COMMIT_MSG Once we finish the testing on projects side and no failure then we will merge the devstack and tempest base patches. Important things to note: =================== * Do not forgot to add the story and task link to your patch so that we can track it smoothly. * Use gerrit topic 'migrate-to-focal' * Do not backport any of the patches. References: ========= Goal doc: https://governance.openstack.org/tc/goals/selected/victoria/migrate-ci-cd-jobs-to-ubuntu-focal.html Storyboard tracking: https://storyboard.openstack.org/#!/story/2007865 [1] https://github.com/PyCQA/pyflakes/issues/367 [2] https://review.opendev.org/#/c/739315/ [3] https://review.opendev.org/#/c/739334/ [4] https://github.com/pallets/markupsafe/issues/116 [5] https://zuul.opendev.org/t/openstack/build/7ecd9cf100194bc99b3b70fa1e6de032 -gmann From zhangbailin at inspur.com Mon Aug 3 00:10:59 2020 From: zhangbailin at inspur.com (=?utf-8?B?QnJpbiBaaGFuZyjlvKDnmb7mnpcp?=) Date: Mon, 3 Aug 2020 00:10:59 +0000 Subject: =?utf-8?B?562U5aSNOiBbbGlzdHMub3BlbnN0YWNrLm9yZ+S7o+WPkV1SZTogW0dsYW5j?= =?utf-8?Q?e]_Proposing_Dan_Smith_for_glance_core?= In-Reply-To: <8635120d-11d6-136e-2581-40d3d451d1aa@gmail.com> References: <8635120d-11d6-136e-2581-40d3d451d1aa@gmail.com> Message-ID: <03ece5d405c74b2d9292301c2e3be7b8@inspur.com> +1 发件人: Jay Bryant [mailto:jungleboyj at gmail.com] 发送时间: 2020年7月31日 23:39 收件人: openstack-discuss at lists.openstack.org 主题: [lists.openstack.org代发]Re: [Glance] Proposing Dan Smith for glance core On 7/31/2020 8:10 AM, Sean McGinnis wrote: On 7/30/20 10:25 AM, Abhishek Kekane wrote: Hi All, I'd like to propose adding Dan Smith to the glance core group. Dan Smith has contributed to stabilize image import workflow as well as multiple stores of glance. He is also contributing in tempest and nova to set up CI/tempest jobs around image import and multiple stores. Being involved on the mailing-list and IRC channels, Dan is always helpful to the community and here to help. Please respond with +1/-1 until 03rd August, 2020 1400 UTC. Cheers, Abhishek +1 Not a Glance core but definitely +1 from me. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Mon Aug 3 07:05:58 2020 From: zigo at debian.org (Thomas Goirand) Date: Mon, 3 Aug 2020 09:05:58 +0200 Subject: Two subnets under same network context. In-Reply-To: <1596364031962.69223@univ-lyon1.fr> References: <1596364031962.69223@univ-lyon1.fr> Message-ID: <01345dce-deb8-de63-4a01-bf86c6ac893e@debian.org> On 8/2/20 12:27 PM, CHANU ROMAIN wrote: > Hello, > > > Network object is an isolation layer, it's defined by the cloud > administrator: isolation type (VLAN, VXLAN...), physical NIC.. The > subnet is a free value to cloud users, this mechanism allows multiples > users to use same L3 networks (overlapping). So the network is used by > admin to isolate the client and subnet is used by client to "isolate" > his instances (webfont  / db ...). No, this isn't the way it works. Thomas From zigo at debian.org Mon Aug 3 07:14:19 2020 From: zigo at debian.org (Thomas Goirand) Date: Mon, 3 Aug 2020 09:14:19 +0200 Subject: Two subnets under same network context. In-Reply-To: References: Message-ID: On 8/1/20 11:36 PM, Anil Jangam wrote: > Hi,  > > I have observed that one can create two subnets under the same network > scope. See below an example of the use case.  > > Screen Shot 2020-08-01 at 2.22.15 PM.png > Upon checking the data structures, I saw that the segment type (vlan) > and segment id (55) is associated with the "network" object and not with > the "subnet" (I was under impression that the segment type (vlan) and > segment id (55) would be allocated to the "subnet").  > > When I create the VM instances, they always pick the IP address from the > SUBNET1-2 IP range. If the segment (vlan 55) is associated with > "network" then what is the reason two "subnets" are allowed under it?  > > Does it mean that VM instances from both these subnets would be > configured under the same VLAN?  > > /anil. Hi, If you want to use segments, with a different address range depending on where a compute is physically located (for example, a rack...), then you should first set a different name for the physical network of your nodes. This is done by tweaking these: [ml2_type_flat] flat_networks = rack-number-1 [ml2_type_vlan] network_vlan_ranges = rack-number-1 Then you can: 1/ create a network scope 2/ create a network using that scope, a vlan and "--provider-physical-network rack-number-1" and --provider-segment 3/ create a subnet pool using the network scope created above 4/ create a subnet attached to the subnet pool and network segment Then you can create more network segment + subnet couples addressing different location. Once you're done, VMs will get a different range depending on the rack they are in. Cheers, Thomas Goirand (zigo) From ignaziocassano at gmail.com Mon Aug 3 07:53:37 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 3 Aug 2020 09:53:37 +0200 Subject: [openstack][stein][manila-ui] error Message-ID: Hello, I installed manila on openstack stein and it works by command line mat the manila ui does not work and in httpd error log I read: [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR django.request Internal Server Error: /dashboard/project/shares/ [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback (most recent call last): [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response = get_response(request) [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response = self.process_exception_by_middleware(e, request) [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response = wrapped_callback(request, *callback_args, **callback_kwargs) [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return view_func(request, *args, **kwargs) [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return view_func(request, *args, **kwargs) [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return view_func(request, *args, **kwargs) [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return view_func(request, *args, **kwargs) [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return view_func(request, *args, **kwargs) [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return self.dispatch(request, *args, **kwargs) [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return handler(request, *args, **kwargs) [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = self.construct_tables() [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in construct_tables [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = self.handle_table(table) [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in handle_table [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = self._get_data_dict() [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in _get_data_dict [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] data.extend(func()) [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in wrapped [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = cache[key] = func(*args, **kwargs) [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", line 57, in get_shares_data [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] share_nets = manila.share_network_list(self.request) [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in share_network_list [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return manilaclient(request).share_networks.list(detailed=detailed, [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] AttributeError: 'NoneType' object has no attribute 'share_networks' Please, anyone could help ? Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From bdobreli at redhat.com Mon Aug 3 08:13:31 2020 From: bdobreli at redhat.com (Bogdan Dobrelya) Date: Mon, 3 Aug 2020 10:13:31 +0200 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? Message-ID: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> There is a trend of writing action plugins, see [0], for simple things, like just calling a module in a loop. I'm not sure that is the direction TripleO should go. If ansible is inefficient in this sort of tasks without custom python code written, we should fix ansible. Otherwise, what is the ultimate goal of that trend? Is that having only action plugins in roles and playbooks? Please kindly asking the community to stop that, make a step back and reiterate with the taken approach. Thank you. [0] https://review.opendev.org/716108 -- Best regards, Bogdan Dobrelya, Irc #bogdando From thierry at openstack.org Mon Aug 3 09:56:16 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 3 Aug 2020 11:56:16 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: <274f7289-136a-e829-bdf5-1c819355ce77@openstack.org> Sean McGinnis wrote: > Posting here to raise awareness, and start discussion about next steps. > > It appears there is no one working on Cloudkitty anymore. No patches > have been merged for several months now, including simple bot proposed > patches. It would appear no one is maintaining this project anymore. > [...] Thanks for raising this, Sean. I reached out to the maintainers at Objectif Libre to check on their status. Maybe it's just a COVID19 + summer vacancy situation... Let's see what they say. -- Thierry Carrez (ttx) From thierry at openstack.org Mon Aug 3 10:15:06 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 3 Aug 2020 12:15:06 +0200 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> Message-ID: <88c24f3a-7d29-aa39-ed12-803279cc90c1@openstack.org> Ken Giusti wrote: > On Mon, Jul 27, 2020 at 1:18 PM Dan Smith > wrote: >> The primary concern was about something other than nova sitting on our >> bus making calls to our internal services. I imagine that the proposal >> to bake it into oslo.messaging is for the same purpose, and I'd probably >> have the same concern. At the time I think we agreed that if we were >> going to support direct-to-service health checks, they should be teensy >> HTTP servers with oslo healthchecks middleware. Further loading down >> rabbit with those pings doesn't seem like the best plan to >> me. Especially since Nova (compute) services already check in over RPC >> periodically and the success of that is discoverable en masse through >> the API. > > While initially in favor of this feature Dan's concern has me > reconsidering this. > > Now I believe that if the purpose of this feature is to check the > operational health of a service _using_ oslo.messaging, then I'm against > it.   A naked ping to a generic service point in an application doesn't > prove the operating health of that application beyond its connection to > rabbit. While I understand the need to further avoid loading down Rabbit, I like the universality of this solution, solving a real operational issue. Obviously that creates a trade-off (further loading rabbit to get more operational insights), but nobody forces you to run those ping calls, they would be opt-in. So the proposed code in itself does not weigh down Rabbit, or make anything sit on the bus. > Connectivity monitoring between an application and rabbit is > done using the keepalive connection heartbeat mechanism built into the > rabbit protocol, which O.M. supports today. I'll let Arnaud answer, but I suspect the operational need is code-external checking of the rabbit->agent chain, not code-internal checking of the agent->rabbit chain. The heartbeat mechanism is used by the agent to keep the Rabbit connection alive, ensuring it works in most of the cases. The check described above is to catch the corner cases where it still doesn't. -- Thierry Carrez (ttx) From sshnaidm at redhat.com Mon Aug 3 10:36:12 2020 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Mon, 3 Aug 2020 13:36:12 +0300 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> Message-ID: Hi, Bogdan thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter, lookup, inventory or any other plugin we already use? Action plugins are also used wide in Ansible itself, for example templates plugin is implemented with action plugin [2]. If Ansible can use it, why can't we? I don't think there is something with "fixing" Ansible, it's not a bug, this is a useful extension. What regards the mentioned action plugin for podman containers, it allows to spawn containers remotely while skipping the connection part for every cycle. I'm not sure you can "fix" Ansible not to do that, it's not a bug. We may not see the difference in a few hosts in CI, but it might be very efficient when we deploy on 100+ hosts oro even 1000+ hosts. In order to evaluate this on bigger setups to understand its value we configured both options - to use action plugin or usual module. If better performance of action plugin will be proven, we can switch to use it, if it doesn't make a difference on bigger setups - then I think we can easily switch back to using an usual module. Thanks [1] https://docs.ansible.com/ansible/latest/plugins/plugins.html [2] https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/template.py On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya wrote: > There is a trend of writing action plugins, see [0], for simple things, > like just calling a module in a loop. I'm not sure that is the direction > TripleO should go. If ansible is inefficient in this sort of tasks > without custom python code written, we should fix ansible. Otherwise, > what is the ultimate goal of that trend? Is that having only action > plugins in roles and playbooks? > > Please kindly asking the community to stop that, make a step back and > reiterate with the taken approach. Thank you. > > [0] https://review.opendev.org/716108 > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando > > > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From balazs.gibizer at est.tech Mon Aug 3 11:12:18 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 03 Aug 2020 13:12:18 +0200 Subject: [nova] If any spec freeze exception now? In-Reply-To: References: Message-ID: On Fri, Jul 31, 2020 at 17:23, Rambo wrote: > Hi,all: > I have a spec which is support volume backed server > rebuild[0].This spec was accepted in Stein, but some of the work did > not finish, so repropose it for Victoria.And this spec is depend on > the cinder reimage api [1], now the reimage api is almost all > completed. So I sincerely wish this spec will approved in Victoria. > If this spec is approved, I will achieve it at once. > I was +2 before on this spec. I see the value of it. I see that Lee, Sylvain and Sean had negative comments after my review. Also I saw that the spec was updated since. It would be good the get a re-review from those folks to see if their issues has been resolved. In general let's try to decide on the feature freeze exception for this on the weekly meeting. I hope folks will re-review the spec until Thursday. I saw you added it as a topic to the meeting agenda, thanks. (I moved that topic to the Open Discussion section) Cheers, gibi > > > Ref: > > [0]:https://blueprints.launchpad.net/nova/+spec/volume-backed-server-rebuild > > [1]:https://blueprints.launchpad.net/cinder/+spec/add-volume-re-image-api > > Best Regards > Rambo From balazs.gibizer at est.tech Mon Aug 3 11:16:26 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Mon, 03 Aug 2020 13:16:26 +0200 Subject: [nova] Victoria Milestone 2 Spec Freeze Message-ID: Hi, Last Thursday we reached Milestone 2 which means Nova is in Spec Freeze now. If you have a spec close to be approved and you wish to request a spec freeze exception then please send a mail to the ML about it. We will make the final decision on the weekly meeting on Thursday. Cheers, gibi From bdobreli at redhat.com Mon Aug 3 12:25:37 2020 From: bdobreli at redhat.com (Bogdan Dobrelya) Date: Mon, 3 Aug 2020 14:25:37 +0200 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> Message-ID: On 8/3/20 12:36 PM, Sagi Shnaidman wrote: > Hi, Bogdan > > thanks for raising this up, although I'm not sure I understand what it > is the problem with using action plugins. > Action plugins are well known official extensions for Ansible, as any > other plugins - callback, strategy, inventory etc [1]. It is not any > hack or unsupported workaround, it's a known and official feature of > Ansible. Why can't we use it? What makes it different from filter, I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy? > lookup, inventory or any other plugin we already use? > Action plugins are also used wide in Ansible itself, for example > templates plugin is implemented with action plugin [2]. If Ansible can > use it, why can't we? I don't think there is something with "fixing" > Ansible, it's not a bug, this is a useful extension. > What regards the mentioned action plugin for podman containers, it > allows to spawn containers remotely while skipping the connection part > for every cycle. I'm not sure you can "fix" Ansible not to do that, it's > not a bug. We may not see the difference in a few hosts in CI, but it > might be very efficient when we deploy on 100+ hosts oro even 1000+ > hosts. In order to evaluate this on bigger setups to understand its > value we configured both options - to use action plugin or usual module. > If better performance of action plugin will be proven, we can switch to > use it, if it doesn't make a difference on bigger setups - then I think > we can easily switch back to using an usual module. > > Thanks > > [1] https://docs.ansible.com/ansible/latest/plugins/plugins.html > [2] > https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/template.py > > On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya > wrote: > > There is a trend of writing action plugins, see [0], for simple things, > like just calling a module in a loop. I'm not sure that is the > direction > TripleO should go. If ansible is inefficient in this sort of tasks > without custom python code written, we should fix ansible. Otherwise, > what is the ultimate goal of that trend? Is that having only action > plugins in roles and playbooks? > > Please kindly asking the community to stop that, make a step back and > reiterate with the taken approach. Thank you. > > [0] https://review.opendev.org/716108 > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando > > > > > -- > Best regards > Sagi Shnaidman -- Best regards, Bogdan Dobrelya, Irc #bogdando From monika.samal at outlook.com Mon Aug 3 08:38:43 2020 From: monika.samal at outlook.com (Monika Samal) Date: Mon, 3 Aug 2020 08:38:43 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , Message-ID: Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal Cc: Fabian Zimmermann ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > Cc: Michael Johnson ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > Cc: Michael Johnson ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From monika.samal at outlook.com Mon Aug 3 08:53:17 2020 From: monika.samal at outlook.com (Monika Samal) Date: Mon, 3 Aug 2020 08:53:17 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , , Message-ID: After Michael suggestion I was able to create load balancer but there is error in status. [cid:de900175-3754-4942-a53d-43c78e425e62] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson Cc: Fabian Zimmermann ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal Cc: Fabian Zimmermann ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > Cc: Michael Johnson ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > Cc: Michael Johnson ; Amy Marrich ; openstack-discuss ; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26283 bytes Desc: image.png URL: From lyarwood at redhat.com Mon Aug 3 12:55:22 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Mon, 3 Aug 2020 13:55:22 +0100 Subject: [nova] openstack-tox-lower-constraints broken Message-ID: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> Hello all, $subject, I've raised the following bug: openstack-tox-lower-constraints failing due to unmet dependency on decorator==4.0.0 https://launchpad.net/bugs/1890123 I'm trying to resolve this below but I honestly feel like I'm going around in circles: https://review.opendev.org/#/q/topic:bug/1890123 If anyone has any tooling and/or recommendations for resolving issues like this I'd appreciate it! Cheers, -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From dev.faz at gmail.com Mon Aug 3 13:38:21 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Mon, 3 Aug 2020 15:38:21 +0200 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: Did you check the (nova) flavor you use in octavia. Fabian Monika Samal schrieb am Mo., 3. Aug. 2020, 10:53: > After Michael suggestion I was able to create load balancer but there is > error in status. > > > > PFB the error link: > > http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ > ------------------------------ > *From:* Monika Samal > *Sent:* Monday, August 3, 2020 2:08 PM > *To:* Michael Johnson > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Thanks a ton Michael for helping me out > ------------------------------ > *From:* Michael Johnson > *Sent:* Friday, July 31, 2020 3:57 AM > *To:* Monika Samal > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Just to close the loop on this, the octavia.conf file had > "project_name = admin" instead of "project_name = service" in the > [service_auth] section. This was causing the keystone errors when > Octavia was communicating with neutron. > > I don't know if that is a bug in kolla-ansible or was just a local > configuration issue. > > Michael > > On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > > > Hello Fabian,, > > > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > > > Regards, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > Hi, > > > > just to debug, could you replace the auth_type password with v3password? > > > > And do a curl against your :5000 and :35357 urls and paste the output. > > > > Fabian > > > > Monika Samal schrieb am Do., 30. Juli 2020, > 22:15: > > > > Hello Fabian, > > > > http://paste.openstack.org/show/796477/ > > > > Thanks, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > The sections should be > > > > service_auth > > keystone_authtoken > > > > if i read the docs correctly. Maybe you can just paste your config > (remove/change passwords) to paste.openstack.org and post the link? > > > > Fabian > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26283 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 26283 bytes Desc: not available URL: From sean.mcginnis at gmx.com Mon Aug 3 14:00:52 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 3 Aug 2020 09:00:52 -0500 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> Message-ID: On 8/3/20 7:55 AM, Lee Yarwood wrote: > Hello all, > > $subject, I've raised the following bug: > > openstack-tox-lower-constraints failing due to unmet dependency on decorator==4.0.0 > https://launchpad.net/bugs/1890123 > > I'm trying to resolve this below but I honestly feel like I'm going > around in circles: > > https://review.opendev.org/#/q/topic:bug/1890123 > > If anyone has any tooling and/or recommendations for resolving issues > like this I'd appreciate it! > > Cheers, This appears to be broken for everyone. I initially saw the decorator thing with Cinder, but after looking closer realized it's not that package. The root issue (or at least one level closer to the root issue, that seems to be causing the decorator failure) is that the lower-constraints are not actually being enforced. Even though the logs should it is passing "-c [path to lower-constraints.txt]". So even though things should be constrained to a lower version, presumably a version that works with a different version of decorator, pip is still installing a newer package than what the constraints should allow. There was a pip release on the 28th. Things don't look like they started failing until the 31st for us though, so either that is not it, or there was just a delay before our nodes started picking up the newer version. I tested locally, and at least with version 19.3.1, I am getting the correctly constrained packages installed. Still looking, but thought I would share in case that info triggers any ideas for anyone else. Sean From sean.mcginnis at gmx.com Mon Aug 3 14:12:47 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 3 Aug 2020 09:12:47 -0500 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> Message-ID: <6a92c9c8-9cc5-4b8e-4204-13545b40e5a2@gmx.com> > The root issue (or at least one level closer to the root issue, that > seems to be causing the decorator failure) is that the lower-constraints > are not actually being enforced. Even though the logs should it is > passing "-c [path to lower-constraints.txt]". So even though things > should be constrained to a lower version, presumably a version that > works with a different version of decorator, pip is still installing a > newer package than what the constraints should allow. > > There was a pip release on the 28th. Things don't look like they started > failing until the 31st for us though, so either that is not it, or there > was just a delay before our nodes started picking up the newer version. > > I tested locally, and at least with version 19.3.1, I am getting the > correctly constrained packages installed. > > Still looking, but thought I would share in case that info triggers any > ideas for anyone else. > I upgraded my pip and rebuilt the venv. The new pip has some good warnings emitted about some incompatible conflicts, so that part is good, but it did not change the package installation behavior. I am still able to get the correctly constrained packages installed locally on Fedora 32. So at least so far, it doesn't appear to be a pip issue. From mnaser at vexxhost.com Mon Aug 3 14:21:33 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Mon, 3 Aug 2020 10:21:33 -0400 Subject: [largescale-sig] RPC ping In-Reply-To: <20200727095744.GK31915@sync> References: <20200727095744.GK31915@sync> Message-ID: I have a few operational suggestions on how I think we could do this best: 1. I think exposing a healthcheck endpoint that _actually_ runs the ping and responds with a 200 OK makes a lot more sense in terms of being able to run it inside something like Kubernetes, you end up with a "who makes the ping and who responds to it" type of scenario which can be tricky though I'm sure we can figure that out 2. I've found that newer releases of RabbitMQ really help with those un-usable queues after a split, I haven't had any issues at all with newer releases, so that could be something to help your life be a lot easier. 3. You mentioned you're moving towards Kubernetes, we're doing the same and building an operator: https://opendev.org/vexxhost/openstack-operator -- Because the operator manages the whole thing and Kubernetes does it's thing too, we started moving towards 1 (single) rabbitmq per service, which reaaaaaaally helped a lot in stabilizing things. Oslo messaging is a lot better at recovering when a single service IP is pointing towards it because it doesn't do weird things like have threads trying to connect to other Rabbit ports. Just a thought. 4. In terms of telemetry and making sure you avoid that issue, we track the consumption rates of queues inside OpenStack. OpenStack consumption rate should be constant and never growing, anytime it grows, we instantly detect that something is fishy. However, the other issue comes in that when you restart any openstack service, it 'forgets' all it's existing queues and then you have a set of building up queues until they automatically expire which happens around 30 minutes-ish, so it makes that alarm of "things are not being consumed" a little noisy if you're restarting services Sorry for the wall of super unorganized text, all over the place here but thought I'd chime in with my 2 cents :) On Mon, Jul 27, 2020 at 6:04 AM Arnaud Morin wrote: > > Hey all, > > TLDR: I propose a change to oslo_messaging to allow doing a ping over RPC, > this is useful to monitor liveness of agents. > > > Few weeks ago, I proposed a patch to oslo_messaging [1], which is adding a > ping endpoint to RPC dispatcher. > It means that every openstack service which is using oslo_messaging RPC > endpoints (almosts all OpenStack services and agents - e.g. neutron > server + agents, nova + computes, etc.) will then be able to answer to a > specific "ping" call over RPC. > > I decided to propose this patch in my company mainly for 2 reasons: > 1 - we are struggling monitoring our nova compute and neutron agents in a > correct way: > > 1.1 - sometimes our agents are disconnected from RPC, but the python process > is still running. > 1.2 - sometimes the agent is still connected, but the queue / binding on > rabbit cluster is not working anymore (after a rabbit split for > example). This one is very hard to debug, because the agent is still > reporting health correctly on neutron server, but it's not able to > receive messages anymore. > > > 2 - we are trying to monitor agents running in k8s pods: > when running a python agent (neutron l3-agent for example) in a k8s pod, we > wanted to find a way to monitor if it is still live of not. > > > Adding a RPC ping endpoint could help us solve both these issues. > Note that we still need an external mechanism (out of OpenStack) to do this > ping. > We also think it could be nice for other OpenStackers, and especially > large scale ops. > > Feel free to comment. > > > [1] https://review.opendev.org/#/c/735385/ > > > -- > Arnaud Morin > > -- Mohammed Naser VEXXHOST, Inc. From iurygregory at gmail.com Mon Aug 3 14:42:38 2020 From: iurygregory at gmail.com (Iury Gregory) Date: Mon, 3 Aug 2020 16:42:38 +0200 Subject: [ironic] let's talk about grenade In-Reply-To: References: Message-ID: Hello Everyone, We will meet this Thursday (August 6th) at 2pm - 3pm UTC Time on bluejeans [1]. Thank you! [1] https://bluejeans.com/imelofer Em qua., 29 de jul. de 2020 às 20:37, Iury Gregory escreveu: > Hello everyone, > > Since we didn't get many responses I will keep the doodle open till Friday > =) > > Em seg., 27 de jul. de 2020 às 17:55, Iury Gregory > escreveu: > >> Hello everyone, >> >> I'm still on the fight to move our ironic-grenade-dsvm-multinode-multitenant >> to zuulv3 [1], you can find some of my findings on the etherpad [2] under `Move >> to Zuul v3 Jobs (Iurygregory)`. >> >> If you are interested in helping out we are going to schedule a meeting >> to discuss about this, please use the doodle in [3]. I will close the >> doodle on Wed July 29. >> >> Thanks! >> >> [1] https://review.opendev.org/705030 >> [2] https://etherpad.openstack.org/p/IronicWhiteBoard >> [3] https://doodle.com/poll/m69b5zwnsbgcysct >> >> -- >> >> >> *Att[]'sIury Gregory Melo Ferreira * >> *MSc in Computer Science at UFCG* >> *Part of the puppet-manager-core team in OpenStack* >> *Software Engineer at Red Hat Czech* >> *Social*: https://www.linkedin.com/in/iurygregory >> *E-mail: iurygregory at gmail.com * >> > > > -- > > > *Att[]'sIury Gregory Melo Ferreira * > *MSc in Computer Science at UFCG* > *Part of the puppet-manager-core team in OpenStack* > *Software Engineer at Red Hat Czech* > *Social*: https://www.linkedin.com/in/iurygregory > *E-mail: iurygregory at gmail.com * > -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From dev.faz at gmail.com Mon Aug 3 14:46:57 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Mon, 3 Aug 2020 16:46:57 +0200 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal schrieb am Mo., 3. Aug. 2020, 15:46: > It's registered > > Get Outlook for Android > ------------------------------ > *From:* Fabian Zimmermann > *Sent:* Monday, August 3, 2020 7:08:21 PM > *To:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Did you check the (nova) flavor you use in octavia. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 10:53: > > After Michael suggestion I was able to create load balancer but there is > error in status. > > > > PFB the error link: > > http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ > ------------------------------ > *From:* Monika Samal > *Sent:* Monday, August 3, 2020 2:08 PM > *To:* Michael Johnson > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Thanks a ton Michael for helping me out > ------------------------------ > *From:* Michael Johnson > *Sent:* Friday, July 31, 2020 3:57 AM > *To:* Monika Samal > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Just to close the loop on this, the octavia.conf file had > "project_name = admin" instead of "project_name = service" in the > [service_auth] section. This was causing the keystone errors when > Octavia was communicating with neutron. > > I don't know if that is a bug in kolla-ansible or was just a local > configuration issue. > > Michael > > On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > > > Hello Fabian,, > > > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > > > Regards, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > Hi, > > > > just to debug, could you replace the auth_type password with v3password? > > > > And do a curl against your :5000 and :35357 urls and paste the output. > > > > Fabian > > > > Monika Samal schrieb am Do., 30. Juli 2020, > 22:15: > > > > Hello Fabian, > > > > http://paste.openstack.org/show/796477/ > > > > Thanks, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > The sections should be > > > > service_auth > > keystone_authtoken > > > > if i read the docs correctly. Maybe you can just paste your config > (remove/change passwords) to paste.openstack.org and post the link? > > > > Fabian > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Mon Aug 3 15:31:52 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 3 Aug 2020 10:31:52 -0500 Subject: [release] Release countdown for week R-10 August 3 - 7 Message-ID: <20200803153152.GA3471444@sm-workstation> Development Focus ----------------- We are now past the Victoria-2 milestone, and entering the last development phase of the cycle. Teams should be focused on implementing planned work for the cycle. Now is a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- Looking ahead to the end of the release cycle, please be aware of the feature freeze dates. Those vary depending on deliverable type: * General libraries (except client libraries) need to have their last feature release before Non-client library freeze (September 3). Their stable branches are cut early. * Client libraries (think python-*client libraries) need to have their last feature release before Client library freeze (September 10) * Deliverables following a cycle-with-rc model (that would be most services) observe a Feature freeze on that same date, September 10. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. After feature freeze, cycle-with-rc deliverables need to produce a first release candidate (and a stable branch) before RC1 deadline (September 24) * Deliverables following cycle-with-intermediary model can release as necessary, but in all cases before Final RC deadline (October 8) Upcoming Deadlines & Dates -------------------------- Ussuri Cycle-trailing deadline: August 13 (R-9 week) Non-client library freeze: September 3 (R-6 week) Client library freeze: September 10 (R-5 week) Ussuri-3 milestone: September 10 (R-5 week) Victoria release: October 14 From johnsomor at gmail.com Mon Aug 3 15:40:40 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Mon, 3 Aug 2020 08:40:40 -0700 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann wrote: > Seems like the flavor is missing or empty '' - check for typos and enable > debug. > > Check if the nova req contains valid information/flavor. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 15:46: > >> It's registered >> >> Get Outlook for Android >> ------------------------------ >> *From:* Fabian Zimmermann >> *Sent:* Monday, August 3, 2020 7:08:21 PM >> *To:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Did you check the (nova) flavor you use in octavia. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 10:53: >> >> After Michael suggestion I was able to create load balancer but there is >> error in status. >> >> >> >> PFB the error link: >> >> http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Monday, August 3, 2020 2:08 PM >> *To:* Michael Johnson >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Thanks a ton Michael for helping me out >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Friday, July 31, 2020 3:57 AM >> *To:* Monika Samal >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Just to close the loop on this, the octavia.conf file had >> "project_name = admin" instead of "project_name = service" in the >> [service_auth] section. This was causing the keystone errors when >> Octavia was communicating with neutron. >> >> I don't know if that is a bug in kolla-ansible or was just a local >> configuration issue. >> >> Michael >> >> On Thu, Jul 30, 2020 at 1:39 PM Monika Samal >> wrote: >> > >> > Hello Fabian,, >> > >> > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ >> > >> > Regards, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:57 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > Hi, >> > >> > just to debug, could you replace the auth_type password with v3password? >> > >> > And do a curl against your :5000 and :35357 urls and paste the output. >> > >> > Fabian >> > >> > Monika Samal schrieb am Do., 30. Juli 2020, >> 22:15: >> > >> > Hello Fabian, >> > >> > http://paste.openstack.org/show/796477/ >> > >> > Thanks, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:38 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > The sections should be >> > >> > service_auth >> > keystone_authtoken >> > >> > if i read the docs correctly. Maybe you can just paste your config >> (remove/change passwords) to paste.openstack.org and post the link? >> > >> > Fabian >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ashlee at openstack.org Mon Aug 3 18:12:07 2020 From: ashlee at openstack.org (Ashlee Ferguson) Date: Mon, 3 Aug 2020 13:12:07 -0500 Subject: CFP Deadline Tomorrow - Virtual Open Infrastructure Summit Message-ID: <3353421C-E038-47F5-B68F-3828232639EB@openstack.org> Hi everyone, It’s time to submit your Open Infrastructure virtual Summit presentations[1]! The CFP deadline is tomorrow. Submit sessions featuring open source projects including Airship, Ansible, Ceph, Kata Containers, Kubernetes, ONAP, OpenStack, OPNFV, StarlingX and Zuul. As a reminder, these are the 2020 Tracks: 5G, NFV & Edge AI, Machine Learning & HPC CI/CD Container Infrastructure Getting Started Hands-on Workshops Open Development Private & Hybrid Cloud Public Cloud Security Get your presentations, panels, and workshops in before August 4 at 11:59 pm PT (August 5 at 6:59 am UTC). The content submission process for the Forum and Project Teams Gathering (PTG) will be managed separately in the upcoming months. The Summit Programming Committee has shared topics by Track for community members interested in speaking at the upcoming Summit. Check out the submission tips[2]! Then don’t forget to register[3] for the virtual Open Infrastructure Summit taking place October 19-23, 2020 at no cost to you. Need more time? Reach out to speakersupport at openstack.org with any questions or concerns. Cheers, Ashlee [1] https://cfp.openstack.org/ [2] https://superuser.openstack.org/articles/virtual-open-infrastructure-summit-cfp/ [3] https://openinfrasummit2020.eventbrite.com Ashlee Ferguson Community & Events Coordinator OpenStack Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimo.sgaravatto at gmail.com Mon Aug 3 18:14:21 2020 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Mon, 3 Aug 2020 20:14:21 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." Message-ID: We have just updated a small OpenStack cluster to Train. Everything seems working, but "cinder-status upgrade check" complains that services and volumes must have a service UUID [*]. What does this exactly mean? Thanks, Massimo [*] +--------------------------------------------------------------------+ | Check: Service UUIDs | | Result: Failure | | Details: Services and volumes must have a service UUID. Please fix | | this issue by running Queens online data migrations. | -------------- next part -------------- An HTML attachment was scrubbed... URL: From victoria at vmartinezdelacruz.com Mon Aug 3 19:12:06 2020 From: victoria at vmartinezdelacruz.com (=?UTF-8?Q?Victoria_Mart=C3=ADnez_de_la_Cruz?=) Date: Mon, 3 Aug 2020 16:12:06 -0300 Subject: [manila] Doc-a-thon event coming up next Thursday (Aug 6th) In-Reply-To: References: Message-ID: Hi everybody, An update on this. We decided to take over the upstream meeting directly and start *at* the slot of the Manila weekly meeting. We will join the Jitsi bridge [0] at 3pm UTC time and start going through the list of bugs we have in [1]. There is no finish time, you can join and leave the bridge freely. We will also use IRC Freenode channel #openstack-manila if needed. If the time slot doesn't work for you (we are aware this is not a friendly slot for EMEA/APAC), you can still go through the bug list in [1], claim a bug and work on it. If things go well, we plan to do this again in a different slot so everybody that wants to collaborate can do it. Looking forward to see you there, Cheers, V [0] https://meetpad.opendev.org/ManilaV-ReleaseDocAThon [1] https://ethercalc.openstack.org/ur17jprbprxx On Fri, Jul 31, 2020 at 2:05 PM Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> wrote: > Hi folks, > > We will be organizing a doc-a-thon next Thursday, August 6th, with the > main goal of improving our docs for the next release. We will be gathering > on our Freenode channel #openstack-manila after our weekly meeting (3pm > UTC) and also using a videoconference tool (exact details TBC) to go over a > curated list of opened doc bugs we have here [0]. > > *Your* participation is truly valued, being you an already Manila > contributor or if you are interested in contributing and you didn't know > how, so looking forward to seeing you there :) > > Cheers, > > Victoria > > [0] https://ethercalc.openstack.org/ur17jprbprxx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gagehugo at gmail.com Mon Aug 3 19:19:07 2020 From: gagehugo at gmail.com (Gage Hugo) Date: Mon, 3 Aug 2020 14:19:07 -0500 Subject: [openstack-helm] IRC Meeting Canceled 08/04 Message-ID: Hello everyone, Since I will be unavailable tomorrow and there's currently no agenda, I am going to cancel the meeting for tomorrow. We will meet again next week at the regular time. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From victoria at vmartinezdelacruz.com Mon Aug 3 19:21:16 2020 From: victoria at vmartinezdelacruz.com (=?UTF-8?Q?Victoria_Mart=C3=ADnez_de_la_Cruz?=) Date: Mon, 3 Aug 2020 16:21:16 -0300 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: Hi Ignazio, How did you deploy Manila and Manila UI? Can you point me toward the docs you used? Also, which is the specific workflow you are following to reach that trace? Just opening the dashboard and clicking on the Shares tab? Cheers, V On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano wrote: > Hello, I installed manila on openstack stein and it works by command line > mat the manila ui does not work and in httpd error log I read: > > [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR > django.request Internal Server Error: /dashboard/project/shares/ > [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback (most > recent call last): > [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line > 41, in inner > [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response = > get_response(request) > [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, > in _get_response > [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response = > self.process_exception_by_middleware(e, request) > [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, > in _get_response > [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response = > wrapped_callback(request, *callback_args, **callback_kwargs) > [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec > [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return > view_func(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec > [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return > view_func(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec > [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return > view_func(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec > [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return > view_func(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec > [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return > view_func(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, > in view > [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return > self.dispatch(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, > in dispatch > [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return > handler(request, *args, **kwargs) > [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get > [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = > self.construct_tables() > [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in > construct_tables > [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = > self.handle_table(table) > [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in > handle_table > [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = > self._get_data_dict() > [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in > _get_data_dict > [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] > data.extend(func()) > [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in > wrapped > [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = > cache[key] = func(*args, **kwargs) > [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", > line 57, in get_shares_data > [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] share_nets = > manila.share_network_list(self.request) > [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File > "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in > share_network_list > [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return > manilaclient(request).share_networks.list(detailed=detailed, > [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] AttributeError: > 'NoneType' object has no attribute 'share_networks' > > Please, anyone could help ? > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz at redhat.com Mon Aug 3 19:28:51 2020 From: aschultz at redhat.com (Alex Schultz) Date: Mon, 3 Aug 2020 13:28:51 -0600 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> Message-ID: On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya wrote: > > On 8/3/20 12:36 PM, Sagi Shnaidman wrote: > > Hi, Bogdan > > > > thanks for raising this up, although I'm not sure I understand what it > > is the problem with using action plugins. > > Action plugins are well known official extensions for Ansible, as any > > other plugins - callback, strategy, inventory etc [1]. It is not any > > hack or unsupported workaround, it's a known and official feature of > > Ansible. Why can't we use it? What makes it different from filter, > > I believe the cases that require the use of those should be justified. > For the given example, that manages containers in a loop via calling a > module, what the written custom callback plugin buys for us? That brings > code to maintain, extra complexity, like handling possible corner cases > in async mode, dry-run mode etc. But what is justification aside of > looks handy? I disagree that we shouldn't use action plugins or modules. Tasks themselves are expensive at scale. We saw that when we switched away from paunch to container management in pure ansible tasks. This exposed that looping tasks are even more expensive and complex error handling and workflows are better suited for modules or action plugins than a series of tasks. This is not something to be "fixed in ansible". This is the nature of the executor and strategy related interactions. Should everything be converted to modules and plugins? no. Should everything be tasks only? no. It's a balance that must be struck between when a specific set of complex tasks need extra data processing or error handling. Switching to modules or action plugins allows us to unit test our logic. Using tasks do not have such a concept outside of writing complex molecule testing. IMHO it's safer to switch to modules/action plugins than writing task logic. IMHO the issue that I see with the switch to Action plugins is the increased load on the ansible "controller" node during execution. Modules may be better depending on the task being managed. But I believe with unit testing, action plugins or modules provide a cleaner and more testable solution than writing roles consisting only of tasks. > > > lookup, inventory or any other plugin we already use? > > Action plugins are also used wide in Ansible itself, for example > > templates plugin is implemented with action plugin [2]. If Ansible can > > use it, why can't we? I don't think there is something with "fixing" > > Ansible, it's not a bug, this is a useful extension. > > What regards the mentioned action plugin for podman containers, it > > allows to spawn containers remotely while skipping the connection part > > for every cycle. I'm not sure you can "fix" Ansible not to do that, it's > > not a bug. We may not see the difference in a few hosts in CI, but it > > might be very efficient when we deploy on 100+ hosts oro even 1000+ > > hosts. In order to evaluate this on bigger setups to understand its > > value we configured both options - to use action plugin or usual module. > > If better performance of action plugin will be proven, we can switch to > > use it, if it doesn't make a difference on bigger setups - then I think > > we can easily switch back to using an usual module. > > > > Thanks > > > > [1] https://docs.ansible.com/ansible/latest/plugins/plugins.html > > [2] > > https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/template.py > > > > On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya > > wrote: > > > > There is a trend of writing action plugins, see [0], for simple things, > > like just calling a module in a loop. I'm not sure that is the > > direction > > TripleO should go. If ansible is inefficient in this sort of tasks > > without custom python code written, we should fix ansible. Otherwise, > > what is the ultimate goal of that trend? Is that having only action > > plugins in roles and playbooks? > > > > Please kindly asking the community to stop that, make a step back and > > reiterate with the taken approach. Thank you. > > > > [0] https://review.opendev.org/716108 > > > > > > -- > > Best regards, > > Bogdan Dobrelya, > > Irc #bogdando > > > > > > > > > > -- > > Best regards > > Sagi Shnaidman > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando > > From ignaziocassano at gmail.com Mon Aug 3 19:32:25 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 3 Aug 2020 21:32:25 +0200 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: Hello Victoria, I installed manila with yum on centos 7. Yes, I open the dashboard and I click on shares tab. I think the problem is I not using share networks because I am using netapp drivers without share management option. Looking at the code the dashboard check if there are shares under shared networks. My understading is that shared networks should be created only when shared management option is true. Ignazio Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> ha scritto: > Hi Ignazio, > > How did you deploy Manila and Manila UI? Can you point me toward the docs > you used? > > Also, which is the specific workflow you are following to reach that > trace? Just opening the dashboard and clicking on the Shares tab? > > Cheers, > > V > > On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano > wrote: > >> Hello, I installed manila on openstack stein and it works by command line >> mat the manila ui does not work and in httpd error log I read: >> >> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >> django.request Internal Server Error: /dashboard/project/shares/ >> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback (most >> recent call last): >> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >> 41, in inner >> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response = >> get_response(request) >> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >> in _get_response >> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response = >> self.process_exception_by_middleware(e, request) >> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >> in _get_response >> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response = >> wrapped_callback(request, *callback_args, **callback_kwargs) >> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >> in view >> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >> self.dispatch(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >> in dispatch >> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >> handler(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = >> self.construct_tables() >> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >> construct_tables >> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = >> self.handle_table(table) >> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >> handle_table >> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >> self._get_data_dict() >> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >> _get_data_dict >> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >> data.extend(func()) >> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >> wrapped >> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >> cache[key] = func(*args, **kwargs) >> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >> line 57, in get_shares_data >> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] share_nets = >> manila.share_network_list(self.request) >> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >> share_network_list >> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >> manilaclient(request).share_networks.list(detailed=detailed, >> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] AttributeError: >> 'NoneType' object has no attribute 'share_networks' >> >> Please, anyone could help ? >> Ignazio >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Aug 3 19:34:34 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 3 Aug 2020 21:34:34 +0200 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: PS I followed installation guide under docs.openstack.org. Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> ha scritto: > Hi Ignazio, > > How did you deploy Manila and Manila UI? Can you point me toward the docs > you used? > > Also, which is the specific workflow you are following to reach that > trace? Just opening the dashboard and clicking on the Shares tab? > > Cheers, > > V > > On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano > wrote: > >> Hello, I installed manila on openstack stein and it works by command line >> mat the manila ui does not work and in httpd error log I read: >> >> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >> django.request Internal Server Error: /dashboard/project/shares/ >> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback (most >> recent call last): >> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >> 41, in inner >> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response = >> get_response(request) >> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >> in _get_response >> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response = >> self.process_exception_by_middleware(e, request) >> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >> in _get_response >> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response = >> wrapped_callback(request, *callback_args, **callback_kwargs) >> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >> view_func(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >> in view >> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >> self.dispatch(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >> in dispatch >> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >> handler(request, *args, **kwargs) >> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = >> self.construct_tables() >> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >> construct_tables >> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = >> self.handle_table(table) >> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >> handle_table >> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >> self._get_data_dict() >> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >> _get_data_dict >> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >> data.extend(func()) >> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >> wrapped >> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >> cache[key] = func(*args, **kwargs) >> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >> line 57, in get_shares_data >> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] share_nets = >> manila.share_network_list(self.request) >> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >> share_network_list >> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >> manilaclient(request).share_networks.list(detailed=detailed, >> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] AttributeError: >> 'NoneType' object has no attribute 'share_networks' >> >> Please, anyone could help ? >> Ignazio >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Aug 3 19:41:34 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 3 Aug 2020 21:41:34 +0200 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: PS ps Sorry If aI am writing again. The command: manila list let me to show shares I created with command line. The dashboard gives errors I reported in my first email. Looking at manila.py line 280 it checks shares under share networks. Ignazio Il Lun 3 Ago 2020, 21:34 Ignazio Cassano ha scritto: > PS > I followed installation guide under docs.openstack.org. > > > Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < > victoria at vmartinezdelacruz.com> ha scritto: > >> Hi Ignazio, >> >> How did you deploy Manila and Manila UI? Can you point me toward the docs >> you used? >> >> Also, which is the specific workflow you are following to reach that >> trace? Just opening the dashboard and clicking on the Shares tab? >> >> Cheers, >> >> V >> >> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano >> wrote: >> >>> Hello, I installed manila on openstack stein and it works by command >>> line mat the manila ui does not work and in httpd error log I read: >>> >>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>> django.request Internal Server Error: /dashboard/project/shares/ >>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback (most >>> recent call last): >>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>> 41, in inner >>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response = >>> get_response(request) >>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>> in _get_response >>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response = >>> self.process_exception_by_middleware(e, request) >>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>> in _get_response >>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response = >>> wrapped_callback(request, *callback_args, **callback_kwargs) >>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >>> view_func(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >>> view_func(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >>> view_func(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >>> view_func(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >>> view_func(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>> in view >>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >>> self.dispatch(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>> in dispatch >>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >>> handler(request, *args, **kwargs) >>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = >>> self.construct_tables() >>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>> construct_tables >>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = >>> self.handle_table(table) >>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>> handle_table >>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >>> self._get_data_dict() >>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>> _get_data_dict >>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>> data.extend(func()) >>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>> wrapped >>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >>> cache[key] = func(*args, **kwargs) >>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>> line 57, in get_shares_data >>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] share_nets >>> = manila.share_network_list(self.request) >>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>> share_network_list >>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >>> manilaclient(request).share_networks.list(detailed=detailed, >>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] AttributeError: >>> 'NoneType' object has no attribute 'share_networks' >>> >>> Please, anyone could help ? >>> Ignazio >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Mon Aug 3 20:05:25 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 3 Aug 2020 13:05:25 -0700 Subject: [ironic][stable] Include ironic-core in ironic-stable-maint ? Message-ID: Greetings awesome humans, I have a conundrum, and largely it is over stable branch maintenance. In essence, our stable branch approvers are largely down to Dmitry, Riccardo, and Myself. I think this needs to change and I'd like to propose that we go ahead and change ironic-stable-maint to just include ironic-core in order to prevent the bottleneck and conflict and risk which this presents. I strongly believe that our existing cores would all do the right thing if presented with the question of if a change needed to be merged. So honestly I'm not concerned by this proposal. Plus, some of our sub-projects have operated this way for quite some time. Thoughts, concerns, worries? -Julia From ignaziocassano at gmail.com Mon Aug 3 20:25:55 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 3 Aug 2020 22:25:55 +0200 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: I mean I am using dhss false Il Lun 3 Ago 2020, 21:41 Ignazio Cassano ha scritto: > PS ps > Sorry If aI am writing again. > The command: > manila list let me to show shares I created with command line. > The dashboard gives errors I reported in my first email. > Looking at manila.py line 280 it checks shares under share networks. > Ignazio > > > Il Lun 3 Ago 2020, 21:34 Ignazio Cassano ha > scritto: > >> PS >> I followed installation guide under docs.openstack.org. >> >> >> Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < >> victoria at vmartinezdelacruz.com> ha scritto: >> >>> Hi Ignazio, >>> >>> How did you deploy Manila and Manila UI? Can you point me toward the >>> docs you used? >>> >>> Also, which is the specific workflow you are following to reach that >>> trace? Just opening the dashboard and clicking on the Shares tab? >>> >>> Cheers, >>> >>> V >>> >>> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano >>> wrote: >>> >>>> Hello, I installed manila on openstack stein and it works by command >>>> line mat the manila ui does not work and in httpd error log I read: >>>> >>>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>>> django.request Internal Server Error: /dashboard/project/shares/ >>>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback >>>> (most recent call last): >>>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>>> 41, in inner >>>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response = >>>> get_response(request) >>>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>>> in _get_response >>>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response = >>>> self.process_exception_by_middleware(e, request) >>>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>>> in _get_response >>>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response = >>>> wrapped_callback(request, *callback_args, **callback_kwargs) >>>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >>>> view_func(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >>>> view_func(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >>>> view_func(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >>>> view_func(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >>>> view_func(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>>> in view >>>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >>>> self.dispatch(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>>> in dispatch >>>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >>>> handler(request, *args, **kwargs) >>>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = >>>> self.construct_tables() >>>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>>> construct_tables >>>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = >>>> self.handle_table(table) >>>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>>> handle_table >>>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >>>> self._get_data_dict() >>>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>>> _get_data_dict >>>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>>> data.extend(func()) >>>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>>> wrapped >>>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >>>> cache[key] = func(*args, **kwargs) >>>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>>> line 57, in get_shares_data >>>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] share_nets >>>> = manila.share_network_list(self.request) >>>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>>> share_network_list >>>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >>>> manilaclient(request).share_networks.list(detailed=detailed, >>>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] >>>> AttributeError: 'NoneType' object has no attribute 'share_networks' >>>> >>>> Please, anyone could help ? >>>> Ignazio >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gouthampravi at gmail.com Mon Aug 3 21:00:53 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Mon, 3 Aug 2020 14:00:53 -0700 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: On Mon, Aug 3, 2020 at 1:31 PM Ignazio Cassano wrote: > I mean I am using dhss false > > Il Lun 3 Ago 2020, 21:41 Ignazio Cassano ha > scritto: > >> PS ps >> Sorry If aI am writing again. >> The command: >> manila list let me to show shares I created with command line. >> The dashboard gives errors I reported in my first email. >> Looking at manila.py line 280 it checks shares under share networks. >> Ignazio >> >> >> Il Lun 3 Ago 2020, 21:34 Ignazio Cassano ha >> scritto: >> >>> PS >>> I followed installation guide under docs.openstack.org. >>> >>> >>> Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < >>> victoria at vmartinezdelacruz.com> ha scritto: >>> >>>> Hi Ignazio, >>>> >>>> How did you deploy Manila and Manila UI? Can you point me toward the >>>> docs you used? >>>> >>>> Also, which is the specific workflow you are following to reach that >>>> trace? Just opening the dashboard and clicking on the Shares tab? >>>> >>>> Cheers, >>>> >>>> V >>>> >>>> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano < >>>> ignaziocassano at gmail.com> wrote: >>>> >>>>> Hello, I installed manila on openstack stein and it works by command >>>>> line mat the manila ui does not work and in httpd error log I read: >>>>> >>>>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>>>> django.request Internal Server Error: /dashboard/project/shares/ >>>>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback >>>>> (most recent call last): >>>>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>>>> 41, in inner >>>>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response >>>>> = get_response(request) >>>>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>>>> in _get_response >>>>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response >>>>> = self.process_exception_by_middleware(e, request) >>>>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>>>> in _get_response >>>>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response >>>>> = wrapped_callback(request, *callback_args, **callback_kwargs) >>>>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>>>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >>>>> view_func(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >>>>> view_func(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >>>>> view_func(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>>>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >>>>> view_func(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>>>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >>>>> view_func(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>>>> in view >>>>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >>>>> self.dispatch(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>>>> in dispatch >>>>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >>>>> handler(request, *args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>>>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled = >>>>> self.construct_tables() >>>>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>>>> construct_tables >>>>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled = >>>>> self.handle_table(table) >>>>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>>>> handle_table >>>>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >>>>> self._get_data_dict() >>>>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>>>> _get_data_dict >>>>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>>>> data.extend(func()) >>>>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>>>> wrapped >>>>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >>>>> cache[key] = func(*args, **kwargs) >>>>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>>>> line 57, in get_shares_data >>>>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] >>>>> share_nets = manila.share_network_list(self.request) >>>>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>>>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>>>> share_network_list >>>>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] >>>>> AttributeError: 'NoneType' object has no attribute 'share_networks' >>>>> >>>> Looking at the error here, and the code - it could be that the UI isn't able to retrieve the manila service endpoint from the service catalog. If this is the case, you must be able to see a "DEBUG" level log in your httpd error log with "no share service configured". Do you see it? As the user you're using on horizon, can you perform "openstack catalog list" and check whether the "sharev2" service type exists in that list? > >>>>> Please, anyone could help ? >>>>> Ignazio >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Aug 3 21:45:55 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 3 Aug 2020 23:45:55 +0200 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: Hello Goutham,tomorrow I will check the catalog. Must I enable the debug option in dashboard local_setting or in manila.conf? Thanks Ignazio Il Lun 3 Ago 2020, 23:01 Goutham Pacha Ravi ha scritto: > > > > On Mon, Aug 3, 2020 at 1:31 PM Ignazio Cassano > wrote: > >> I mean I am using dhss false >> >> Il Lun 3 Ago 2020, 21:41 Ignazio Cassano ha >> scritto: >> >>> PS ps >>> Sorry If aI am writing again. >>> The command: >>> manila list let me to show shares I created with command line. >>> The dashboard gives errors I reported in my first email. >>> Looking at manila.py line 280 it checks shares under share networks. >>> Ignazio >>> >>> >>> Il Lun 3 Ago 2020, 21:34 Ignazio Cassano ha >>> scritto: >>> >>>> PS >>>> I followed installation guide under docs.openstack.org. >>>> >>>> >>>> Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < >>>> victoria at vmartinezdelacruz.com> ha scritto: >>>> >>>>> Hi Ignazio, >>>>> >>>>> How did you deploy Manila and Manila UI? Can you point me toward the >>>>> docs you used? >>>>> >>>>> Also, which is the specific workflow you are following to reach that >>>>> trace? Just opening the dashboard and clicking on the Shares tab? >>>>> >>>>> Cheers, >>>>> >>>>> V >>>>> >>>>> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano < >>>>> ignaziocassano at gmail.com> wrote: >>>>> >>>>>> Hello, I installed manila on openstack stein and it works by command >>>>>> line mat the manila ui does not work and in httpd error log I read: >>>>>> >>>>>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>>>>> django.request Internal Server Error: /dashboard/project/shares/ >>>>>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback >>>>>> (most recent call last): >>>>>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>>>>> 41, in inner >>>>>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] response >>>>>> = get_response(request) >>>>>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>>>>> in _get_response >>>>>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] response >>>>>> = self.process_exception_by_middleware(e, request) >>>>>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>>>>> in _get_response >>>>>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] response >>>>>> = wrapped_callback(request, *callback_args, **callback_kwargs) >>>>>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>>>>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >>>>>> view_func(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >>>>>> view_func(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >>>>>> view_func(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>>>>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >>>>>> view_func(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>>>>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >>>>>> view_func(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>>>>> in view >>>>>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >>>>>> self.dispatch(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>>>>> in dispatch >>>>>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >>>>>> handler(request, *args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>>>>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled >>>>>> = self.construct_tables() >>>>>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>>>>> construct_tables >>>>>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled >>>>>> = self.handle_table(table) >>>>>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>>>>> handle_table >>>>>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >>>>>> self._get_data_dict() >>>>>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>>>>> _get_data_dict >>>>>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>>>>> data.extend(func()) >>>>>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>>>>> wrapped >>>>>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >>>>>> cache[key] = func(*args, **kwargs) >>>>>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>>>>> line 57, in get_shares_data >>>>>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] >>>>>> share_nets = manila.share_network_list(self.request) >>>>>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>>>>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>>>>> share_network_list >>>>>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >>>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] >>>>>> AttributeError: 'NoneType' object has no attribute 'share_networks' >>>>>> >>>>> > Looking at the error here, and the code - it could be that the UI isn't > able to retrieve the manila service endpoint from the service catalog. If > this is the case, you must be able to see a "DEBUG" level log in your httpd > error log with "no share service configured". Do you see it? > > As the user you're using on horizon, can you perform "openstack catalog > list" and check whether the "sharev2" service type exists in that list? > > >> >>>>>> Please, anyone could help ? >>>>>> Ignazio >>>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From victoria at vmartinezdelacruz.com Tue Aug 4 00:53:09 2020 From: victoria at vmartinezdelacruz.com (=?UTF-8?Q?Victoria_Mart=C3=ADnez_de_la_Cruz?=) Date: Mon, 3 Aug 2020 21:53:09 -0300 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: In local_settings.py under openstack-dashboard. And then restart the webserver. Did you copy the enable and local files from manila-ui under Horizon's namespace? Check out https://docs.openstack.org/manila-ui/latest/install/installation.html We can continue debugging tomorrow, we will find out what is going on. Cheers, V On Mon, Aug 3, 2020, 6:46 PM Ignazio Cassano wrote: > Hello Goutham,tomorrow I will check the catalog. > Must I enable the debug option in dashboard local_setting or in > manila.conf? > Thanks > Ignazio > > > Il Lun 3 Ago 2020, 23:01 Goutham Pacha Ravi ha > scritto: > >> >> >> >> On Mon, Aug 3, 2020 at 1:31 PM Ignazio Cassano >> wrote: >> >>> I mean I am using dhss false >>> >>> Il Lun 3 Ago 2020, 21:41 Ignazio Cassano ha >>> scritto: >>> >>>> PS ps >>>> Sorry If aI am writing again. >>>> The command: >>>> manila list let me to show shares I created with command line. >>>> The dashboard gives errors I reported in my first email. >>>> Looking at manila.py line 280 it checks shares under share networks. >>>> Ignazio >>>> >>>> >>>> Il Lun 3 Ago 2020, 21:34 Ignazio Cassano ha >>>> scritto: >>>> >>>>> PS >>>>> I followed installation guide under docs.openstack.org. >>>>> >>>>> >>>>> Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < >>>>> victoria at vmartinezdelacruz.com> ha scritto: >>>>> >>>>>> Hi Ignazio, >>>>>> >>>>>> How did you deploy Manila and Manila UI? Can you point me toward the >>>>>> docs you used? >>>>>> >>>>>> Also, which is the specific workflow you are following to reach that >>>>>> trace? Just opening the dashboard and clicking on the Shares tab? >>>>>> >>>>>> Cheers, >>>>>> >>>>>> V >>>>>> >>>>>> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano < >>>>>> ignaziocassano at gmail.com> wrote: >>>>>> >>>>>>> Hello, I installed manila on openstack stein and it works by command >>>>>>> line mat the manila ui does not work and in httpd error log I read: >>>>>>> >>>>>>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>>>>>> django.request Internal Server Error: /dashboard/project/shares/ >>>>>>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback >>>>>>> (most recent call last): >>>>>>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>>>>>> 41, in inner >>>>>>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] >>>>>>> response = get_response(request) >>>>>>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>>>>>> in _get_response >>>>>>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] >>>>>>> response = self.process_exception_by_middleware(e, request) >>>>>>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>>>>>> in _get_response >>>>>>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] >>>>>>> response = wrapped_callback(request, *callback_args, **callback_kwargs) >>>>>>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>>>>>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >>>>>>> view_func(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >>>>>>> view_func(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >>>>>>> view_func(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>>>>>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >>>>>>> view_func(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>>>>>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >>>>>>> view_func(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>>>>>> in view >>>>>>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >>>>>>> self.dispatch(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>>>>>> in dispatch >>>>>>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >>>>>>> handler(request, *args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>>>>>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] handled >>>>>>> = self.construct_tables() >>>>>>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>>>>>> construct_tables >>>>>>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] handled >>>>>>> = self.handle_table(table) >>>>>>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>>>>>> handle_table >>>>>>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >>>>>>> self._get_data_dict() >>>>>>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>>>>>> _get_data_dict >>>>>>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>>>>>> data.extend(func()) >>>>>>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>>>>>> wrapped >>>>>>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value = >>>>>>> cache[key] = func(*args, **kwargs) >>>>>>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>>>>>> line 57, in get_shares_data >>>>>>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] >>>>>>> share_nets = manila.share_network_list(self.request) >>>>>>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>>>>>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>>>>>> share_network_list >>>>>>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >>>>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>>>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] >>>>>>> AttributeError: 'NoneType' object has no attribute 'share_networks' >>>>>>> >>>>>> >> Looking at the error here, and the code - it could be that the UI isn't >> able to retrieve the manila service endpoint from the service catalog. If >> this is the case, you must be able to see a "DEBUG" level log in your httpd >> error log with "no share service configured". Do you see it? >> >> As the user you're using on horizon, can you perform "openstack catalog >> list" and check whether the "sharev2" service type exists in that list? >> >> >>> >>>>>>> Please, anyone could help ? >>>>>>> Ignazio >>>>>>> >>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.king at gmail.com Mon Aug 3 21:58:53 2020 From: thomas.king at gmail.com (Thomas King) Date: Mon, 3 Aug 2020 15:58:53 -0600 Subject: [Openstack-mentoring] Neutron subnet with DHCP relay - continued In-Reply-To: References: Message-ID: I've been using named physical networks so long, I completely forgot using wildcards! Is this the answer???? https://docs.openstack.org/mitaka/config-reference/networking/networking_options_reference.html#modular-layer-2-ml2-flat-type-configuration-options Tom King On Tue, Jul 28, 2020 at 3:46 PM Thomas King wrote: > Ruslanas has been a tremendous help. To catch up the discussion lists... > 1. I enabled Neutron segments. > 2. I renamed the existing segments for each network so they'll make sense. > 3. I attempted to create a segment for a remote subnet (it is using DHCP > relay) and this was the error that is blocking me. This is where the docs > do not cover: > [root at sea-maas-controller ~(keystone_admin)]# openstack network segment > create --physical-network remote146-30-32 --network-type flat --network > baremetal seg-remote-146-30-32 > BadRequestException: 400: Client Error for url: > http://10.146.30.65:9696/v2.0/segments, Invalid input for operation: > physical_network 'remote146-30-32' unknown for flat provider network. > > I've asked Ruslanas to clarify how their physical networks correspond to > their remote networks. They have a single provider network and multiple > segments tied to multiple physical networks. > > However, if anyone can shine some light on this, I would greatly > appreciate it. How should neutron's configurations accommodate remote > networks<->Neutron segments when I have only one physical network > attachment for provisioning? > > Thanks! > Tom King > > On Wed, Jul 15, 2020 at 3:33 PM Thomas King wrote: > >> That helps a lot, thank you! >> >> "I use only one network..." >> This bit seems to go completely against the Neutron segments >> documentation. When you have access, please let me know if Triple-O is >> using segments or some other method. >> >> I greatly appreciate this, this is a tremendous help. >> >> Tom King >> >> On Wed, Jul 15, 2020 at 1:07 PM Ruslanas Gžibovskis >> wrote: >> >>> Hi Thomas, >>> >>> I have a bit complicated setup from tripleo side :) I use only one >>> network (only ControlPlane). thanks to Harold, he helped to make it work >>> for me. >>> >>> Yes, as written in the tripleo docs for leaf networks, it use the same >>> neutron network, different subnets. so neutron network is ctlplane (I >>> think) and have ctlplane-subnet, remote-provision and remote-KI :)) that >>> generates additional lines in "ip r s" output for routing "foreign" subnets >>> through correct gw, if you would have isolated networks, by vlans and ports >>> this would apply for each subnet different gw... I believe you >>> know/understand that part. >>> >>> remote* subnets have dhcp-relay setup by network team... do not ask >>> details for that. I do not know how to, but can ask :) >>> >>> >>> in undercloud/tripleo i have 2 dhcp servers, one is for introspection, >>> another for provide/cleanup and deployment process. >>> >>> all of those subnets have organization level tagged networks and are >>> tagged on network devices, but they are untagged on provisioning >>> interfaces/ports, as in general pxe should be untagged, but some nic's can >>> do vlan untag on nic/bios level. but who cares!? >>> >>> I just did a brief check on your first post, I think I have simmilar >>> setup to yours :)) I will check in around 12hours :)) more deaply, as will >>> be at work :))) >>> >>> >>> P.S. sorry for wrong terms, I am bad at naming. >>> >>> >>> On Wed, 15 Jul 2020, 21:13 Thomas King, wrote: >>> >>>> Ruslanas, that would be excellent! >>>> >>>> I will reply to you directly for details later unless the maillist >>>> would like the full thread. >>>> >>>> Some preliminary questions: >>>> >>>> - Do you have a separate physical interface for the segment(s) used >>>> for your remote subnets? >>>> The docs state each segment must have a unique physical network >>>> name, which suggests a separate physical interface for each segment unless >>>> I'm misunderstanding something. >>>> - Are your provisioning segments all on the same Neutron network? >>>> - Are you using tagged switchports or access switchports to your >>>> Ironic server(s)? >>>> >>>> Thanks, >>>> Tom King >>>> >>>> On Wed, Jul 15, 2020 at 12:26 AM Ruslanas Gžibovskis >>>> wrote: >>>> >>>>> I have deployed that with tripleO, but now we are recabling and >>>>> redeploying it. So once I have it running I can share my configs, just name >>>>> which you want :) >>>>> >>>>> On Tue, 14 Jul 2020 at 18:40, Thomas King >>>>> wrote: >>>>> >>>>>> I have. That's the Triple-O docs and they don't go through the normal >>>>>> .conf files to explain how it works outside of Triple-O. It has some ideas >>>>>> but no running configurations. >>>>>> >>>>>> Tom King >>>>>> >>>>>> On Tue, Jul 14, 2020 at 3:01 AM Ruslanas Gžibovskis >>>>>> wrote: >>>>>> >>>>>>> hi, have you checked: >>>>>>> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/routed_spine_leaf_network.html >>>>>>> ? >>>>>>> I am following this link. I only have one network, having different >>>>>>> issues tho ;) >>>>>>> >>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From akekane at redhat.com Tue Aug 4 04:37:11 2020 From: akekane at redhat.com (Abhishek Kekane) Date: Tue, 4 Aug 2020 10:07:11 +0530 Subject: =?UTF-8?Q?Re=3A_=5Blists=2Eopenstack=2Eorg=E4=BB=A3=E5=8F=91=5DRe=3A_=5BGlance=5D_Proposin?= =?UTF-8?Q?g_Dan_Smith_for_glance_core?= In-Reply-To: <03ece5d405c74b2d9292301c2e3be7b8@inspur.com> References: <8635120d-11d6-136e-2581-40d3d451d1aa@gmail.com> <03ece5d405c74b2d9292301c2e3be7b8@inspur.com> Message-ID: Hi All, After hearing only positive responses, I have added Dan to the Core members list. Welcome aboard Dan. Cheers, Abhishek On Mon, 3 Aug, 2020, 05:44 Brin Zhang(张百林), wrote: > +1 > > > > *发件人:* Jay Bryant [mailto:jungleboyj at gmail.com] > *发送时间:* 2020年7月31日 23:39 > *收件人:* openstack-discuss at lists.openstack.org > *主题:* [lists.openstack.org代发]Re: [Glance] Proposing Dan Smith for glance > core > > > > On 7/31/2020 8:10 AM, Sean McGinnis wrote: > > On 7/30/20 10:25 AM, Abhishek Kekane wrote: > > Hi All, > > I'd like to propose adding Dan Smith to the glance core group. > > > > Dan Smith has contributed to stabilize image import workflow as well as > multiple stores of glance. > > He is also contributing in tempest and nova to set up CI/tempest jobs > around image import and multiple stores. > > > > Being involved on the mailing-list and IRC channels, Dan is always helpful > to the community and here to help. > > Please respond with +1/-1 until 03rd August, 2020 1400 UTC. > > Cheers, > Abhishek > > +1 > > Not a Glance core but definitely +1 from me. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emiller at genesishosting.com Tue Aug 4 05:02:49 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Tue, 4 Aug 2020 00:02:49 -0500 Subject: [nova] Hyper-V hosts Message-ID: <046E9C0290DD9149B106B72FC9156BEA04814461@gmsxchsvr01.thecreation.com> Hi, I thought I'd look into support of Hyper-V hosts for Windows Server environments, but it looks like the latest cloudbase Windows Hyper-V OpenStack Installer is for Train, and nothing seems to discuss the use of Hyper-V in Windows Server 2019. Has it been abandoned? Is anyone using Hyper-V with OpenStack successfully? One of the reasons we thought we might support it is to provide nested support for VMs with GPUs and/or vGPUs, and thought this would work better than with KVM, specifically with AMD EPYC systems. It seems that when "options kvm-amd nested=1" is used in a modprobe.d config file, Windows machines lock up when started. I think this has been an issue for a while with AMD processors, but thought it was fixed recently (I don't remember where I saw this, though). Would love to hear about any experiences related to Hyper-V and/or nested hypervisor support on AMD EPYC processors. Thanks! Eric From ignaziocassano at gmail.com Tue Aug 4 05:49:48 2020 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 4 Aug 2020 07:49:48 +0200 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: Hello Victoria and Goutham, thank you for your great help. Unfortunately I made I mistake in my ansible playbook for installing manila: it created manila services more times, so some entries in the catalog did not have an endpoint associated. I removed the duplicated service entries where catalog was absent and now it works. Many thanks Ignazio Il giorno mar 4 ago 2020 alle ore 02:53 Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> ha scritto: > In local_settings.py under openstack-dashboard. And then restart the > webserver. > > Did you copy the enable and local files from manila-ui under Horizon's > namespace? Check out > https://docs.openstack.org/manila-ui/latest/install/installation.html > > We can continue debugging tomorrow, we will find out what is going on. > > Cheers, > > V > > > On Mon, Aug 3, 2020, 6:46 PM Ignazio Cassano > wrote: > >> Hello Goutham,tomorrow I will check the catalog. >> Must I enable the debug option in dashboard local_setting or in >> manila.conf? >> Thanks >> Ignazio >> >> >> Il Lun 3 Ago 2020, 23:01 Goutham Pacha Ravi ha >> scritto: >> >>> >>> >>> >>> On Mon, Aug 3, 2020 at 1:31 PM Ignazio Cassano >>> wrote: >>> >>>> I mean I am using dhss false >>>> >>>> Il Lun 3 Ago 2020, 21:41 Ignazio Cassano ha >>>> scritto: >>>> >>>>> PS ps >>>>> Sorry If aI am writing again. >>>>> The command: >>>>> manila list let me to show shares I created with command line. >>>>> The dashboard gives errors I reported in my first email. >>>>> Looking at manila.py line 280 it checks shares under share networks. >>>>> Ignazio >>>>> >>>>> >>>>> Il Lun 3 Ago 2020, 21:34 Ignazio Cassano >>>>> ha scritto: >>>>> >>>>>> PS >>>>>> I followed installation guide under docs.openstack.org. >>>>>> >>>>>> >>>>>> Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < >>>>>> victoria at vmartinezdelacruz.com> ha scritto: >>>>>> >>>>>>> Hi Ignazio, >>>>>>> >>>>>>> How did you deploy Manila and Manila UI? Can you point me toward the >>>>>>> docs you used? >>>>>>> >>>>>>> Also, which is the specific workflow you are following to reach that >>>>>>> trace? Just opening the dashboard and clicking on the Shares tab? >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> V >>>>>>> >>>>>>> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano < >>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>> >>>>>>>> Hello, I installed manila on openstack stein and it works by >>>>>>>> command line mat the manila ui does not work and in httpd error log I read: >>>>>>>> >>>>>>>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>>>>>>> django.request Internal Server Error: /dashboard/project/shares/ >>>>>>>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback >>>>>>>> (most recent call last): >>>>>>>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>>>>>>> 41, in inner >>>>>>>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] >>>>>>>> response = get_response(request) >>>>>>>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>>>>>>> in _get_response >>>>>>>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] >>>>>>>> response = self.process_exception_by_middleware(e, request) >>>>>>>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>>>>>>> in _get_response >>>>>>>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] >>>>>>>> response = wrapped_callback(request, *callback_args, **callback_kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>>>>>>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] return >>>>>>>> view_func(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>>>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] return >>>>>>>> view_func(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>>>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] return >>>>>>>> view_func(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>>>>>>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] return >>>>>>>> view_func(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>>>>>>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] return >>>>>>>> view_func(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>>>>>>> in view >>>>>>>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] return >>>>>>>> self.dispatch(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>>>>>>> in dispatch >>>>>>>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] return >>>>>>>> handler(request, *args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>>>>>>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] >>>>>>>> handled = self.construct_tables() >>>>>>>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>>>>>>> construct_tables >>>>>>>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] >>>>>>>> handled = self.handle_table(table) >>>>>>>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>>>>>>> handle_table >>>>>>>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data = >>>>>>>> self._get_data_dict() >>>>>>>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>>>>>>> _get_data_dict >>>>>>>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>>>>>>> data.extend(func()) >>>>>>>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>>>>>>> wrapped >>>>>>>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value >>>>>>>> = cache[key] = func(*args, **kwargs) >>>>>>>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>>>>>>> line 57, in get_shares_data >>>>>>>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] >>>>>>>> share_nets = manila.share_network_list(self.request) >>>>>>>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>>>>>>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>>>>>>> share_network_list >>>>>>>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] return >>>>>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>>>>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] >>>>>>>> AttributeError: 'NoneType' object has no attribute 'share_networks' >>>>>>>> >>>>>>> >>> Looking at the error here, and the code - it could be that the UI isn't >>> able to retrieve the manila service endpoint from the service catalog. If >>> this is the case, you must be able to see a "DEBUG" level log in your httpd >>> error log with "no share service configured". Do you see it? >>> >>> As the user you're using on horizon, can you perform "openstack catalog >>> list" and check whether the "sharev2" service type exists in that list? >>> >>> >>>> >>>>>>>> Please, anyone could help ? >>>>>>>> Ignazio >>>>>>>> >>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dev.faz at gmail.com Tue Aug 4 05:54:49 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Tue, 4 Aug 2020 07:54:49 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: Hi, i never had this issue, but did you run the post upgrade data migrations? Fabian Massimo Sgaravatto schrieb am Mo., 3. Aug. 2020, 20:21: > We have just updated a small OpenStack cluster to Train. > Everything seems working, but "cinder-status upgrade check" complains that > services and volumes must have a service UUID [*]. > What does this exactly mean? > > Thanks, Massimo > > [*] > +--------------------------------------------------------------------+ > | Check: Service UUIDs | > | Result: Failure | > | Details: Services and volumes must have a service UUID. Please fix | > | this issue by running Queens online data migrations. | > -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimo.sgaravatto at gmail.com Tue Aug 4 07:14:00 2020 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Tue, 4 Aug 2020 09:14:00 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: Do you mean "su -s /bin/sh -c "cinder-manage db sync" cinder" ? Yes: this was run Cheers, Massimo On Tue, Aug 4, 2020 at 7:54 AM Fabian Zimmermann wrote: > Hi, > > i never had this issue, but did you run the post upgrade data migrations? > > Fabian > > Massimo Sgaravatto schrieb am Mo., 3. Aug. > 2020, 20:21: > >> We have just updated a small OpenStack cluster to Train. >> Everything seems working, but "cinder-status upgrade check" complains >> that services and volumes must have a service UUID [*]. >> What does this exactly mean? >> >> Thanks, Massimo >> >> [*] >> +--------------------------------------------------------------------+ >> | Check: Service UUIDs | >> | Result: Failure | >> | Details: Services and volumes must have a service UUID. Please fix | >> | this issue by running Queens online data migrations. | >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dev.faz at gmail.com Tue Aug 4 07:20:09 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Tue, 4 Aug 2020 09:20:09 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: Hi, No i mean the "online data migrations" https://docs.openstack.org/cinder/rocky/upgrade.html Fabian Massimo Sgaravatto schrieb am Di., 4. Aug. 2020, 09:14: > Do you mean "su -s /bin/sh -c "cinder-manage db sync" cinder" ? > Yes: this was run > > Cheers, Massimo > > On Tue, Aug 4, 2020 at 7:54 AM Fabian Zimmermann > wrote: > >> Hi, >> >> i never had this issue, but did you run the post upgrade data migrations? >> >> Fabian >> >> Massimo Sgaravatto schrieb am Mo., 3. >> Aug. 2020, 20:21: >> >>> We have just updated a small OpenStack cluster to Train. >>> Everything seems working, but "cinder-status upgrade check" complains >>> that services and volumes must have a service UUID [*]. >>> What does this exactly mean? >>> >>> Thanks, Massimo >>> >>> [*] >>> +--------------------------------------------------------------------+ >>> | Check: Service UUIDs | >>> | Result: Failure | >>> | Details: Services and volumes must have a service UUID. Please fix | >>> | this issue by running Queens online data migrations. | >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimo.sgaravatto at gmail.com Tue Aug 4 07:46:35 2020 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Tue, 4 Aug 2020 09:46:35 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: Thanks. I tried but it says there is nothing to migrate: +-----------------------------------------+--------------+-----------+ | Migration | Total Needed | Completed | +-----------------------------------------+--------------+-----------+ | untyped_snapshots_online_data_migration | 0 | 0 | | untyped_volumes_online_data_migration | 0 | 0 | +-----------------------------------------+--------------+-----------+ On Tue, Aug 4, 2020 at 9:20 AM Fabian Zimmermann wrote: > Hi, > > No i mean the "online data migrations" > > https://docs.openstack.org/cinder/rocky/upgrade.html > > Fabian > > Massimo Sgaravatto schrieb am Di., 4. Aug. > 2020, 09:14: > >> Do you mean "su -s /bin/sh -c "cinder-manage db sync" cinder" ? >> Yes: this was run >> >> Cheers, Massimo >> >> On Tue, Aug 4, 2020 at 7:54 AM Fabian Zimmermann >> wrote: >> >>> Hi, >>> >>> i never had this issue, but did you run the post upgrade data migrations? >>> >>> Fabian >>> >>> Massimo Sgaravatto schrieb am Mo., 3. >>> Aug. 2020, 20:21: >>> >>>> We have just updated a small OpenStack cluster to Train. >>>> Everything seems working, but "cinder-status upgrade check" complains >>>> that services and volumes must have a service UUID [*]. >>>> What does this exactly mean? >>>> >>>> Thanks, Massimo >>>> >>>> [*] >>>> +--------------------------------------------------------------------+ >>>> | Check: Service UUIDs | >>>> | Result: Failure | >>>> | Details: Services and volumes must have a service UUID. Please fix | >>>> | this issue by running Queens online data migrations. | >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Tue Aug 4 08:08:19 2020 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 4 Aug 2020 09:08:19 +0100 Subject: [ironic][stable] Include ironic-core in ironic-stable-maint ? In-Reply-To: References: Message-ID: On Mon, 3 Aug 2020 at 21:06, Julia Kreger wrote: > > Greetings awesome humans, > > I have a conundrum, and largely it is over stable branch maintenance. > > In essence, our stable branch approvers are largely down to Dmitry, > Riccardo, and Myself. I think this needs to change and I'd like to > propose that we go ahead and change ironic-stable-maint to just > include ironic-core in order to prevent the bottleneck and conflict > and risk which this presents. > > I strongly believe that our existing cores would all do the right > thing if presented with the question of if a change needed to be > merged. So honestly I'm not concerned by this proposal. Plus, some of > our sub-projects have operated this way for quite some time. > > Thoughts, concerns, worries? > Makes sense to me. We operate this way in Kolla. It might be good to make sure that current cores are all aware of what 'the right thing' is, that it is written down, and that we include it in the core onboarding process. > -Julia > From mark at stackhpc.com Tue Aug 4 08:11:39 2020 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 4 Aug 2020 09:11:39 +0100 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: On Thu, 30 Jul 2020 at 14:43, Rafael Weingärtner wrote: > > We are working on it. So far we have 3 open proposals there, but we do not have enough karma to move things along. > Besides these 3 open proposals, we do have more ongoing extensions that have not yet been proposed to the community. It's good to hear you want to help improve cloudkitty, however it sounds like what is required is help with maintaining the project. Is that something you could be involved with? Mark > > On Thu, Jul 30, 2020 at 10:22 AM Sean McGinnis wrote: >> >> Posting here to raise awareness, and start discussion about next steps. >> >> It appears there is no one working on Cloudkitty anymore. No patches >> have been merged for several months now, including simple bot proposed >> patches. It would appear no one is maintaining this project anymore. >> >> I know there is a need out there for this type of functionality, so >> maybe this will raise awareness and get some attention to it. But >> barring that, I am wondering if we should start the process to retire >> this project. >> >> From a Victoria release perspective, it is milestone-2 week, so we >> should make a decision if any of the Cloudkitty deliverables should be >> included in this release or not. We can certainly force releases of >> whatever is the latest, but I think that is a bit risky since these >> repos have never merged the job template change for victoria and >> therefore are not even testing with Python 3.8. That is an official >> runtime for Victoria, so we run the risk of having issues with the code >> if someone runs under 3.8 but we have not tested to make sure there are >> no problems doing so. >> >> I am hoping this at least starts the discussion. I will not propose any >> release patches to remove anything until we have had a chance to discuss >> the situation. >> >> Sean >> >> > > > -- > Rafael Weingärtner From dev.faz at gmail.com Tue Aug 4 08:17:45 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Tue, 4 Aug 2020 10:17:45 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: Hmm, the err msg tells to run the queens version of the tool. Maybe something went wrong, but the db version got incremented? Just guessing. Did you try to find the commit/change that introduced the msg? Maybe it refers to the action required to fix it / or check the db online migrarions scripts what they would/should do. Fabian Massimo Sgaravatto schrieb am Di., 4. Aug. 2020, 09:46: > Thanks. > I tried but it says there is nothing to migrate: > > +-----------------------------------------+--------------+-----------+ > | Migration | Total Needed | Completed | > +-----------------------------------------+--------------+-----------+ > | untyped_snapshots_online_data_migration | 0 | 0 | > | untyped_volumes_online_data_migration | 0 | 0 | > +-----------------------------------------+--------------+-----------+ > > On Tue, Aug 4, 2020 at 9:20 AM Fabian Zimmermann > wrote: > >> Hi, >> >> No i mean the "online data migrations" >> >> https://docs.openstack.org/cinder/rocky/upgrade.html >> >> Fabian >> >> Massimo Sgaravatto schrieb am Di., 4. >> Aug. 2020, 09:14: >> >>> Do you mean "su -s /bin/sh -c "cinder-manage db sync" cinder" ? >>> Yes: this was run >>> >>> Cheers, Massimo >>> >>> On Tue, Aug 4, 2020 at 7:54 AM Fabian Zimmermann >>> wrote: >>> >>>> Hi, >>>> >>>> i never had this issue, but did you run the post upgrade data >>>> migrations? >>>> >>>> Fabian >>>> >>>> Massimo Sgaravatto schrieb am Mo., 3. >>>> Aug. 2020, 20:21: >>>> >>>>> We have just updated a small OpenStack cluster to Train. >>>>> Everything seems working, but "cinder-status upgrade check" complains >>>>> that services and volumes must have a service UUID [*]. >>>>> What does this exactly mean? >>>>> >>>>> Thanks, Massimo >>>>> >>>>> [*] >>>>> +--------------------------------------------------------------------+ >>>>> | Check: Service UUIDs | >>>>> | Result: Failure | >>>>> | Details: Services and volumes must have a service UUID. Please fix | >>>>> | this issue by running Queens online data migrations. | >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Tue Aug 4 08:54:39 2020 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 4 Aug 2020 10:54:39 +0200 Subject: [ironic][stable] Include ironic-core in ironic-stable-maint ? In-Reply-To: References: Message-ID: <01691703-aa8a-c24b-bc8e-7671a55d1d34@openstack.org> Julia Kreger wrote: > [...] > In essence, our stable branch approvers are largely down to Dmitry, > Riccardo, and Myself. I think this needs to change and I'd like to > propose that we go ahead and change ironic-stable-maint to just > include ironic-core in order to prevent the bottleneck and conflict > and risk which this presents. > > I strongly believe that our existing cores would all do the right > thing if presented with the question of if a change needed to be > merged. So honestly I'm not concerned by this proposal. Plus, some of > our sub-projects have operated this way for quite some time. > > Thoughts, concerns, worries? Sounds good to me. Stable branch backport approvals follow different rules from development branch changes, which is why historically we used separate groups -- so that all -core do not need to know the stable policy rules. But today -core groups evolve less quickly and can probably be taught the stable policy, so I'm not too concerned either. Maybe it's a good time to remind them of the stable policy doc though, in particular the "appropriate fixes" section: https://docs.openstack.org/project-team-guide/stable-branches.html Cheers, -- Thierry From jesse at odyssey4.me Tue Aug 4 09:23:30 2020 From: jesse at odyssey4.me (Jesse Pretorius) Date: Tue, 4 Aug 2020 09:23:30 +0000 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> Message-ID: <2231a9649ccef5f1c712ac509fb22cb199f6b88b.camel@odyssey4.me> On Mon, 2020-08-03 at 13:28 -0600, Alex Schultz wrote: On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya < bdobreli at redhat.com > wrote: On 8/3/20 12:36 PM, Sagi Shnaidman wrote: Hi, Bogdan thanks for raising this up, although I'm not sure I understand what it is the problem with using action plugins. Action plugins are well known official extensions for Ansible, as any other plugins - callback, strategy, inventory etc [1]. It is not any hack or unsupported workaround, it's a known and official feature of Ansible. Why can't we use it? What makes it different from filter, I believe the cases that require the use of those should be justified. For the given example, that manages containers in a loop via calling a module, what the written custom callback plugin buys for us? That brings code to maintain, extra complexity, like handling possible corner cases in async mode, dry-run mode etc. But what is justification aside of looks handy? I disagree that we shouldn't use action plugins or modules. Tasks themselves are expensive at scale. We saw that when we switched away from paunch to container management in pure ansible tasks. This exposed that looping tasks are even more expensive and complex error handling and workflows are better suited for modules or action plugins than a series of tasks. This is not something to be "fixed in ansible". This is the nature of the executor and strategy related interactions. Should everything be converted to modules and plugins? no. Should everything be tasks only? no. It's a balance that must be struck between when a specific set of complex tasks need extra data processing or error handling. Switching to modules or action plugins allows us to unit test our logic. Using tasks do not have such a concept outside of writing complex molecule testing. IMHO it's safer to switch to modules/action plugins than writing task logic. I agree with Alex. Writing complex logic or trying to do error handling in tasks or jinja is not only very slow in execution, but gives us no way to properly test. Using ansible extensions like modules, action plugins, filters, etc gives us something that we can unit test, do better error handling with and therefore provides the abilty to produce a better quality result. While it is true that it does give us more of a downstream burden, our community is well versed in reading python code and testing python code properly. Sometimes it might seem easier to an author to prototype something using ansible/jinja, but if the result is complex then an extension of some kind should be considered as a iteration and it should be unit tested. I'd go as far as to say that if we add module, we should force the requirement for unit testing through some means to ensure good code quality and maintainability. Another benefit to using modules is that the Ansible tasks read more like a sequence of events that need to happen, which is exactly the spirit that Ansible has always advocated. When complex logic is implemented in tasks or in jinja, trying to follow the orchestration sequence becomes a *lot* harder. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bdobreli at redhat.com Tue Aug 4 09:35:06 2020 From: bdobreli at redhat.com (Bogdan Dobrelya) Date: Tue, 4 Aug 2020 11:35:06 +0200 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> Message-ID: <385dc8d7-198f-64ce-908f-49ab823ed229@redhat.com> On 8/3/20 9:28 PM, Alex Schultz wrote: > On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya wrote: >> >> On 8/3/20 12:36 PM, Sagi Shnaidman wrote: >>> Hi, Bogdan >>> >>> thanks for raising this up, although I'm not sure I understand what it >>> is the problem with using action plugins. >>> Action plugins are well known official extensions for Ansible, as any >>> other plugins - callback, strategy, inventory etc [1]. It is not any >>> hack or unsupported workaround, it's a known and official feature of >>> Ansible. Why can't we use it? What makes it different from filter, >> >> I believe the cases that require the use of those should be justified. >> For the given example, that manages containers in a loop via calling a >> module, what the written custom callback plugin buys for us? That brings >> code to maintain, extra complexity, like handling possible corner cases >> in async mode, dry-run mode etc. But what is justification aside of >> looks handy? > > I disagree that we shouldn't use action plugins or modules. Tasks > themselves are expensive at scale. We saw that when we switched away > from paunch to container management in pure ansible tasks. This > exposed that looping tasks are even more expensive and complex error > handling and workflows are better suited for modules or action plugins > than a series of tasks. This is not something to be "fixed in > ansible". This is the nature of the executor and strategy related > interactions. Should everything be converted to modules and plugins? > no. Should everything be tasks only? no. It's a balance that must be > struck between when a specific set of complex tasks need extra data > processing or error handling. Switching to modules or action plugins > allows us to unit test our logic. Using tasks do not have such a I can understand that ansible should not be fixed for some composition tasks what require iterations and have complex logic for its "unit of work". And such ones also should be unit tested indeed. What I do not fully understand though is then what abandoning paunch for its action plugin had bought for us in the end? Paunch was self-contained and had no external dependencies on fast-changing ansible frameworks. There was also no need for paunch to handle the ansible-specific execution strategies and nuances, like "what if that action plugin is called in async or in the check-mode?" Unit tests exited in paunch as well. It was easy to backport changes within a single code base. So, looking back retrospectively, was rewriting paunch as an action plugin a simplification of the deployment framework? Please reply to yourself honestly. It does pretty same things but differently and added external framework. It is now also self-contained action plugin, since traditional tasks cannot be used in loops for this goal because of performance reasons. To summarize, action plugins may be a good solution indeed, but perhaps we should go back and use paunch instead of ansible? Same applies for *some* other tasks? That would also provide a balance, for action plugins, tasks and common sense. > concept outside of writing complex molecule testing. IMHO it's safer > to switch to modules/action plugins than writing task logic. > > IMHO the issue that I see with the switch to Action plugins is the > increased load on the ansible "controller" node during execution. > Modules may be better depending on the task being managed. But I > believe with unit testing, action plugins or modules provide a cleaner > and more testable solution than writing roles consisting only of > tasks. > > > >> >>> lookup, inventory or any other plugin we already use? >>> Action plugins are also used wide in Ansible itself, for example >>> templates plugin is implemented with action plugin [2]. If Ansible can >>> use it, why can't we? I don't think there is something with "fixing" >>> Ansible, it's not a bug, this is a useful extension. >>> What regards the mentioned action plugin for podman containers, it >>> allows to spawn containers remotely while skipping the connection part >>> for every cycle. I'm not sure you can "fix" Ansible not to do that, it's >>> not a bug. We may not see the difference in a few hosts in CI, but it >>> might be very efficient when we deploy on 100+ hosts oro even 1000+ >>> hosts. In order to evaluate this on bigger setups to understand its >>> value we configured both options - to use action plugin or usual module. >>> If better performance of action plugin will be proven, we can switch to >>> use it, if it doesn't make a difference on bigger setups - then I think >>> we can easily switch back to using an usual module. >>> >>> Thanks >>> >>> [1] https://docs.ansible.com/ansible/latest/plugins/plugins.html >>> [2] >>> https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/template.py >>> >>> On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya >> > wrote: >>> >>> There is a trend of writing action plugins, see [0], for simple things, >>> like just calling a module in a loop. I'm not sure that is the >>> direction >>> TripleO should go. If ansible is inefficient in this sort of tasks >>> without custom python code written, we should fix ansible. Otherwise, >>> what is the ultimate goal of that trend? Is that having only action >>> plugins in roles and playbooks? >>> >>> Please kindly asking the community to stop that, make a step back and >>> reiterate with the taken approach. Thank you. >>> >>> [0] https://review.opendev.org/716108 >>> >>> >>> -- >>> Best regards, >>> Bogdan Dobrelya, >>> Irc #bogdando >>> >>> >>> >>> >>> -- >>> Best regards >>> Sagi Shnaidman >> >> >> -- >> Best regards, >> Bogdan Dobrelya, >> Irc #bogdando >> >> > -- Best regards, Bogdan Dobrelya, Irc #bogdando From sshnaidm at redhat.com Tue Aug 4 09:38:42 2020 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Tue, 4 Aug 2020 12:38:42 +0300 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: <385dc8d7-198f-64ce-908f-49ab823ed229@redhat.com> References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> <385dc8d7-198f-64ce-908f-49ab823ed229@redhat.com> Message-ID: Hi, Actually this discussion prompted me to investigate more how to optimize containers setup on a large number of hosts. As I saw from action plugin work, it still copies the module file each loop cycle, which is not very efficient behavior. That's why I started work on a podman container module which can start a bunch of containers in one call and accepts a list of containers as an input. In this case the module file will be transferred to the remote host only once and all containers execution will be done by python on the remote host. That way we'll avoid unnecessary establishing connections, copying files, setting permissions etc what happens every time we cycle over data. It will be done only once. I'll send a patch soon for review. Thanks On Tue, Aug 4, 2020 at 12:35 PM Bogdan Dobrelya wrote: > On 8/3/20 9:28 PM, Alex Schultz wrote: > > On Mon, Aug 3, 2020 at 6:34 AM Bogdan Dobrelya > wrote: > >> > >> On 8/3/20 12:36 PM, Sagi Shnaidman wrote: > >>> Hi, Bogdan > >>> > >>> thanks for raising this up, although I'm not sure I understand what it > >>> is the problem with using action plugins. > >>> Action plugins are well known official extensions for Ansible, as any > >>> other plugins - callback, strategy, inventory etc [1]. It is not any > >>> hack or unsupported workaround, it's a known and official feature of > >>> Ansible. Why can't we use it? What makes it different from filter, > >> > >> I believe the cases that require the use of those should be justified. > >> For the given example, that manages containers in a loop via calling a > >> module, what the written custom callback plugin buys for us? That brings > >> code to maintain, extra complexity, like handling possible corner cases > >> in async mode, dry-run mode etc. But what is justification aside of > >> looks handy? > > > > I disagree that we shouldn't use action plugins or modules. Tasks > > themselves are expensive at scale. We saw that when we switched away > > from paunch to container management in pure ansible tasks. This > > exposed that looping tasks are even more expensive and complex error > > handling and workflows are better suited for modules or action plugins > > than a series of tasks. This is not something to be "fixed in > > ansible". This is the nature of the executor and strategy related > > interactions. Should everything be converted to modules and plugins? > > no. Should everything be tasks only? no. It's a balance that must be > > struck between when a specific set of complex tasks need extra data > > processing or error handling. Switching to modules or action plugins > > allows us to unit test our logic. Using tasks do not have such a > > I can understand that ansible should not be fixed for some composition > tasks what require iterations and have complex logic for its "unit of > work". And such ones also should be unit tested indeed. What I do not > fully understand though is then what abandoning paunch for its action > plugin had bought for us in the end? > > Paunch was self-contained and had no external dependencies on > fast-changing ansible frameworks. There was also no need for paunch to > handle the ansible-specific execution strategies and nuances, like "what > if that action plugin is called in async or in the check-mode?" Unit > tests exited in paunch as well. It was easy to backport changes within a > single code base. > > So, looking back retrospectively, was rewriting paunch as an action > plugin a simplification of the deployment framework? Please reply to > yourself honestly. It does pretty same things but differently and added > external framework. It is now also self-contained action plugin, since > traditional tasks cannot be used in loops for this goal because of > performance reasons. > > To summarize, action plugins may be a good solution indeed, but perhaps > we should go back and use paunch instead of ansible? Same applies for > *some* other tasks? That would also provide a balance, for action > plugins, tasks and common sense. > > > concept outside of writing complex molecule testing. IMHO it's safer > > to switch to modules/action plugins than writing task logic. > > > > IMHO the issue that I see with the switch to Action plugins is the > > increased load on the ansible "controller" node during execution. > > Modules may be better depending on the task being managed. But I > > believe with unit testing, action plugins or modules provide a cleaner > > and more testable solution than writing roles consisting only of > > tasks. > > > > > > > >> > >>> lookup, inventory or any other plugin we already use? > >>> Action plugins are also used wide in Ansible itself, for example > >>> templates plugin is implemented with action plugin [2]. If Ansible can > >>> use it, why can't we? I don't think there is something with "fixing" > >>> Ansible, it's not a bug, this is a useful extension. > >>> What regards the mentioned action plugin for podman containers, it > >>> allows to spawn containers remotely while skipping the connection part > >>> for every cycle. I'm not sure you can "fix" Ansible not to do that, > it's > >>> not a bug. We may not see the difference in a few hosts in CI, but it > >>> might be very efficient when we deploy on 100+ hosts oro even 1000+ > >>> hosts. In order to evaluate this on bigger setups to understand its > >>> value we configured both options - to use action plugin or usual > module. > >>> If better performance of action plugin will be proven, we can switch to > >>> use it, if it doesn't make a difference on bigger setups - then I think > >>> we can easily switch back to using an usual module. > >>> > >>> Thanks > >>> > >>> [1] https://docs.ansible.com/ansible/latest/plugins/plugins.html > >>> [2] > >>> > https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/template.py > >>> > >>> On Mon, Aug 3, 2020 at 11:19 AM Bogdan Dobrelya >>> > wrote: > >>> > >>> There is a trend of writing action plugins, see [0], for simple > things, > >>> like just calling a module in a loop. I'm not sure that is the > >>> direction > >>> TripleO should go. If ansible is inefficient in this sort of tasks > >>> without custom python code written, we should fix ansible. > Otherwise, > >>> what is the ultimate goal of that trend? Is that having only > action > >>> plugins in roles and playbooks? > >>> > >>> Please kindly asking the community to stop that, make a step back > and > >>> reiterate with the taken approach. Thank you. > >>> > >>> [0] https://review.opendev.org/716108 > >>> > >>> > >>> -- > >>> Best regards, > >>> Bogdan Dobrelya, > >>> Irc #bogdando > >>> > >>> > >>> > >>> > >>> -- > >>> Best regards > >>> Sagi Shnaidman > >> > >> > >> -- > >> Best regards, > >> Bogdan Dobrelya, > >> Irc #bogdando > >> > >> > > > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando > > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From marino.mrc at gmail.com Tue Aug 4 09:55:52 2020 From: marino.mrc at gmail.com (Marco Marino) Date: Tue, 4 Aug 2020 11:55:52 +0200 Subject: [ironic][tripleo][ussuri] Problem with bare metal provisioning and old RAID controllers Message-ID: Hi, I'm trying to install openstack Ussuri on Centos 8 hardware using tripleo. I'm using a relatively old hardware (dell PowerEdge R620) with old RAID controllers, deprecated in RHEL8/Centos8. Here is some basic information: # lspci | grep -i raid 00:1f.2 RAID bus controller: Intel Corporation C600/X79 series chipset SATA RAID Controller (rev 05) 02:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2008 [Falcon] (rev 03) I'm able to manually install centos 8 using DUD driver from here -> https://elrepo.org/linux/dud/el8/x86_64/dd-megaraid_sas-07.710.50.00-1.el8_2.elrepo.iso (basically I add inst.dd and I use an usb pendrive with iso). Is there a way to do bare metal provisioning using openstack on this kind of server? At the moment, when I launch "openstack overcloud node introspect --provide controller1" it doesn't recognize disks (local_gb = 0 in properties) and in inspector logs I see: Jun 22 11:12:42 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:42.261 1543 DEBUG root [-] Still waiting for the root device to appear, attempt 1 of 10 wait_for_disks /usr/lib/python3.6/site-packages/ironic_python_agent/hardware.py:652 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.299 1543 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): udevadm settle execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.357 1543 DEBUG oslo_concurrency.processutils [-] CMD "udevadm settle" returned: 0 in 0.058s execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.392 1543 DEBUG ironic_lib.utils [-] Execution completed, command line is "udevadm settle" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.426 1543 DEBUG ironic_lib.utils [-] Command stdout is: "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.460 1543 DEBUG ironic_lib.utils [-] Command stderr is: "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.496 1543 WARNING root [-] Path /dev/disk/by-path is inaccessible, /dev/disk/by-path/* version of block device name is unavailable Cause: [Errno 2] No such file or directory: '/dev/disk/by-path': FileNotFoundError: [Errno 2] No such file or directory: '/dev/disk/by-path' Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.549 1543 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.647 1543 DEBUG oslo_concurrency.processutils [-] CMD "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" returned: 0 in 0.097s execute /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.683 1543 DEBUG ironic_lib.utils [-] Execution completed, command line is "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.719 1543 DEBUG ironic_lib.utils [-] Command stdout is: "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: 2018-06-22 11:12:45.755 1543 DEBUG ironic_lib.utils [-] Command stderr is: "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 Is there a way to solve the issue? For example, can I modify ramdisk and include DUD driver? I tried this guide: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/partner_integration/overcloud_images#initrd_modifying_the_initial_ramdisks but I don't know how to include an ISO instead of an rpm packet as described in the example. Thank you, Marco -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Tue Aug 4 10:30:04 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Tue, 4 Aug 2020 12:30:04 +0200 Subject: [ironic][tripleo][ussuri] Problem with bare metal provisioning and old RAID controllers In-Reply-To: References: Message-ID: Hi, On Tue, Aug 4, 2020 at 11:58 AM Marco Marino wrote: > Hi, I'm trying to install openstack Ussuri on Centos 8 hardware using > tripleo. I'm using a relatively old hardware (dell PowerEdge R620) with old > RAID controllers, deprecated in RHEL8/Centos8. Here is some basic > information: > # lspci | grep -i raid > 00:1f.2 RAID bus controller: Intel Corporation C600/X79 series chipset > SATA RAID Controller (rev 05) > 02:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2008 [Falcon] > (rev 03) > > I'm able to manually install centos 8 using DUD driver from here -> > https://elrepo.org/linux/dud/el8/x86_64/dd-megaraid_sas-07.710.50.00-1.el8_2.elrepo.iso > (basically I add inst.dd and I use an usb pendrive with iso). > Is there a way to do bare metal provisioning using openstack on this kind > of server? At the moment, when I launch "openstack overcloud node > introspect --provide controller1" it doesn't recognize disks (local_gb = 0 > in properties) and in inspector logs I see: > Jun 22 11:12:42 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:42.261 1543 DEBUG root [-] Still waiting for the root > device to appear, attempt 1 of 10 wait_for_disks > /usr/lib/python3.6/site-packages/ironic_python_agent/hardware.py:652 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.299 1543 DEBUG oslo_concurrency.processutils [-] > Running cmd (subprocess): udevadm settle execute > /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.357 1543 DEBUG oslo_concurrency.processutils [-] CMD > "udevadm settle" returned: 0 in 0.058s execute > /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.392 1543 DEBUG ironic_lib.utils [-] Execution > completed, command line is "udevadm settle" execute > /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.426 1543 DEBUG ironic_lib.utils [-] Command stdout is: > "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.460 1543 DEBUG ironic_lib.utils [-] Command stderr is: > "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.496 1543 WARNING root [-] Path /dev/disk/by-path is > inaccessible, /dev/disk/by-path/* version of block device name is > unavailable Cause: [Errno 2] No such file or directory: > '/dev/disk/by-path': FileNotFoundError: [Errno 2] No such file or > directory: '/dev/disk/by-path' > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.549 1543 DEBUG oslo_concurrency.processutils [-] > Running cmd (subprocess): lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE execute > /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.647 1543 DEBUG oslo_concurrency.processutils [-] CMD > "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" returned: 0 in 0.097s execute > /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.683 1543 DEBUG ironic_lib.utils [-] Execution > completed, command line is "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" > execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.719 1543 DEBUG ironic_lib.utils [-] Command stdout is: > "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 > Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: > 2018-06-22 11:12:45.755 1543 DEBUG ironic_lib.utils [-] Command stderr is: > "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 > > Is there a way to solve the issue? For example, can I modify ramdisk and > include DUD driver? I tried this guide: > https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/partner_integration/overcloud_images#initrd_modifying_the_initial_ramdisks > > but I don't know how to include an ISO instead of an rpm packet as > described in the example. > Indeed, I don't think you can use ISO as it is, you'll need to figure out what is inside. If it's an RPM (as I assume), you'll need to extract it and install into the ramdisk. If nothing helps, you can try building a ramdisk with CentOS 7, the (very) recent versions of ironic-python-agent-builder allow using Python 3 on CentOS 7. Dmitry > Thank you, > Marco > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marino.mrc at gmail.com Tue Aug 4 10:57:13 2020 From: marino.mrc at gmail.com (Marco Marino) Date: Tue, 4 Aug 2020 12:57:13 +0200 Subject: [ironic][tripleo][ussuri] Problem with bare metal provisioning and old RAID controllers In-Reply-To: References: Message-ID: Here is what I did: # /usr/lib/dracut/skipcpio /home/stack/images/ironic-python-agent.initramfs | zcat | cpio -ivd | pax -r # mount dd-megaraid_sas-07.710.50.00-1.el8_2.elrepo.iso /mnt/ # rpm2cpio /mnt/rpms/x86_64/kmod-megaraid_sas-07.710.50.00-1.el8_2.elrepo.x86_64.rpm | pax -r # find . 2>/dev/null | cpio --quiet -c -o | gzip -8 > /home/stack/images/ironic-python-agent.initramfs # chown stack: /home/stack/images/ironic-python-agent.initramfs (undercloud) [stack at undercloud ~]$ openstack overcloud image upload --update-existing --image-path /home/stack/images/ At this point I checked that agent.ramdisk in /var/lib/ironic/httpboot has an update timestamp Then (undercloud) [stack at undercloud ~]$ openstack overcloud node introspect --provide controller2 /usr/lib64/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__ return f(*args, **kwds) PLAY [Baremetal Introspection for multiple Ironic Nodes] *********************** 2020-08-04 12:32:26.684368 | ecf4bbd2-e605-20dd-3da9-000000000008 | TASK | Check for required inputs 2020-08-04 12:32:26.739797 | ecf4bbd2-e605-20dd-3da9-000000000008 | SKIPPED | Check for required inputs | localhost | item=node_uuids 2020-08-04 12:32:26.746684 | ecf4bbd2-e605-20dd-3da9-00000000000a | TASK | Set node_uuids_intro fact [WARNING]: Failure using method (v2_playbook_on_task_start) in callback plugin (): maximum recursion depth exceeded while calling a Python object 2020-08-04 12:32:26.828985 | ecf4bbd2-e605-20dd-3da9-00000000000a | OK | Set node_uuids_intro fact | localhost 2020-08-04 12:32:26.834281 | ecf4bbd2-e605-20dd-3da9-00000000000c | TASK | Notice 2020-08-04 12:32:26.911106 | ecf4bbd2-e605-20dd-3da9-00000000000c | SKIPPED | Notice | localhost 2020-08-04 12:32:26.916344 | ecf4bbd2-e605-20dd-3da9-00000000000e | TASK | Set concurrency fact 2020-08-04 12:32:26.994087 | ecf4bbd2-e605-20dd-3da9-00000000000e | OK | Set concurrency fact | localhost 2020-08-04 12:32:27.005932 | ecf4bbd2-e605-20dd-3da9-000000000010 | TASK | Check if validation enabled 2020-08-04 12:32:27.116425 | ecf4bbd2-e605-20dd-3da9-000000000010 | SKIPPED | Check if validation enabled | localhost 2020-08-04 12:32:27.129120 | ecf4bbd2-e605-20dd-3da9-000000000011 | TASK | Run Validations 2020-08-04 12:32:27.239850 | ecf4bbd2-e605-20dd-3da9-000000000011 | SKIPPED | Run Validations | localhost 2020-08-04 12:32:27.251796 | ecf4bbd2-e605-20dd-3da9-000000000012 | TASK | Fail if validations are disabled 2020-08-04 12:32:27.362050 | ecf4bbd2-e605-20dd-3da9-000000000012 | SKIPPED | Fail if validations are disabled | localhost 2020-08-04 12:32:27.373947 | ecf4bbd2-e605-20dd-3da9-000000000014 | TASK | Start baremetal introspection 2020-08-04 12:48:19.944028 | ecf4bbd2-e605-20dd-3da9-000000000014 | CHANGED | Start baremetal introspection | localhost 2020-08-04 12:48:19.966517 | ecf4bbd2-e605-20dd-3da9-000000000015 | TASK | Nodes that passed introspection 2020-08-04 12:48:20.130913 | ecf4bbd2-e605-20dd-3da9-000000000015 | OK | Nodes that passed introspection | localhost | result={ "changed": false, "msg": " 00c5e81b-1e5d-442b-b64f-597a604051f7" } 2020-08-04 12:48:20.142919 | ecf4bbd2-e605-20dd-3da9-000000000016 | TASK | Nodes that failed introspection 2020-08-04 12:48:20.305004 | ecf4bbd2-e605-20dd-3da9-000000000016 | OK | Nodes that failed introspection | localhost | result={ "changed": false, "failed_when_result": false, "msg": " All nodes completed introspection successfully!" } 2020-08-04 12:48:20.316860 | ecf4bbd2-e605-20dd-3da9-000000000017 | TASK | Node introspection failed and no results are provided 2020-08-04 12:48:20.427675 | ecf4bbd2-e605-20dd-3da9-000000000017 | SKIPPED | Node introspection failed and no results are provided | localhost PLAY RECAP ********************************************************************* localhost : ok=5 changed=1 unreachable=0 failed=0 skipped=6 rescued=0 ignored=0 [WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin (): _output() missing 1 required positional argument: 'color' Successfully introspected nodes: ['controller2'] Exception occured while running the command Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", line 340, in prepare_command cmdline_args = self.loader.load_file('args', string_types, encoding=None) File "/usr/lib/python3.6/site-packages/ansible_runner/loader.py", line 164, in load_file contents = parsed_data = self.get_contents(path) File "/usr/lib/python3.6/site-packages/ansible_runner/loader.py", line 98, in get_contents raise ConfigurationError('specified path does not exist %s' % path) ansible_runner.exceptions.ConfigurationError: specified path does not exist /tmp/tripleop89yr8i8/args During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run return super(Command, self).run(parsed_args) File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run return_code = self.take_action(parsed_args) or 0 File "/usr/lib/python3.6/site-packages/tripleoclient/v2/overcloud_node.py", line 210, in take_action node_uuids=parsed_args.node_uuids, File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/baremetal.py", line 134, in provide 'node_uuids': node_uuids File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 659, in run_ansible_playbook runner_config.prepare() File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", line 174, in prepare self.prepare_command() File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", line 346, in prepare_command self.command = self.generate_ansible_command() File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", line 415, in generate_ansible_command v = 'v' * self.verbosity TypeError: can't multiply sequence by non-int of type 'ClientManager' can't multiply sequence by non-int of type 'ClientManager' (undercloud) [stack at undercloud ~]$ and (undercloud) [stack at undercloud ~]$ openstack baremetal node show controller2 .... | properties | {'local_gb': '0', 'cpus': '24', 'cpu_arch': 'x86_64', 'memory_mb': '32768', 'capabilities': 'cpu_vt:true,cpu_aes:true,cpu_hugepages:true,cpu_hugepages_1g:true,cpu_txt:true'} It seems that megaraid driver is correctly inserted in ramdisk: # lsinitrd /var/lib/ironic/httpboot/agent.ramdisk | grep megaraid /bin/lsinitrd: line 276: warning: command substitution: ignored null byte in input -rw-r--r-- 1 root root 50 Apr 28 21:55 etc/depmod.d/kmod-megaraid_sas.conf drwxr-xr-x 2 root root 0 Aug 4 12:13 usr/lib/modules/4.18.0-193.6.3.el8_2.x86_64/kernel/drivers/scsi/megaraid -rw-r--r-- 1 root root 68240 Aug 4 12:13 usr/lib/modules/4.18.0-193.6.3.el8_2.x86_64/kernel/drivers/scsi/megaraid/megaraid_sas.ko.xz drwxr-xr-x 2 root root 0 Apr 28 21:55 usr/lib/modules/4.18.0-193.el8.x86_64/extra/megaraid_sas -rw-r--r-- 1 root root 309505 Apr 28 21:55 usr/lib/modules/4.18.0-193.el8.x86_64/extra/megaraid_sas/megaraid_sas.ko drwxr-xr-x 2 root root 0 Apr 28 21:55 usr/share/doc/kmod-megaraid_sas-07.710.50.00 -rw-r--r-- 1 root root 18092 Apr 28 21:55 usr/share/doc/kmod-megaraid_sas-07.710.50.00/GPL-v2.0.txt -rw-r--r-- 1 root root 1152 Apr 28 21:55 usr/share/doc/kmod-megaraid_sas-07.710.50.00/greylist.txt If the solution is to use a Centos7 ramdisk, please can you give me some hint? I have no idea on how to build a new ramdisk from scratch Thank you Il giorno mar 4 ago 2020 alle ore 12:33 Dmitry Tantsur ha scritto: > Hi, > > On Tue, Aug 4, 2020 at 11:58 AM Marco Marino wrote: > >> Hi, I'm trying to install openstack Ussuri on Centos 8 hardware using >> tripleo. I'm using a relatively old hardware (dell PowerEdge R620) with old >> RAID controllers, deprecated in RHEL8/Centos8. Here is some basic >> information: >> # lspci | grep -i raid >> 00:1f.2 RAID bus controller: Intel Corporation C600/X79 series chipset >> SATA RAID Controller (rev 05) >> 02:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2008 [Falcon] >> (rev 03) >> >> I'm able to manually install centos 8 using DUD driver from here -> >> https://elrepo.org/linux/dud/el8/x86_64/dd-megaraid_sas-07.710.50.00-1.el8_2.elrepo.iso >> (basically I add inst.dd and I use an usb pendrive with iso). >> Is there a way to do bare metal provisioning using openstack on this kind >> of server? At the moment, when I launch "openstack overcloud node >> introspect --provide controller1" it doesn't recognize disks (local_gb = 0 >> in properties) and in inspector logs I see: >> Jun 22 11:12:42 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:42.261 1543 DEBUG root [-] Still waiting for the root >> device to appear, attempt 1 of 10 wait_for_disks >> /usr/lib/python3.6/site-packages/ironic_python_agent/hardware.py:652 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.299 1543 DEBUG oslo_concurrency.processutils [-] >> Running cmd (subprocess): udevadm settle execute >> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.357 1543 DEBUG oslo_concurrency.processutils [-] CMD >> "udevadm settle" returned: 0 in 0.058s execute >> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.392 1543 DEBUG ironic_lib.utils [-] Execution >> completed, command line is "udevadm settle" execute >> /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.426 1543 DEBUG ironic_lib.utils [-] Command stdout is: >> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.460 1543 DEBUG ironic_lib.utils [-] Command stderr is: >> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.496 1543 WARNING root [-] Path /dev/disk/by-path is >> inaccessible, /dev/disk/by-path/* version of block device name is >> unavailable Cause: [Errno 2] No such file or directory: >> '/dev/disk/by-path': FileNotFoundError: [Errno 2] No such file or >> directory: '/dev/disk/by-path' >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.549 1543 DEBUG oslo_concurrency.processutils [-] >> Running cmd (subprocess): lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE execute >> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.647 1543 DEBUG oslo_concurrency.processutils [-] CMD >> "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" returned: 0 in 0.097s execute >> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.683 1543 DEBUG ironic_lib.utils [-] Execution >> completed, command line is "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" >> execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.719 1543 DEBUG ironic_lib.utils [-] Command stdout is: >> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 >> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >> 2018-06-22 11:12:45.755 1543 DEBUG ironic_lib.utils [-] Command stderr is: >> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 >> >> Is there a way to solve the issue? For example, can I modify ramdisk and >> include DUD driver? I tried this guide: >> https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/partner_integration/overcloud_images#initrd_modifying_the_initial_ramdisks >> >> but I don't know how to include an ISO instead of an rpm packet as >> described in the example. >> > > Indeed, I don't think you can use ISO as it is, you'll need to figure out > what is inside. If it's an RPM (as I assume), you'll need to extract it and > install into the ramdisk. > > If nothing helps, you can try building a ramdisk with CentOS 7, the (very) > recent versions of ironic-python-agent-builder allow using Python 3 on > CentOS 7. > > Dmitry > > >> Thank you, >> Marco >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From victoria at vmartinezdelacruz.com Tue Aug 4 11:43:05 2020 From: victoria at vmartinezdelacruz.com (=?UTF-8?Q?Victoria_Mart=C3=ADnez_de_la_Cruz?=) Date: Tue, 4 Aug 2020 08:43:05 -0300 Subject: [openstack][stein][manila-ui] error In-Reply-To: References: Message-ID: Glad to hear it is working ok now! Cheers, V On Tue, Aug 4, 2020 at 2:50 AM Ignazio Cassano wrote: > Hello Victoria and Goutham, thank you for your great help. > Unfortunately I made I mistake in my ansible playbook for installing > manila: it created manila services more times, so some entries in the > catalog did not have an endpoint associated. > I removed the duplicated service entries where catalog was absent and now > it works. > Many thanks > Ignazio > > Il giorno mar 4 ago 2020 alle ore 02:53 Victoria Martínez de la Cruz < > victoria at vmartinezdelacruz.com> ha scritto: > >> In local_settings.py under openstack-dashboard. And then restart the >> webserver. >> >> Did you copy the enable and local files from manila-ui under Horizon's >> namespace? Check out >> https://docs.openstack.org/manila-ui/latest/install/installation.html >> >> We can continue debugging tomorrow, we will find out what is going on. >> >> Cheers, >> >> V >> >> >> On Mon, Aug 3, 2020, 6:46 PM Ignazio Cassano >> wrote: >> >>> Hello Goutham,tomorrow I will check the catalog. >>> Must I enable the debug option in dashboard local_setting or in >>> manila.conf? >>> Thanks >>> Ignazio >>> >>> >>> Il Lun 3 Ago 2020, 23:01 Goutham Pacha Ravi ha >>> scritto: >>> >>>> >>>> >>>> >>>> On Mon, Aug 3, 2020 at 1:31 PM Ignazio Cassano < >>>> ignaziocassano at gmail.com> wrote: >>>> >>>>> I mean I am using dhss false >>>>> >>>>> Il Lun 3 Ago 2020, 21:41 Ignazio Cassano >>>>> ha scritto: >>>>> >>>>>> PS ps >>>>>> Sorry If aI am writing again. >>>>>> The command: >>>>>> manila list let me to show shares I created with command line. >>>>>> The dashboard gives errors I reported in my first email. >>>>>> Looking at manila.py line 280 it checks shares under share networks. >>>>>> Ignazio >>>>>> >>>>>> >>>>>> Il Lun 3 Ago 2020, 21:34 Ignazio Cassano >>>>>> ha scritto: >>>>>> >>>>>>> PS >>>>>>> I followed installation guide under docs.openstack.org. >>>>>>> >>>>>>> >>>>>>> Il Lun 3 Ago 2020, 21:21 Victoria Martínez de la Cruz < >>>>>>> victoria at vmartinezdelacruz.com> ha scritto: >>>>>>> >>>>>>>> Hi Ignazio, >>>>>>>> >>>>>>>> How did you deploy Manila and Manila UI? Can you point me toward >>>>>>>> the docs you used? >>>>>>>> >>>>>>>> Also, which is the specific workflow you are following to reach >>>>>>>> that trace? Just opening the dashboard and clicking on the Shares tab? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> >>>>>>>> V >>>>>>>> >>>>>>>> On Mon, Aug 3, 2020 at 4:55 AM Ignazio Cassano < >>>>>>>> ignaziocassano at gmail.com> wrote: >>>>>>>> >>>>>>>>> Hello, I installed manila on openstack stein and it works by >>>>>>>>> command line mat the manila ui does not work and in httpd error log I read: >>>>>>>>> >>>>>>>>> [Mon Aug 03 07:45:26.697408 2020] [:error] [pid 3506291] ERROR >>>>>>>>> django.request Internal Server Error: /dashboard/project/shares/ >>>>>>>>> [Mon Aug 03 07:45:26.697437 2020] [:error] [pid 3506291] Traceback >>>>>>>>> (most recent call last): >>>>>>>>> [Mon Aug 03 07:45:26.697442 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line >>>>>>>>> 41, in inner >>>>>>>>> [Mon Aug 03 07:45:26.697446 2020] [:error] [pid 3506291] >>>>>>>>> response = get_response(request) >>>>>>>>> [Mon Aug 03 07:45:26.697450 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, >>>>>>>>> in _get_response >>>>>>>>> [Mon Aug 03 07:45:26.697453 2020] [:error] [pid 3506291] >>>>>>>>> response = self.process_exception_by_middleware(e, request) >>>>>>>>> [Mon Aug 03 07:45:26.697466 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, >>>>>>>>> in _get_response >>>>>>>>> [Mon Aug 03 07:45:26.697471 2020] [:error] [pid 3506291] >>>>>>>>> response = wrapped_callback(request, *callback_args, **callback_kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697475 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec >>>>>>>>> [Mon Aug 03 07:45:26.697479 2020] [:error] [pid 3506291] >>>>>>>>> return view_func(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697482 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>>>>> [Mon Aug 03 07:45:26.697485 2020] [:error] [pid 3506291] >>>>>>>>> return view_func(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697489 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec >>>>>>>>> [Mon Aug 03 07:45:26.697492 2020] [:error] [pid 3506291] >>>>>>>>> return view_func(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697496 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec >>>>>>>>> [Mon Aug 03 07:45:26.697499 2020] [:error] [pid 3506291] >>>>>>>>> return view_func(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697502 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec >>>>>>>>> [Mon Aug 03 07:45:26.697506 2020] [:error] [pid 3506291] >>>>>>>>> return view_func(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697509 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, >>>>>>>>> in view >>>>>>>>> [Mon Aug 03 07:45:26.697513 2020] [:error] [pid 3506291] >>>>>>>>> return self.dispatch(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697516 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, >>>>>>>>> in dispatch >>>>>>>>> [Mon Aug 03 07:45:26.697520 2020] [:error] [pid 3506291] >>>>>>>>> return handler(request, *args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697523 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get >>>>>>>>> [Mon Aug 03 07:45:26.697526 2020] [:error] [pid 3506291] >>>>>>>>> handled = self.construct_tables() >>>>>>>>> [Mon Aug 03 07:45:26.697530 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in >>>>>>>>> construct_tables >>>>>>>>> [Mon Aug 03 07:45:26.697533 2020] [:error] [pid 3506291] >>>>>>>>> handled = self.handle_table(table) >>>>>>>>> [Mon Aug 03 07:45:26.697537 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in >>>>>>>>> handle_table >>>>>>>>> [Mon Aug 03 07:45:26.697540 2020] [:error] [pid 3506291] data >>>>>>>>> = self._get_data_dict() >>>>>>>>> [Mon Aug 03 07:45:26.697544 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 43, in >>>>>>>>> _get_data_dict >>>>>>>>> [Mon Aug 03 07:45:26.697547 2020] [:error] [pid 3506291] >>>>>>>>> data.extend(func()) >>>>>>>>> [Mon Aug 03 07:45:26.697550 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py", line 109, in >>>>>>>>> wrapped >>>>>>>>> [Mon Aug 03 07:45:26.697554 2020] [:error] [pid 3506291] value >>>>>>>>> = cache[key] = func(*args, **kwargs) >>>>>>>>> [Mon Aug 03 07:45:26.697557 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py", >>>>>>>>> line 57, in get_shares_data >>>>>>>>> [Mon Aug 03 07:45:26.697561 2020] [:error] [pid 3506291] >>>>>>>>> share_nets = manila.share_network_list(self.request) >>>>>>>>> [Mon Aug 03 07:45:26.697564 2020] [:error] [pid 3506291] File >>>>>>>>> "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py", line 280, in >>>>>>>>> share_network_list >>>>>>>>> [Mon Aug 03 07:45:26.697568 2020] [:error] [pid 3506291] >>>>>>>>> return manilaclient(request).share_networks.list(detailed=detailed, >>>>>>>>> [Mon Aug 03 07:45:26.697571 2020] [:error] [pid 3506291] >>>>>>>>> AttributeError: 'NoneType' object has no attribute 'share_networks' >>>>>>>>> >>>>>>>> >>>> Looking at the error here, and the code - it could be that the UI isn't >>>> able to retrieve the manila service endpoint from the service catalog. If >>>> this is the case, you must be able to see a "DEBUG" level log in your httpd >>>> error log with "no share service configured". Do you see it? >>>> >>>> As the user you're using on horizon, can you perform "openstack catalog >>>> list" and check whether the "sharev2" service type exists in that list? >>>> >>>> >>>>> >>>>>>>>> Please, anyone could help ? >>>>>>>>> Ignazio >>>>>>>>> >>>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From yan.y.zhao at intel.com Tue Aug 4 08:37:08 2020 From: yan.y.zhao at intel.com (Yan Zhao) Date: Tue, 4 Aug 2020 16:37:08 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200730112930.6f4c5762@x1.home> References: <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200729131255.68730f68@x1.home> <20200730034104.GB32327@joy-OptiPlex-7040> <20200730112930.6f4c5762@x1.home> Message-ID: <20200804083708.GA30485@joy-OptiPlex-7040> > > yes, include a device_api field is better. > > for mdev, "device_type=vfio-mdev", is it right? > > No, vfio-mdev is not a device API, it's the driver that attaches to the > mdev bus device to expose it through vfio. The device_api exposes the > actual interface of the vfio device, it's also vfio-pci for typical > mdev devices found on x86, but may be vfio-ccw, vfio-ap, etc... See > VFIO_DEVICE_API_PCI_STRING and friends. > ok. got it. > > > > > device_id=8086591d > > > > > > Is device_id interpreted relative to device_type? How does this > > > relate to mdev_type? If we have an mdev_type, doesn't that fully > > > defined the software API? > > > > > it's parent pci id for mdev actually. > > If we need to specify the parent PCI ID then something is fundamentally > wrong with the mdev_type. The mdev_type should define a unique, > software compatible interface, regardless of the parent device IDs. If > a i915-GVTg_V5_2 means different things based on the parent device IDs, > then then different mdev_types should be reported for those parent > devices. > hmm, then do we allow vendor specific fields? or is it a must that a vendor specific field should have corresponding vendor attribute? another thing is that the definition of mdev_type in GVT only corresponds to vGPU computing ability currently, e.g. i915-GVTg_V5_2, is 1/2 of a gen9 IGD, i915-GVTg_V4_2 is 1/2 of a gen8 IGD. It is too coarse-grained to live migration compatibility. Do you think we need to update GVT's definition of mdev_type? And is there any guide in mdev_type definition? > > > > > mdev_type=i915-GVTg_V5_2 > > > > > > And how are non-mdev devices represented? > > > > > non-mdev can opt to not include this field, or as you said below, a > > vendor signature. > > > > > > > aggregator=1 > > > > > pv_mode="none+ppgtt+context" > > > > > > These are meaningless vendor specific matches afaict. > > > > > yes, pv_mode and aggregator are vendor specific fields. > > but they are important to decide whether two devices are compatible. > > pv_mode means whether a vGPU supports guest paravirtualized api. > > "none+ppgtt+context" means guest can not use pv, or use ppgtt mode pv or > > use context mode pv. > > > > > > > interface_version=3 > > > > > > Not much granularity here, I prefer Sean's previous > > > .[.bugfix] scheme. > > > > > yes, .[.bugfix] scheme may be better, but I'm not sure if > > it works for a complicated scenario. > > e.g for pv_mode, > > (1) initially, pv_mode is not supported, so it's pv_mode=none, it's 0.0.0, > > (2) then, pv_mode=ppgtt is supported, pv_mode="none+ppgtt", it's 0.1.0, > > indicating pv_mode=none can migrate to pv_mode="none+ppgtt", but not vice versa. > > (3) later, pv_mode=context is also supported, > > pv_mode="none+ppgtt+context", so it's 0.2.0. > > > > But if later, pv_mode=ppgtt is removed. pv_mode="none+context", how to > > name its version? "none+ppgtt" (0.1.0) is not compatible to > > "none+context", but "none+ppgtt+context" (0.2.0) is compatible to > > "none+context". > > If pv_mode=ppgtt is removed, then the compatible versions would be > 0.0.0 or 1.0.0, ie. the major version would be incremented due to > feature removal. > > > Maintain such scheme is painful to vendor driver. > > Migration compatibility is painful, there's no way around that. I > think the version scheme is an attempt to push some of that low level > burden on the vendor driver, otherwise the management tools need to > work on an ever growing matrix of vendor specific features which is > going to become unwieldy and is largely meaningless outside of the > vendor driver. Instead, the vendor driver can make strategic decisions > about where to continue to maintain a support burden and make explicit > decisions to maintain or break compatibility. The version scheme is a > simplification and abstraction of vendor driver features in order to > create a small, logical compatibility matrix. Compromises necessarily > need to be made for that to occur. > ok. got it. > > > > > COMPATIBLE: > > > > > device_type=pci > > > > > device_id=8086591d > > > > > mdev_type=i915-GVTg_V5_{val1:int:1,2,4,8} > > > > this mixed notation will be hard to parse so i would avoid that. > > > > > > Some background, Intel has been proposing aggregation as a solution to > > > how we scale mdev devices when hardware exposes large numbers of > > > assignable objects that can be composed in essentially arbitrary ways. > > > So for instance, if we have a workqueue (wq), we might have an mdev > > > type for 1wq, 2wq, 3wq,... Nwq. It's not really practical to expose a > > > discrete mdev type for each of those, so they want to define a base > > > type which is composable to other types via this aggregation. This is > > > what this substitution and tagging is attempting to accomplish. So > > > imagine this set of values for cases where it's not practical to unroll > > > the values for N discrete types. > > > > > > > > aggregator={val1}/2 > > > > > > So the {val1} above would be substituted here, though an aggregation > > > factor of 1/2 is a head scratcher... > > > > > > > > pv_mode={val2:string:"none+ppgtt","none+context","none+ppgtt+context"} > > > > > > I'm lost on this one though. I think maybe it's indicating that it's > > > compatible with any of these, so do we need to list it? Couldn't this > > > be handled by Sean's version proposal where the minor version > > > represents feature compatibility? > > yes, it's indicating that it's compatible with any of these. > > Sean's version proposal may also work, but it would be painful for > > vendor driver to maintain the versions when multiple similar features > > are involved. > > This is something vendor drivers need to consider when adding and > removing features. > > > > > > interface_version={val3:int:2,3} > > > > > > What does this turn into in a few years, 2,7,12,23,75,96,... > > > > > is a range better? > > I was really trying to point out that sparseness becomes an issue if > the vendor driver is largely disconnected from how their feature > addition and deprecation affects migration support. Thanks, > ok. we'll use the x.y.z scheme then. Thanks Yan From moreira.belmiro.email.lists at gmail.com Tue Aug 4 08:55:05 2020 From: moreira.belmiro.email.lists at gmail.com (Belmiro Moreira) Date: Tue, 4 Aug 2020 10:55:05 +0200 Subject: [TC] [PTG] Victoria vPTG Summary of Conversations and Action Items In-Reply-To: References: Message-ID: Hi everyone, the problem described in the "OpenStack User-facing APIs" is something that we face daily in our deployment. Different CLIs for different operations. I'm really interested in driving this action item. Belmiro On Fri, Jun 12, 2020 at 9:38 PM Kendall Nelson wrote: > Hello Everyone! > > I hope you all had a productive and enjoyable PTG! While it’s still > reasonably fresh, I wanted to take a moment to summarize discussions and > actions that came out of TC discussions. > > If there is a particular action item you are interested in taking, please > reply on this thread! > > For the long version, check out the etherpad from the PTG[1]. > > Tuesday > > ====== > > Ussuri Retrospective > > ---------------------------- > > As usual we accomplished a lot. Some of the things we accomplished were > around enumerating operating systems per release (again), removing python2 > support, and adding the ideas repository. Towards the end of the release, > we had a lot of discussions around what to do with leaderless projects, the > role of PTLs, and what to do with projects that were missing PTL candidates > for the next release. We discussed office hours, their history and reason > for existence, and clarified how we can strengthen communication amongst > ourselves, the projects, and the larger community. > > TC Onboarding > > -------------------- > > It was brought up that those elected most recently (and even new members > the election before) felt like there wasn’t enough onboarding into the TC. > Through discussion about what we can do to better support returning members > is to better document the daily, weekly and monthly tasks TC members are > supposed to be doing. Kendall Nelson proposed a patch to start adding more > detail to a guide for TC members already[2]. It was also proposed that we > have a sort of mentorship or shadow program for people interested in > joining the TC or new TC members by more experienced TC members. The > discussion about the shadow/mentorship program is to be continued. > > TC/UC Merge > > ------------------ > > Thierry gave an update on the merge of the committees. The simplified > version is that the current proposal is that UC members are picked from TC > members, the UC operates within the TC, and that we are already setup for > this given the number of TC members that have AUC status. None of this > requires a by-laws change. One next step that has already begun is the > merging of the openstack-users ML into openstack-discuss ML. Other next > steps are to decide when to do the actual transition (disbanding the > separate UC, probably at the next election?) and when to setup AUC’s to be > defined as extra-ATC’s to be included in the electorate for elections. For > more detail, check out the openstack-discuss ML thread[3]. > > Wednesday > > ========= > > Help Wanted List > > ----------------------- > > We settled on a format for the job postings and have several on the list. > We talked about how often we want to look through, update or add to it. The > proposal is to do this yearly. We need to continue pushing on the board to > dedicate contributors at their companies to work on these items, and get > them to understand that it's an investment that will take longer than a > year in a lot of cases; interns are great, but not enough. > > TC Position on Foundation Member Community Contributions > > > ---------------------------------------------------------------------------------- > > The discussion started with a state of things today - the expectations of > platinum members, the benefits the members get being on the board and why > they should donate contributor resources for these benefits, etc. A variety > of proposals were made: either enforce or remove the minimum contribution > level, give gold members the chance to have increased visibility (perhaps > giving them some of the platinum member advantages) if they supplement > their monetary contributions with contributor contributions, etc. The > #ACTION that was decided was for Mohammed to take these ideas to the board > and see what they think. > > OpenStack User-facing APIs > > -------------------------------------- > > Users are confused about the state of the user facing API’s; they’ve been > told to use the OpenStackClient(OSC) but upon use, they discover that there > are features missing that exist in the python-*clients. Partial > implementation in the OSC is worse than if the service only used their > specific CLI. Members of the OpenStackSDK joined discussions and explained > that many of the barriers that projects used to have behind implementing > certain commands have been resolved. The proposal is to create a pop up > team and that they start with fully migrating Nova, documenting the process > and collecting any other unresolved blocking issues with the hope that one > day we can set the migration of the remaining projects as a community goal. > Supplementally, a new idea was proposed- enforcing new functionality to > services is only added to the SDK (and optionally the OSC) and not the > project’s specific CLI to stop increasing the disparity between the two. > The #ACTION here is to start the pop up team, if you are interested, please > reply! Additionally, if you disagree with this kind of enforcement, please > contact the TC as soon as possible and explain your concerns. > > PTL Role in OpenStack today & Leaderless Projects > > --------------------------------------------------------------------- > > This was a veeeeeeeerrrry long conversation that went in circles a few > times. The very short version is that we, the TC, are willing to let > project teams decide for themselves if they want to have a more > deconstructed kind of PTL role by breaking it into someone responsible for > releases and someone responsible for security issues. This new format also > comes with setting the expectation that for things like project updates and > signing up for PTG time, if someone on the team doesn’t actively take that > on, the default assumption is that the project won’t participate. The > #ACTION we need someone to take on is to write a resolution about how this > will work and how it can be done. Ideally, this would be done before the > next technical election, so that teams can choose it at that point. If you > are interested in taking on the writing of this resolution, please speak up! > > Cross Project Work > > ------------------------- > > -Pop Up Teams- > > The two teams we have right now are Encryption and Secure Consistent > Policy Groups. Both are making slow progress and will continue. > > > > -Reducing Community Goals Per Cycle- > > Historically we have had two goals per cycle, but for smaller teams this > can be a HUGE lift. The #ACTION is to clearly outline the documentation for > the goal proposal and selection process to clarify that selecting only one > goal is fine. No one has claimed this action item yet. > > -Victoria Goal Finalization- > > Currently, we have three proposals and one accepted goal. If we are going > to select a second goal, it needs to be done ASAP as Victoria development > has already begun. All TC members should review the last proposal > requesting selection[4]. > > -Wallaby Cycle Goal Discussion Kick Off- > > Firstly, there is a #ACTION that one or two TC members are needed to guide > the W goal selection. If you are interested, please reply to this thread! > There were a few proposed goals for VIctoria that didn’t make it that could > be the starting point for W discussions, in particular, the rootwrap goal > which would be good for operators. The OpenStackCLI might be another goal > to propose for Wallaby. > > Detecting Unmaintained Projects Early > > --------------------------------------------------- > > The TC liaisons program had been created a few releases ago, but the > initial load on TC members was large. We discussed bringing this program > back and making the project health checks happen twice a release, either > the start or end of the release and once in the middle. TC liaisons will > look at previously proposed releases, release activity of the team, the > state of tempest plugins, if regular meetings are happening, if there are > patches in progress and how busy the project’s IRC channel is to make a > determination. Since more than one liaison will be assigned to each > project, those liaisons can divvy up the work how they see fit. The other > aspect that still needs to be decided is where the health checks will be > recorded- in a wiki? In a meeting and meeting logs? That decision is still > to be continued. The current #ACTION currently unassigned is that we need > to assign liaisons for the Victoria cycle and decide when to do the first > health check. > > Friday > > ===== > > Reducing Systems and Friction to Drive Change > > ---------------------------------------------------------------- > > This was another conversation that went in circles a bit before realizing > that we should make a list of the more specific problems we want to address > and then brainstorm solutions for them. The list we created (including > things already being worked) are as follows: > > - > > TC separate from UC (solution in progress) > - > > Stable releases being approved by a separate team (solution in > progress) > - > > Making repository creation faster (especially for established project > teams) > - > > Create a process blueprint for project team mergers > - > > Requirements Team being one person > - > > Stable Team > - > > Consolidate the agent experience > - > > Figure out how to improve project <--> openstack client/sdk > interaction. > > If you feel compelled to pick one of these things up and start proposing > solutions or add to the list, please do! > > Monitoring in OpenStack (Ceilometer + Telemetry + Gnocchi State) > > > ----------------------------------------------------------------------------------------- > > This conversation is also ongoing, but essentially we talked about the > state of things right now- largely they are not well maintained and there > is added complexity with Ceilometers being partially dependent on Gnocchi. > There are a couple of ideas to look into like using oslo.metrics for the > interface between all the tools or using Ceilometer without Gnocchi if we > can clean up those dependencies. No specific action items here, just please > share your thoughts if you have them. > > Ideas Repo Next Steps > > ------------------------------- > > Out of the Ussuri retrospective, it was brought up that we probably needed > to talk a little more about what we wanted for this repo. Essentially we > just want it to be a place to collect ideas into without worrying about the > how. It should be a place to document ideas we have had (old and new) and > keep all the discussion in one place as opposed to historic email threads, > meetings logs, other IRC logs, etc. We decided it would be good to > periodically go through this repo, likely as a forum session at a summit to > see if there is any updating that could happen or promotion of ideas to > community goals, etc. > > ‘tc:approved-release’ Tag > > --------------------------------- > > This topic was proposed by the Manila team from a discussion they had > earlier in the week. We talked about the history of the tag and how usage > of tags has evolved. At this point, the proposal is to remove the tag as > anything in the releases repo is essentially tc-approved. Ghanshyam has > volunteered to document this and do the removal. The board also needs to be > notified of this and to look at projects.yaml in the governance repo as the > source of truth for TC approved projects. The unassigned #ACTION item is to > review remaining tags and see if there are others that need to be > modified/removed/added to drive common behavior across OpenSack > components. > > Board Proposals > > ---------------------- > > This was a pretty quick summary of all discussions we had that had any > impact on the board and largely decided who would mention them. > > > > Session Feedback > > ------------------------ > > This was also a pretty quick topic compared to many of the others, we > talked about how things went across all our discussions (largely we called > the PTG a success) logistically. We tried to make good use of the raising > hands feature which mostly worked, but it lacks context and its possible > that the conversation has moved on by the time it’s your turn (if you even > remember what you want to say). > > OpenStack 2.0: k8s Native > > ----------------------------------- > > This topic was brought up at the end of our time so we didn’t have time to > discuss it really. Basically Mohammed wanted to start the conversation > about adding k8s as a base service[5] and what we would do if a project > proposed required k8s. Adding services that work with k8s could open a door > to new innovation in OpenStack. Obviously this topic will need to be > discussed further as we barely got started before we had to wrap things up. > > > So. > > > The tldr; > > > Here are the #ACTION items we need owners for: > > - > > Start the User Facing API Pop Up Team > - > > Write a resolution about how the deconstructed PTL roles will work > - > > Update Goal Selection docs to explain that one or more goals is fine; > it doesn’t have to be more than one > - > > Two volunteers to start the W goal selection process > - > > Assign two TC liaisons per project > - > > Review Tags to make sure they are still good for driving common > behavior across all openstack projects > > > Here are the things EVERYONE needs to do: > > - > > Review last goal proposal so that we can decide to accept or reject it > for the V release[4] > - > > Add systems that are barriers to progress in openstack to the Reducing > Systems and Friction list > - > > Continue conversations you find important > > > > Thanks everyone for your hard work and great conversations :) > > Enjoy the attached (photoshopped) team photo :) > > -Kendall (diablo_rojo) > > > > [1] TC PTG Etherpad: https://etherpad.opendev.org/p/tc-victoria-ptg > > [2] TC Guide Patch: https://review.opendev.org/#/c/732983/ > > [3] UC TC Merge Thread: > http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014736.html > > > [4] Proposed V Goal: https://review.opendev.org/#/c/731213/ > > [5] Base Service Description: > https://governance.openstack.org/tc/reference/base-services.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Tue Aug 4 12:22:19 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 4 Aug 2020 07:22:19 -0500 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: On 8/4/20 3:17 AM, Fabian Zimmermann wrote: > Hmm, the err msg tells to run the queens version of the tool. > > Maybe something went wrong, but the db version got incremented? Just > guessing. > > Did you try to find the commit/change that introduced the msg? [snip] > Massimo Sgaravatto > schrieb am Mo., > 3. Aug. 2020, 20:21: > > We have just updated a small OpenStack cluster to > Train. > Everything seems working, but "cinder-status > upgrade check" complains that services and volumes > must have a service UUID [*]. > What does this exactly mean? > > Thanks, Massimo > > [*] > +--------------------------------------------------------------------+ > | Check: Service UUIDs                         | > | Result: Failure                          | > | Details: Services and volumes must have a > service UUID. Please fix | > |   this issue by running Queens online data > migrations.             | > Hmm, this does look concerning. If you are now on Train but a migration is missing from Queens, that would seem to indicate some migrations were missed along the way. Were migrations run in each release prior to getting to Train? -------------- next part -------------- An HTML attachment was scrubbed... URL: From geguileo at redhat.com Tue Aug 4 12:26:06 2020 From: geguileo at redhat.com (Gorka Eguileor) Date: Tue, 4 Aug 2020 14:26:06 +0200 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> Message-ID: <20200804122606.dctnfvxqytfv22ws@localhost> On 03/08, Sean McGinnis wrote: > On 8/3/20 7:55 AM, Lee Yarwood wrote: > > Hello all, > > > > $subject, I've raised the following bug: > > > > openstack-tox-lower-constraints failing due to unmet dependency on decorator==4.0.0 > > https://launchpad.net/bugs/1890123 > > > > I'm trying to resolve this below but I honestly feel like I'm going > > around in circles: > > > > https://review.opendev.org/#/q/topic:bug/1890123 > > > > If anyone has any tooling and/or recommendations for resolving issues > > like this I'd appreciate it! > > > > Cheers, > > This appears to be broken for everyone. I initially saw the decorator > thing with Cinder, but after looking closer realized it's not that package. > > The root issue (or at least one level closer to the root issue, that > seems to be causing the decorator failure) is that the lower-constraints > are not actually being enforced. Even though the logs should it is > passing "-c [path to lower-constraints.txt]". So even though things > should be constrained to a lower version, presumably a version that > works with a different version of decorator, pip is still installing a > newer package than what the constraints should allow. > > There was a pip release on the 28th. Things don't look like they started > failing until the 31st for us though, so either that is not it, or there > was just a delay before our nodes started picking up the newer version. > > I tested locally, and at least with version 19.3.1, I am getting the > correctly constrained packages installed. > > Still looking, but thought I would share in case that info triggers any > ideas for anyone else. > > Sean > > Hi, Looking at one of my patches I see that the right version of dogpile.cache==0.6.5 is being installed [1], but then at another step we download [2] and install [3] version 1.0.1, and we can see that pip is actually complaining that we have incompatibilities [4]. As far as I can see this is because in that pip install we requested to wipe existing installed packages [6] and we are not passing any constraints in that call. I don't know why or where we are doing that though. Cheers, Gorka. [1]: https://zuul.opendev.org/t/openstack/build/49f226f8efb94c088cb2b22c46565d97/log/tox/lower-constraints-1.log#235-236 [2]: https://zuul.opendev.org/t/openstack/build/49f226f8efb94c088cb2b22c46565d97/log/tox/lower-constraints-2.log#148-149 [3]: https://zuul.opendev.org/t/openstack/build/49f226f8efb94c088cb2b22c46565d97/log/tox/lower-constraints-2.log#168-174 [4]: https://zuul.opendev.org/t/openstack/build/49f226f8efb94c088cb2b22c46565d97/log/tox/lower-constraints-2.log#202-203 [5]: https://zuul.opendev.org/t/openstack/build/49f226f8efb94c088cb2b22c46565d97/log/tox/lower-constraints-2.log#3 From fungi at yuggoth.org Tue Aug 4 12:39:22 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 4 Aug 2020 12:39:22 +0000 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: <20200804122606.dctnfvxqytfv22ws@localhost> References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> <20200804122606.dctnfvxqytfv22ws@localhost> Message-ID: <20200804123922.tcxglphtn6x2yona@yuggoth.org> On 2020-08-04 14:26:06 +0200 (+0200), Gorka Eguileor wrote: [...] > Looking at one of my patches I see that the right version of > dogpile.cache==0.6.5 is being installed [1], but then at another step we > download [2] and install [3] version 1.0.1, and we can see that pip is > actually complaining that we have incompatibilities [4]. > > As far as I can see this is because in that pip install we requested to > wipe existing installed packages [6] and we are not passing any > constraints in that call. > > I don't know why or where we are doing that though. [...] Yes, I started digging into this yesterday too. It's affecting all tox jobs, not just lower-constraints jobs (upper-constraints is close enough to unconstrained that this isn't immediately apparent for master branch jobs, but the divergence becomes obvious in stable branch jobs and it's breaking lots of them). It seems this started roughly a week ago. I don't think we're explicitly doing it, this seems to be a behavior baked into tox itself. Most projects are currently applying constraints via the deps parameter in their tox.ini, and tox appears to invoke pip twice: once to install your deps, and then a second time to install the project being tested. The latter phase does not use the deps parameter, and so no constraints get applied. We might be able to work around this by going back to overriding install_command and putting the -c option there instead, but I haven't had an opportunity to test that theory yet. If anyone else has time to pursue this line of investigation, I'd be curious to hear whether it helps. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From sean.mcginnis at gmx.com Tue Aug 4 13:01:16 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 4 Aug 2020 08:01:16 -0500 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: References: Message-ID: <199d0036-99c2-0be0-c464-39dcda8368ca@gmx.com> [adding back the ML] On 8/4/20 7:48 AM, Massimo Sgaravatto wrote: > I am afraid I never ran the online data migration > > This cluster ran Ocata > Then we updated to Rocky. We went though Pike and Queens but just to > run the db-syncs > Then we updated from Rocky to train (again, we went though Stein  but > just to run the db-syncs) > > Am I in troubles now ? > > Thanks, Massimo I know some folks were handling parts of this by running each version in a container. That may be an option to quickly go through the DB migrations. Let's see if anyone responds with any tips to make this easy. From massimo.sgaravatto at gmail.com Tue Aug 4 13:08:51 2020 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Tue, 4 Aug 2020 15:08:51 +0200 Subject: [ops] [cinder[ "Services and volumes must have a service UUID." In-Reply-To: <199d0036-99c2-0be0-c464-39dcda8368ca@gmx.com> References: <199d0036-99c2-0be0-c464-39dcda8368ca@gmx.com> Message-ID: Shouldn't the db sync fail if a needed online data migrations was not done ? PS: Updating is becoming a nightmare: some services now require online data migration, while for others only the db syncs should be done. On Tue, Aug 4, 2020 at 3:01 PM Sean McGinnis wrote: > [adding back the ML] > > On 8/4/20 7:48 AM, Massimo Sgaravatto wrote: > > I am afraid I never ran the online data migration > > > > This cluster ran Ocata > > Then we updated to Rocky. We went though Pike and Queens but just to > > run the db-syncs > > Then we updated from Rocky to train (again, we went though Stein but > > just to run the db-syncs) > > > > Am I in troubles now ? > > > > Thanks, Massimo > > I know some folks were handling parts of this by running each version in > a container. That may be an option to quickly go through the DB migrations. > > Let's see if anyone responds with any tips to make this easy. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Aug 4 13:11:03 2020 From: smooney at redhat.com (Sean Mooney) Date: Tue, 04 Aug 2020 14:11:03 +0100 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: <20200804123922.tcxglphtn6x2yona@yuggoth.org> References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> <20200804122606.dctnfvxqytfv22ws@localhost> <20200804123922.tcxglphtn6x2yona@yuggoth.org> Message-ID: <38f40d528b2689996b9114157e2c578c71e37942.camel@redhat.com> On Tue, 2020-08-04 at 12:39 +0000, Jeremy Stanley wrote: > On 2020-08-04 14:26:06 +0200 (+0200), Gorka Eguileor wrote: > [...] > > Looking at one of my patches I see that the right version of > > dogpile.cache==0.6.5 is being installed [1], but then at another step we > > download [2] and install [3] version 1.0.1, and we can see that pip is > > actually complaining that we have incompatibilities [4]. > > > > As far as I can see this is because in that pip install we requested to > > wipe existing installed packages [6] and we are not passing any > > constraints in that call. > > > > I don't know why or where we are doing that though. > > [...] > > Yes, I started digging into this yesterday too. It's affecting all > tox jobs, not just lower-constraints jobs (upper-constraints is > close enough to unconstrained that this isn't immediately apparent > for master branch jobs, but the divergence becomes obvious in stable > branch jobs and it's breaking lots of them). It seems this started > roughly a week ago. > > I don't think we're explicitly doing it, this seems to be a behavior > baked into tox itself. Most projects are currently applying > constraints via the deps parameter in their tox.ini, and tox appears > to invoke pip twice: once to install your deps, and then a second > time to install the project being tested. The latter phase does not > use the deps parameter, and so no constraints get applied. > > We might be able to work around this by going back to overriding > install_command and putting the -c option there instead, right so stephen asked me to remove that override in one of my recent patches to os-vif that is under view since he made the comment the command we were using was more or less the same as the default we currently set teh -c in deps. so if i understand the workaound correclty we woudl add -c {env:CONSTRAINTS_OPT} to install_command so "install_command = pip install -U {opts} {packages} -c {env:CONSTRAINTS_OPT}" in our case and then for the lower contriats jobs in stead of deps = -c{toxinidir}/lower-constraints.txt -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt -r{toxinidir}/doc/requirements.txt we would do setenv = CONSTRAINTS_OPT=-c{toxinidir}/lower-constraints.txt deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt -r{toxinidir}/doc/requirements.txt that way we can keep the same install command for both but use the correct constrint file. > but I > haven't had an opportunity to test that theory yet. If anyone else > has time to pursue this line of investigation, I'd be curious to > hear whether it helps. From fungi at yuggoth.org Tue Aug 4 13:16:43 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 4 Aug 2020 13:16:43 +0000 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: <38f40d528b2689996b9114157e2c578c71e37942.camel@redhat.com> References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> <20200804122606.dctnfvxqytfv22ws@localhost> <20200804123922.tcxglphtn6x2yona@yuggoth.org> <38f40d528b2689996b9114157e2c578c71e37942.camel@redhat.com> Message-ID: <20200804131643.dsdpoea4proojeky@yuggoth.org> On 2020-08-04 14:11:03 +0100 (+0100), Sean Mooney wrote: [...] > so if i understand the workaound correclty we woudl add -c > {env:CONSTRAINTS_OPT} to install_command so "install_command = pip > install -U {opts} {packages} -c {env:CONSTRAINTS_OPT}" in our case > and then for the lower contriats jobs in stead of > > deps = > -c{toxinidir}/lower-constraints.txt > -r{toxinidir}/requirements.txt > -r{toxinidir}/test-requirements.txt > -r{toxinidir}/doc/requirements.txt > > we would do > > setenv = > CONSTRAINTS_OPT=-c{toxinidir}/lower-constraints.txt > deps = > -r{toxinidir}/requirements.txt > -r{toxinidir}/test-requirements.txt > -r{toxinidir}/doc/requirements.txt > > that way we can keep the same install command for both but use the > correct constrint file. [...] Yep, Sean McGinnis is trying a variant of that in https://review.opendev.org/744698 now to see if it alters tox's behavior like we expect. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From rafaelweingartner at gmail.com Tue Aug 4 13:20:06 2020 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Tue, 4 Aug 2020 10:20:06 -0300 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: I am not sure how the projects/communities here in OpenStack are maintained and conducted, but I could for sure help. I am a committer and PMC for some Apache projects; therefore, I am a bit familiar with some processes in OpenSource communities. On Tue, Aug 4, 2020 at 5:11 AM Mark Goddard wrote: > On Thu, 30 Jul 2020 at 14:43, Rafael Weingärtner > wrote: > > > > We are working on it. So far we have 3 open proposals there, but we do > not have enough karma to move things along. > > Besides these 3 open proposals, we do have more ongoing extensions that > have not yet been proposed to the community. > > It's good to hear you want to help improve cloudkitty, however it > sounds like what is required is help with maintaining the project. Is > that something you could be involved with? > Mark > > > > > On Thu, Jul 30, 2020 at 10:22 AM Sean McGinnis > wrote: > >> > >> Posting here to raise awareness, and start discussion about next steps. > >> > >> It appears there is no one working on Cloudkitty anymore. No patches > >> have been merged for several months now, including simple bot proposed > >> patches. It would appear no one is maintaining this project anymore. > >> > >> I know there is a need out there for this type of functionality, so > >> maybe this will raise awareness and get some attention to it. But > >> barring that, I am wondering if we should start the process to retire > >> this project. > >> > >> From a Victoria release perspective, it is milestone-2 week, so we > >> should make a decision if any of the Cloudkitty deliverables should be > >> included in this release or not. We can certainly force releases of > >> whatever is the latest, but I think that is a bit risky since these > >> repos have never merged the job template change for victoria and > >> therefore are not even testing with Python 3.8. That is an official > >> runtime for Victoria, so we run the risk of having issues with the code > >> if someone runs under 3.8 but we have not tested to make sure there are > >> no problems doing so. > >> > >> I am hoping this at least starts the discussion. I will not propose any > >> release patches to remove anything until we have had a chance to discuss > >> the situation. > >> > >> Sean > >> > >> > > > > > > -- > > Rafael Weingärtner > -- Rafael Weingärtner -------------- next part -------------- An HTML attachment was scrubbed... URL: From CAPSEY at augusta.edu Tue Aug 4 13:39:35 2020 From: CAPSEY at augusta.edu (Apsey, Christopher) Date: Tue, 4 Aug 2020 13:39:35 +0000 Subject: [nova] Hyper-V hosts In-Reply-To: <046E9C0290DD9149B106B72FC9156BEA04814461@gmsxchsvr01.thecreation.com> References: <046E9C0290DD9149B106B72FC9156BEA04814461@gmsxchsvr01.thecreation.com> Message-ID: You currently need a hyper-v host that is running at least Windows Insider Build 19640 in order to use Epyc with nested virtualization [1]. See if the beta compute driver works[2]. The hyper-v driver for ussuri has release notes[3], so it should be OK, although I haven't personally tried it. Chris Apsey [1] https://github.com/MicrosoftDocs/Virtualization-Documentation/issues/1276 [2] https://www.cloudbase.it/downloads/HyperVNovaCompute_Beta.msi [3] https://docs.openstack.org/releasenotes/compute-hyperv/ussuri.html -----Original Message----- From: Eric K. Miller Sent: Tuesday, August 4, 2020 1:03 AM To: openstack-discuss at lists.openstack.org Subject: [EXTERNAL] [nova] Hyper-V hosts CAUTION: EXTERNAL SENDER This email originated from an external source. Please exercise caution before opening attachments, clicking links, replying, or providing information to the sender. If you believe it to be fraudulent, contact the AU Cybersecurity Hotline at 72-CYBER (2-9237 / 706-722-9237) or 72CYBER at augusta.edu Hi, I thought I'd look into support of Hyper-V hosts for Windows Server environments, but it looks like the latest cloudbase Windows Hyper-V OpenStack Installer is for Train, and nothing seems to discuss the use of Hyper-V in Windows Server 2019. Has it been abandoned? Is anyone using Hyper-V with OpenStack successfully? One of the reasons we thought we might support it is to provide nested support for VMs with GPUs and/or vGPUs, and thought this would work better than with KVM, specifically with AMD EPYC systems. It seems that when "options kvm-amd nested=1" is used in a modprobe.d config file, Windows machines lock up when started. I think this has been an issue for a while with AMD processors, but thought it was fixed recently (I don't remember where I saw this, though). Would love to hear about any experiences related to Hyper-V and/or nested hypervisor support on AMD EPYC processors. Thanks! Eric From rosmaita.fossdev at gmail.com Tue Aug 4 13:48:37 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Tue, 4 Aug 2020 09:48:37 -0400 Subject: [cinder] victoria virtual mid-cycle next week Message-ID: <64a4d8e5-d271-8b96-eed2-167f8daf900a@gmail.com> The date/time selection poll has closed, and I am happy to announce the unanimous choice. Session Two of the Cinder Victoria virtual mid-cycle will be held: DATE: 12 August 2020 TIME: 1400-1600 UTC LOCATION: https://bluejeans.com/3228528973 The meeting will be recorded. Please add topics to the etherpad: https://etherpad.opendev.org/p/cinder-victoria-mid-cycles cheers, brian From amotoki at gmail.com Tue Aug 4 14:01:44 2020 From: amotoki at gmail.com (Akihiro Motoki) Date: Tue, 4 Aug 2020 23:01:44 +0900 Subject: [neutron] bug deputy report (Jul 27 - Aug 2) Message-ID: Hi, Sorry for late just before the meeting. This is my bug deputy report last week. General questions ================= * l3-dvr-backlog tag was originally introduced to identify DVR feature gaps. What should we use for OVN L3 feature gaps? * We have no volunteer for FWaaS now. How should we triage fwaas bugs? Needs attentions ================ Both affects neutron behaviors and they are not simple bugs. More opinions would be appreciated. https://bugs.launchpad.net/neutron/+bug/1889631 [OVS][FW] Multicast non-IGMP traffic is allowed by default, not in iptables FW New, Undecided It might be worth RFE. It affects existing deployments. We need more detail discussion on this. https://bugs.launchpad.net/neutron/+bug/1889454 br-int has an unpredictable MTU New, Undecided This is an interesting topic. More opinions would be appreciated. Confirmed ========= [FT][Fullstack] Timeout during OVS bridge creation transaction https://bugs.launchpad.net/neutron/+bug/1889453 Critical, Confirmed In Progress =========== Functional tests neutron.tests.functional.agent.linux.test_linuxbridge_arp_protect failing on Ubuntu 20.04 https://bugs.launchpad.net/neutron/+bug/1889779 High, In Progress CI failture: ImportError: cannot import decorate https://bugs.launchpad.net/neutron/+bug/1890064 High, In Progress, The fix is https://review.opendev.org/#/c/744465/ but blocked by other gate failures Validate subnet when plugging to the router don't works when plugging port https://bugs.launchpad.net/neutron/+bug/1889619 Low, In Progress Won't Fix ========= Functional tests on Ubuntu 20.04 are timed out https://bugs.launchpad.net/neutron/+bug/1889781 High, Won't Fix, it doesn't look like an issue after fixing other issues per slawek ONV feature gaps or cleanups ============================ https://bugs.launchpad.net/neutron/+bug/1889737 [OVN] Stop using neutron.api.rpc.handlers.resources_rpc with OVN as a backend Medium, Confirmed, a kind of cleanup around OVN https://bugs.launchpad.net/neutron/+bug/1889738 [OVN] Stop doing PgDelPortCommand on each router port update Low, Confirmed [OVN] access between Floatings ip and instance with Direct External IP https://bugs.launchpad.net/neutron/+bug/1889388 New, Undecided, OVN feature gap Q: l3-dvr-backlog tag was originally introduced to identify DVR feature gaps. What should we use for OVN L3 feature gaps? FWaaS ====== neutron_tempest_plugin.fwaas.api.test_fwaasv2_extensions failed https://bugs.launchpad.net/neutron/+bug/1889730 New, Undecided, we have no volunteer for FWaaS now. How should we triage fwaas bugs? From mnaser at vexxhost.com Tue Aug 4 14:47:51 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Tue, 4 Aug 2020 10:47:51 -0400 Subject: [tc] weekly update Message-ID: Hi everyone, Here’s an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. We've also included a few references to some important mailing list threads that you should check out. # Patches ## Open Reviews - Declare supported runtimes for Wallaby release https://review.opendev.org/743847 - [draft] Add assert:supports-standalone https://review.opendev.org/722399 - Create starter-kit:kubernetes-in-virt tag https://review.opendev.org/736369 - migrate testing to ubuntu focal https://review.opendev.org/740851 ## Project Updates - Add Keystone Kerberos charm to OpenStack charms https://review.opendev.org/743769 - Deprecate os_congress project https://review.opendev.org/742533 - Add Ceph iSCSI charm to OpenStack charms https://review.opendev.org/744480 ## General Changes - Cleanup the remaining osf repos and their data https://review.opendev.org/739291 - [manila] assert:supports-accessible-upgrade https://review.opendev.org/740509 - V goals, Zuul v3 migration: update links and grenade https://review.opendev.org/741987 ## Abandoned Changes - DNM: testing gate on ubuntu focal https://review.opendev.org/743249 # Email Threads - Legacy Zuul Jobs Update 1: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016058.html - Community PyCharm Licenses: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016039.html - Release Countdown R-10: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016243.html - CloudKitty Status: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016171.html - Migrate CI Jobs to Ubuntu Focal Update: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016222.html - TC Monthly Meeting Reminder: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016196.html # Other Reminders - Aug 4: CFP for Open Infra Summit Closes Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc. From mkopec at redhat.com Tue Aug 4 15:10:35 2020 From: mkopec at redhat.com (Martin Kopec) Date: Tue, 4 Aug 2020 17:10:35 +0200 Subject: [openstack][tempest] Deprecation of scenario.img_dir option Message-ID: Hello all, we deprecated *scenario.img_dir* option in Tempest by this patch [1]. *scenario.img_file* should contain the full path of an image from now on. However, to make the transition easier for all the Tempest dependent projects, Tempest currently accepts both behaviors - the old where the path to an image consisted of *scenario.img_dir* + *scenario.img_file* and the new one where *scenario.img_file* contains the full path to an image. It will be accepting both ways for one whole release - 25. I proposed patches to projects I found they use scenario.img_dir option, see this link [2]. If you maintain a project from the list [2], please review. If your project somehow uses scenario.img_dir or img_file option and is not in the list [2], please make appropriate changes. [1] https://review.opendev.org/#/c/710996 [2] https://review.opendev.org/#/q/topic:remove_img_dir+(status:open+OR+status:merged) Regards, -- Martin Kopec Quality Engineer Red Hat EMEA -------------- next part -------------- An HTML attachment was scrubbed... URL: From monika.samal at outlook.com Tue Aug 4 15:28:35 2020 From: monika.samal at outlook.com (Monika Samal) Date: Tue, 4 Aug 2020 15:28:35 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , Message-ID: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann Cc: Monika Samal ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralonsoh at redhat.com Tue Aug 4 17:05:49 2020 From: ralonsoh at redhat.com (Rodolfo Alonso Hernandez) Date: Tue, 4 Aug 2020 18:05:49 +0100 Subject: [neutron][OVS firewall] Multicast non-IGMP traffic is allowed by default, not in iptables FW (LP#1889631) Message-ID: Hello all: First of all, the link: https://bugs.launchpad.net/neutron/+bug/1889631 To sum up the bug: in iptables FW, the non-IGMP multicast traffic from 224.0.0.x was blocked; this is not happening in OVS FW. That was discussed today in the Neutron meeting today [1]. We face two possible situations here: - If we block this traffic now, some deployments using the OVS FW will experience an unexpected network blockage. - Deployments migrating from iptables to OVS FW, now won't be able to explicitly allow this traffic (or block it by default). This also breaks the current API, because some rules won't have any effect (those ones allowing this traffic). A possible solution is to add a new knob in the FW configuration; this config option will allow to block or not this traffic by default. Remember that the FW can only create permissive rules, not blocking ones. Any feedback is welcome! Regards. [1] http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-08-04-14.00.log.html#l-136 -------------- next part -------------- An HTML attachment was scrubbed... URL: From elfosardo at gmail.com Tue Aug 4 17:09:33 2020 From: elfosardo at gmail.com (Riccardo Pittau) Date: Tue, 4 Aug 2020 19:09:33 +0200 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: <20200804131643.dsdpoea4proojeky@yuggoth.org> References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> <20200804122606.dctnfvxqytfv22ws@localhost> <20200804123922.tcxglphtn6x2yona@yuggoth.org> <38f40d528b2689996b9114157e2c578c71e37942.camel@redhat.com> <20200804131643.dsdpoea4proojeky@yuggoth.org> Message-ID: Hi all! After a very interesting and enlightening discussion with Sean and Clark on IRC (thanks!), we were able to test and verify that the issue is related to the latest released version of virtualenv, v2.0.29, that embeds pip 2.20, apparently the real offender here. I submitted a bug to virtualenv [1] for that, the fix is included in pip 2.20.1. The bump in virtualenv is already up [2] and merged and a new version has been released, v2.0.30 [3], that should solve this issue. [1] https://github.com/pypa/virtualenv/issues/1914 [2] https://github.com/pypa/virtualenv/pull/1915 [3] https://pypi.org/project/virtualenv/20.0.30/ A si biri, Riccardo On Tue, Aug 4, 2020 at 3:26 PM Jeremy Stanley wrote: > On 2020-08-04 14:11:03 +0100 (+0100), Sean Mooney wrote: > [...] > > so if i understand the workaound correclty we woudl add -c > > {env:CONSTRAINTS_OPT} to install_command so "install_command = pip > > install -U {opts} {packages} -c {env:CONSTRAINTS_OPT}" in our case > > and then for the lower contriats jobs in stead of > > > > deps = > > -c{toxinidir}/lower-constraints.txt > > -r{toxinidir}/requirements.txt > > -r{toxinidir}/test-requirements.txt > > -r{toxinidir}/doc/requirements.txt > > > > we would do > > > > setenv = > > CONSTRAINTS_OPT=-c{toxinidir}/lower-constraints.txt > > deps = > > -r{toxinidir}/requirements.txt > > -r{toxinidir}/test-requirements.txt > > -r{toxinidir}/doc/requirements.txt > > > > that way we can keep the same install command for both but use the > > correct constrint file. > [...] > > Yep, Sean McGinnis is trying a variant of that in > https://review.opendev.org/744698 now to see if it alters tox's > behavior like we expect. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Tue Aug 4 17:32:00 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 4 Aug 2020 17:32:00 +0000 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> <20200804122606.dctnfvxqytfv22ws@localhost> <20200804123922.tcxglphtn6x2yona@yuggoth.org> <38f40d528b2689996b9114157e2c578c71e37942.camel@redhat.com> <20200804131643.dsdpoea4proojeky@yuggoth.org> Message-ID: <20200804173200.7fkfmnwr3qofwjsp@yuggoth.org> On 2020-08-04 19:09:33 +0200 (+0200), Riccardo Pittau wrote: > After a very interesting and enlightening discussion with Sean and > Clark on IRC (thanks!), we were able to test and verify that the > issue is related to the latest released version of virtualenv, > v2.0.29, that embeds pip 2.20, apparently the real offender here. [...] That was indeed confusing. Until I skimmed virtualenv's changelog it hadn't dawned on me that all the problem libraries had a "." in their names. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Tue Aug 4 17:38:48 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 4 Aug 2020 17:38:48 +0000 Subject: [nova] openstack-tox-lower-constraints broken In-Reply-To: <20200804173200.7fkfmnwr3qofwjsp@yuggoth.org> References: <20200803125522.rjso5tafqzt3sjoh@lyarwood.usersys.redhat.com> <20200804122606.dctnfvxqytfv22ws@localhost> <20200804123922.tcxglphtn6x2yona@yuggoth.org> <38f40d528b2689996b9114157e2c578c71e37942.camel@redhat.com> <20200804131643.dsdpoea4proojeky@yuggoth.org> <20200804173200.7fkfmnwr3qofwjsp@yuggoth.org> Message-ID: <20200804173848.foqxbkdy72z36dcp@yuggoth.org> On 2020-08-04 17:32:00 +0000 (+0000), Jeremy Stanley wrote: > On 2020-08-04 19:09:33 +0200 (+0200), Riccardo Pittau wrote: > > After a very interesting and enlightening discussion with Sean and > > Clark on IRC (thanks!), we were able to test and verify that the > > issue is related to the latest released version of virtualenv, > > v2.0.29, that embeds pip 2.20, apparently the real offender here. > [...] > > That was indeed confusing. Until I skimmed virtualenv's changelog it > hadn't dawned on me that all the problem libraries had a "." in > their names. Er, pip's changelog I meant, of course. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From cohuck at redhat.com Tue Aug 4 16:35:03 2020 From: cohuck at redhat.com (Cornelia Huck) Date: Tue, 4 Aug 2020 18:35:03 +0200 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200729080503.GB28676@joy-OptiPlex-7040> References: <20200713232957.GD5955@joy-OptiPlex-7040> <9bfa8700-91f5-ebb4-3977-6321f0487a63@redhat.com> <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> Message-ID: <20200804183503.39f56516.cohuck@redhat.com> [sorry about not chiming in earlier] On Wed, 29 Jul 2020 16:05:03 +0800 Yan Zhao wrote: > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: (...) > > Based on the feedback we've received, the previously proposed interface > > is not viable. I think there's agreement that the user needs to be > > able to parse and interpret the version information. Using json seems > > viable, but I don't know if it's the best option. Is there any > > precedent of markup strings returned via sysfs we could follow? I don't think encoding complex information in a sysfs file is a viable approach. Quoting Documentation/filesystems/sysfs.rst: "Attributes should be ASCII text files, preferably with only one value per file. It is noted that it may not be efficient to contain only one value per file, so it is socially acceptable to express an array of values of the same type. Mixing types, expressing multiple lines of data, and doing fancy formatting of data is heavily frowned upon." Even though this is an older file, I think these restrictions still apply. > I found some examples of using formatted string under /sys, mostly under > tracing. maybe we can do a similar implementation. > > #cat /sys/kernel/debug/tracing/events/kvm/kvm_mmio/format Note that this is *not* sysfs (anything under debug/ follows different rules anyway!) > > name: kvm_mmio > ID: 32 > format: > field:unsigned short common_type; offset:0; size:2; signed:0; > field:unsigned char common_flags; offset:2; size:1; signed:0; > field:unsigned char common_preempt_count; offset:3; size:1; signed:0; > field:int common_pid; offset:4; size:4; signed:1; > > field:u32 type; offset:8; size:4; signed:0; > field:u32 len; offset:12; size:4; signed:0; > field:u64 gpa; offset:16; size:8; signed:0; > field:u64 val; offset:24; size:8; signed:0; > > print fmt: "mmio %s len %u gpa 0x%llx val 0x%llx", __print_symbolic(REC->type, { 0, "unsatisfied-read" }, { 1, "read" }, { 2, "write" }), REC->len, REC->gpa, REC->val > > > #cat /sys/devices/pci0000:00/0000:00:02.0/uevent 'uevent' can probably be considered a special case, I would not really want to copy it. > DRIVER=vfio-pci > PCI_CLASS=30000 > PCI_ID=8086:591D > PCI_SUBSYS_ID=8086:2212 > PCI_SLOT_NAME=0000:00:02.0 > MODALIAS=pci:v00008086d0000591Dsv00008086sd00002212bc03sc00i00 > (...) > what about a migration_compatible attribute under device node like > below? > > #cat /sys/bus/pci/devices/0000\:00\:02.0/UUID1/migration_compatible > SELF: > device_type=pci > device_id=8086591d > mdev_type=i915-GVTg_V5_2 > aggregator=1 > pv_mode="none+ppgtt+context" > interface_version=3 > COMPATIBLE: > device_type=pci > device_id=8086591d > mdev_type=i915-GVTg_V5_{val1:int:1,2,4,8} > aggregator={val1}/2 > pv_mode={val2:string:"none+ppgtt","none+context","none+ppgtt+context"} > interface_version={val3:int:2,3} > COMPATIBLE: > device_type=pci > device_id=8086591d > mdev_type=i915-GVTg_V5_{val1:int:1,2,4,8} > aggregator={val1}/2 > pv_mode="" #"" meaning empty, could be absent in a compatible device > interface_version=1 I'd consider anything of a comparable complexity to be a big no-no. If anything, this needs to be split into individual files (with many of them being vendor driver specific anyway.) I think we can list compatible versions in a range/list format, though. Something like cat interface_version 2.1.3 cat interface_version_compatible 2.0.2-2.0.4,2.1.0- (indicating that versions 2.0.{2,3,4} and all versions after 2.1.0 are compatible, considering versions <2 and >2 incompatible by default) Possible compatibility between different mdev types feels a bit odd to me, and should not be included by default (only if it makes sense for a particular vendor driver.) From melwittt at gmail.com Tue Aug 4 21:08:46 2020 From: melwittt at gmail.com (melanie witt) Date: Tue, 4 Aug 2020 14:08:46 -0700 Subject: [placement][gate] functional tests failing Message-ID: <6486f281-5124-4566-af62-55c8a71905bf@gmail.com> Hi all, I recently proposed a change to openstack/placement and found that the functional tests are currently failing. It's because of a recent-ish bump to upper-constraints to allow os-traits 2.4.0: https://review.opendev.org/739330 and placement has a func test that asserts the number of standard traits (more traits are available in 2.4.0). I've proposed a fix for the func test here if anyone could please help review: https://review.opendev.org/744790 Cheers, -melanie From kennelson11 at gmail.com Tue Aug 4 21:22:04 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Tue, 4 Aug 2020 14:22:04 -0700 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: I think the majority of 'maintenance' activities at the moment for Cloudkitty are the reviewing of open patches in gerrit [1] and triaging bugs that are reported in Launchpad[2] as they come in. When things come up on this mailing list that have the cloudkitty tag in the subject line (like this email), weighing in on them would also be helpful. If you need help getting setup with gerrit, I am happy to assist anyway I can :) -Kendall Nelson (diablo_rojo) [1] https://review.opendev.org/#/q/project:openstack/cloudkitty+OR+project:openstack/python-cloudkittyclient+OR+project:openstack/cloudkitty-dashboard [2] https://launchpad.net/cloudkitty On Tue, Aug 4, 2020 at 6:21 AM Rafael Weingärtner < rafaelweingartner at gmail.com> wrote: > I am not sure how the projects/communities here in OpenStack are > maintained and conducted, but I could for sure help. > I am a committer and PMC for some Apache projects; therefore, I am a bit > familiar with some processes in OpenSource communities. > > On Tue, Aug 4, 2020 at 5:11 AM Mark Goddard wrote: > >> On Thu, 30 Jul 2020 at 14:43, Rafael Weingärtner >> wrote: >> > >> > We are working on it. So far we have 3 open proposals there, but we do >> not have enough karma to move things along. >> > Besides these 3 open proposals, we do have more ongoing extensions that >> have not yet been proposed to the community. >> >> It's good to hear you want to help improve cloudkitty, however it >> sounds like what is required is help with maintaining the project. Is >> that something you could be involved with? >> Mark >> >> > >> > On Thu, Jul 30, 2020 at 10:22 AM Sean McGinnis >> wrote: >> >> >> >> Posting here to raise awareness, and start discussion about next steps. >> >> >> >> It appears there is no one working on Cloudkitty anymore. No patches >> >> have been merged for several months now, including simple bot proposed >> >> patches. It would appear no one is maintaining this project anymore. >> >> >> >> I know there is a need out there for this type of functionality, so >> >> maybe this will raise awareness and get some attention to it. But >> >> barring that, I am wondering if we should start the process to retire >> >> this project. >> >> >> >> From a Victoria release perspective, it is milestone-2 week, so we >> >> should make a decision if any of the Cloudkitty deliverables should be >> >> included in this release or not. We can certainly force releases of >> >> whatever is the latest, but I think that is a bit risky since these >> >> repos have never merged the job template change for victoria and >> >> therefore are not even testing with Python 3.8. That is an official >> >> runtime for Victoria, so we run the risk of having issues with the code >> >> if someone runs under 3.8 but we have not tested to make sure there are >> >> no problems doing so. >> >> >> >> I am hoping this at least starts the discussion. I will not propose any >> >> release patches to remove anything until we have had a chance to >> discuss >> >> the situation. >> >> >> >> Sean >> >> >> >> >> > >> > >> > -- >> > Rafael Weingärtner >> > > > -- > Rafael Weingärtner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.king at gmail.com Tue Aug 4 19:41:48 2020 From: thomas.king at gmail.com (Thomas King) Date: Tue, 4 Aug 2020 13:41:48 -0600 Subject: [Openstack-mentoring] Neutron subnet with DHCP relay - continued In-Reply-To: References: Message-ID: Changing the ml2 flat_networks from specific physical networks to a wildcard allowed me to create a segment. I may be unstuck. New config: [ml2_type_flat] flat_networks=* Now to try creating the subnet and try a remote provision. Tom King On Mon, Aug 3, 2020 at 3:58 PM Thomas King wrote: > I've been using named physical networks so long, I completely forgot using > wildcards! > > Is this the answer???? > > https://docs.openstack.org/mitaka/config-reference/networking/networking_options_reference.html#modular-layer-2-ml2-flat-type-configuration-options > > Tom King > > On Tue, Jul 28, 2020 at 3:46 PM Thomas King wrote: > >> Ruslanas has been a tremendous help. To catch up the discussion lists... >> 1. I enabled Neutron segments. >> 2. I renamed the existing segments for each network so they'll make >> sense. >> 3. I attempted to create a segment for a remote subnet (it is using DHCP >> relay) and this was the error that is blocking me. This is where the docs >> do not cover: >> [root at sea-maas-controller ~(keystone_admin)]# openstack network segment >> create --physical-network remote146-30-32 --network-type flat --network >> baremetal seg-remote-146-30-32 >> BadRequestException: 400: Client Error for url: >> http://10.146.30.65:9696/v2.0/segments, Invalid input for operation: >> physical_network 'remote146-30-32' unknown for flat provider network. >> >> I've asked Ruslanas to clarify how their physical networks correspond to >> their remote networks. They have a single provider network and multiple >> segments tied to multiple physical networks. >> >> However, if anyone can shine some light on this, I would greatly >> appreciate it. How should neutron's configurations accommodate remote >> networks<->Neutron segments when I have only one physical network >> attachment for provisioning? >> >> Thanks! >> Tom King >> >> On Wed, Jul 15, 2020 at 3:33 PM Thomas King >> wrote: >> >>> That helps a lot, thank you! >>> >>> "I use only one network..." >>> This bit seems to go completely against the Neutron segments >>> documentation. When you have access, please let me know if Triple-O is >>> using segments or some other method. >>> >>> I greatly appreciate this, this is a tremendous help. >>> >>> Tom King >>> >>> On Wed, Jul 15, 2020 at 1:07 PM Ruslanas Gžibovskis >>> wrote: >>> >>>> Hi Thomas, >>>> >>>> I have a bit complicated setup from tripleo side :) I use only one >>>> network (only ControlPlane). thanks to Harold, he helped to make it work >>>> for me. >>>> >>>> Yes, as written in the tripleo docs for leaf networks, it use the same >>>> neutron network, different subnets. so neutron network is ctlplane (I >>>> think) and have ctlplane-subnet, remote-provision and remote-KI :)) that >>>> generates additional lines in "ip r s" output for routing "foreign" subnets >>>> through correct gw, if you would have isolated networks, by vlans and ports >>>> this would apply for each subnet different gw... I believe you >>>> know/understand that part. >>>> >>>> remote* subnets have dhcp-relay setup by network team... do not ask >>>> details for that. I do not know how to, but can ask :) >>>> >>>> >>>> in undercloud/tripleo i have 2 dhcp servers, one is for introspection, >>>> another for provide/cleanup and deployment process. >>>> >>>> all of those subnets have organization level tagged networks and are >>>> tagged on network devices, but they are untagged on provisioning >>>> interfaces/ports, as in general pxe should be untagged, but some nic's can >>>> do vlan untag on nic/bios level. but who cares!? >>>> >>>> I just did a brief check on your first post, I think I have simmilar >>>> setup to yours :)) I will check in around 12hours :)) more deaply, as will >>>> be at work :))) >>>> >>>> >>>> P.S. sorry for wrong terms, I am bad at naming. >>>> >>>> >>>> On Wed, 15 Jul 2020, 21:13 Thomas King, wrote: >>>> >>>>> Ruslanas, that would be excellent! >>>>> >>>>> I will reply to you directly for details later unless the maillist >>>>> would like the full thread. >>>>> >>>>> Some preliminary questions: >>>>> >>>>> - Do you have a separate physical interface for the segment(s) >>>>> used for your remote subnets? >>>>> The docs state each segment must have a unique physical network >>>>> name, which suggests a separate physical interface for each segment unless >>>>> I'm misunderstanding something. >>>>> - Are your provisioning segments all on the same Neutron network? >>>>> - Are you using tagged switchports or access switchports to your >>>>> Ironic server(s)? >>>>> >>>>> Thanks, >>>>> Tom King >>>>> >>>>> On Wed, Jul 15, 2020 at 12:26 AM Ruslanas Gžibovskis >>>>> wrote: >>>>> >>>>>> I have deployed that with tripleO, but now we are recabling and >>>>>> redeploying it. So once I have it running I can share my configs, just name >>>>>> which you want :) >>>>>> >>>>>> On Tue, 14 Jul 2020 at 18:40, Thomas King >>>>>> wrote: >>>>>> >>>>>>> I have. That's the Triple-O docs and they don't go through the >>>>>>> normal .conf files to explain how it works outside of Triple-O. It has some >>>>>>> ideas but no running configurations. >>>>>>> >>>>>>> Tom King >>>>>>> >>>>>>> On Tue, Jul 14, 2020 at 3:01 AM Ruslanas Gžibovskis < >>>>>>> ruslanas at lpic.lt> wrote: >>>>>>> >>>>>>>> hi, have you checked: >>>>>>>> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/routed_spine_leaf_network.html >>>>>>>> ? >>>>>>>> I am following this link. I only have one network, having different >>>>>>>> issues tho ;) >>>>>>>> >>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From emiller at genesishosting.com Tue Aug 4 23:21:35 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Tue, 4 Aug 2020 18:21:35 -0500 Subject: [nova] Hyper-V hosts In-Reply-To: References: <046E9C0290DD9149B106B72FC9156BEA04814461@gmsxchsvr01.thecreation.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0481446B@gmsxchsvr01.thecreation.com> > You currently need a hyper-v host that is running at least Windows Insider > Build 19640 in order to use Epyc with nested virtualization [1]. See if the beta > compute driver works[2]. The hyper-v driver for ussuri has release notes[3], > so it should be OK, although I haven't personally tried it. > > Chris Apsey Thank you Chris! I must have seen the Windows Insider Build notes somewhere about this. Thanks for the link! Glad to see that development continues on the cloudbase components. We'll take a test run with this in the near future when we have some hardware dedicated to this. Eric From thomas.king at gmail.com Tue Aug 4 22:22:11 2020 From: thomas.king at gmail.com (Thomas King) Date: Tue, 4 Aug 2020 16:22:11 -0600 Subject: [Openstack-mentoring] Neutron subnet with DHCP relay - continued In-Reply-To: References: Message-ID: Getting closer. I was able to create the segment and the subnet for the remote network on that segment. When I attempted to provide the baremetal node, Neutron is unable to create/attach a port to the remote node: WARNING ironic.common.neutron [req-b3f373fc-e76a-4c13-9ebb-41cfc682d31b 4946f15716c04f8585d013e364802c6c 1664a38fc668432ca6bee9189be142d9 - default default] The local_link_connection is required for 'neutron' network interface and is not present in the nodes 3ed87e51-00c5-4b27-95c0-665c8337e49b port ccc335c6-3521-48a5-927d-d7ee13f7f05b I changed its network interface from neutron back to flat and it went past this. I'm now waiting to see if the node will PXE boot. On Tue, Aug 4, 2020 at 1:41 PM Thomas King wrote: > Changing the ml2 flat_networks from specific physical networks to a > wildcard allowed me to create a segment. I may be unstuck. > > New config: > [ml2_type_flat] > flat_networks=* > > Now to try creating the subnet and try a remote provision. > > Tom King > > On Mon, Aug 3, 2020 at 3:58 PM Thomas King wrote: > >> I've been using named physical networks so long, I completely forgot >> using wildcards! >> >> Is this the answer???? >> >> https://docs.openstack.org/mitaka/config-reference/networking/networking_options_reference.html#modular-layer-2-ml2-flat-type-configuration-options >> >> Tom King >> >> On Tue, Jul 28, 2020 at 3:46 PM Thomas King >> wrote: >> >>> Ruslanas has been a tremendous help. To catch up the discussion lists... >>> 1. I enabled Neutron segments. >>> 2. I renamed the existing segments for each network so they'll make >>> sense. >>> 3. I attempted to create a segment for a remote subnet (it is using DHCP >>> relay) and this was the error that is blocking me. This is where the docs >>> do not cover: >>> [root at sea-maas-controller ~(keystone_admin)]# openstack network segment >>> create --physical-network remote146-30-32 --network-type flat --network >>> baremetal seg-remote-146-30-32 >>> BadRequestException: 400: Client Error for url: >>> http://10.146.30.65:9696/v2.0/segments, Invalid input for operation: >>> physical_network 'remote146-30-32' unknown for flat provider network. >>> >>> I've asked Ruslanas to clarify how their physical networks correspond to >>> their remote networks. They have a single provider network and multiple >>> segments tied to multiple physical networks. >>> >>> However, if anyone can shine some light on this, I would greatly >>> appreciate it. How should neutron's configurations accommodate remote >>> networks<->Neutron segments when I have only one physical network >>> attachment for provisioning? >>> >>> Thanks! >>> Tom King >>> >>> On Wed, Jul 15, 2020 at 3:33 PM Thomas King >>> wrote: >>> >>>> That helps a lot, thank you! >>>> >>>> "I use only one network..." >>>> This bit seems to go completely against the Neutron segments >>>> documentation. When you have access, please let me know if Triple-O is >>>> using segments or some other method. >>>> >>>> I greatly appreciate this, this is a tremendous help. >>>> >>>> Tom King >>>> >>>> On Wed, Jul 15, 2020 at 1:07 PM Ruslanas Gžibovskis >>>> wrote: >>>> >>>>> Hi Thomas, >>>>> >>>>> I have a bit complicated setup from tripleo side :) I use only one >>>>> network (only ControlPlane). thanks to Harold, he helped to make it work >>>>> for me. >>>>> >>>>> Yes, as written in the tripleo docs for leaf networks, it use the same >>>>> neutron network, different subnets. so neutron network is ctlplane (I >>>>> think) and have ctlplane-subnet, remote-provision and remote-KI :)) that >>>>> generates additional lines in "ip r s" output for routing "foreign" subnets >>>>> through correct gw, if you would have isolated networks, by vlans and ports >>>>> this would apply for each subnet different gw... I believe you >>>>> know/understand that part. >>>>> >>>>> remote* subnets have dhcp-relay setup by network team... do not ask >>>>> details for that. I do not know how to, but can ask :) >>>>> >>>>> >>>>> in undercloud/tripleo i have 2 dhcp servers, one is for introspection, >>>>> another for provide/cleanup and deployment process. >>>>> >>>>> all of those subnets have organization level tagged networks and are >>>>> tagged on network devices, but they are untagged on provisioning >>>>> interfaces/ports, as in general pxe should be untagged, but some nic's can >>>>> do vlan untag on nic/bios level. but who cares!? >>>>> >>>>> I just did a brief check on your first post, I think I have simmilar >>>>> setup to yours :)) I will check in around 12hours :)) more deaply, as will >>>>> be at work :))) >>>>> >>>>> >>>>> P.S. sorry for wrong terms, I am bad at naming. >>>>> >>>>> >>>>> On Wed, 15 Jul 2020, 21:13 Thomas King, wrote: >>>>> >>>>>> Ruslanas, that would be excellent! >>>>>> >>>>>> I will reply to you directly for details later unless the maillist >>>>>> would like the full thread. >>>>>> >>>>>> Some preliminary questions: >>>>>> >>>>>> - Do you have a separate physical interface for the segment(s) >>>>>> used for your remote subnets? >>>>>> The docs state each segment must have a unique physical network >>>>>> name, which suggests a separate physical interface for each segment unless >>>>>> I'm misunderstanding something. >>>>>> - Are your provisioning segments all on the same Neutron network? >>>>>> - Are you using tagged switchports or access switchports to your >>>>>> Ironic server(s)? >>>>>> >>>>>> Thanks, >>>>>> Tom King >>>>>> >>>>>> On Wed, Jul 15, 2020 at 12:26 AM Ruslanas Gžibovskis < >>>>>> ruslanas at lpic.lt> wrote: >>>>>> >>>>>>> I have deployed that with tripleO, but now we are recabling and >>>>>>> redeploying it. So once I have it running I can share my configs, just name >>>>>>> which you want :) >>>>>>> >>>>>>> On Tue, 14 Jul 2020 at 18:40, Thomas King >>>>>>> wrote: >>>>>>> >>>>>>>> I have. That's the Triple-O docs and they don't go through the >>>>>>>> normal .conf files to explain how it works outside of Triple-O. It has some >>>>>>>> ideas but no running configurations. >>>>>>>> >>>>>>>> Tom King >>>>>>>> >>>>>>>> On Tue, Jul 14, 2020 at 3:01 AM Ruslanas Gžibovskis < >>>>>>>> ruslanas at lpic.lt> wrote: >>>>>>>> >>>>>>>>> hi, have you checked: >>>>>>>>> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/routed_spine_leaf_network.html >>>>>>>>> ? >>>>>>>>> I am following this link. I only have one network, having >>>>>>>>> different issues tho ;) >>>>>>>>> >>>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From melwittt at gmail.com Wed Aug 5 01:50:40 2020 From: melwittt at gmail.com (melanie witt) Date: Tue, 4 Aug 2020 18:50:40 -0700 Subject: [placement][gate] functional tests failing In-Reply-To: <6486f281-5124-4566-af62-55c8a71905bf@gmail.com> References: <6486f281-5124-4566-af62-55c8a71905bf@gmail.com> Message-ID: On 8/4/20 14:08, melanie witt wrote: > Hi all, > > I recently proposed a change to openstack/placement and found that the > functional tests are currently failing. It's because of a recent-ish > bump to upper-constraints to allow os-traits 2.4.0: > > https://review.opendev.org/739330 > > and placement has a func test that asserts the number of standard traits > (more traits are available in 2.4.0). > > I've proposed a fix for the func test here if anyone could please help > review: > > https://review.opendev.org/744790 The fix has merged and the placement gate is all clear! -melanie From jasonanderson at uchicago.edu Wed Aug 5 03:49:55 2020 From: jasonanderson at uchicago.edu (Jason Anderson) Date: Wed, 5 Aug 2020 03:49:55 +0000 Subject: [swift][ceph] Container ACLs don't seem to be respected on Ceph RGW Message-ID: <757BCAB6-CA22-439E-9C0C-BE4DEC7B7927@uchicago.edu> Hi all, Just scratching my head at this for a while and though I’d ask here in case it saves some time. I’m running a Ceph cluster on the Nautilus release and it’s running Swift via the rgw. I have Keystone authentication turned on. Everything works fine in the normal case of creating containers, uploading files, listing containers, etc. However, I notice that ACLs don’t seem to work. I am not overriding "rgw enforce swift acls”, so it is set to the default of true. I can’t seem to share a container or make it public. (Side note, confusingly, the Ceph implementation has a different syntax for public read/write containers, ‘*’ as opposed to ‘*:*’ for public write for example.) Here’s what I’m doing (as admin) swift post —write-acl ‘*’ —read-acl ‘*’ public-container swift stat public-container Account: v1 Container: public-container Objects: 1 Bytes: 5801 Read ACL: * Write ACL: * Sync To: Sync Key: X-Timestamp: 1595883106.23179 X-Container-Bytes-Used-Actual: 8192 X-Storage-Policy: default-placement X-Storage-Class: STANDARD Last-Modified: Wed, 05 Aug 2020 03:42:11 GMT X-Trans-Id: tx000000000000000662156-005f2a2bea-23478-default X-Openstack-Request-Id: tx000000000000000662156-005f2a2bea-23478-default Accept-Ranges: bytes Content-Type: text/plain; charset=utf-8 (as non-admin) swift upload public-container test.txt Warning: failed to create container 'public-container': 409 Conflict: BucketAlreadyExists Object HEAD failed: https://ceph.example.org:7480/swift/v1/public-container/README.md 403 Forbidden swift list public-container Container GET failed: https://ceph.example.org:7480/swift/v1/public-container?format=json 403 Forbidden [first 60 chars of response] b'{"Code":"AccessDenied","BucketName”:”public-container","RequestId":"tx0' Failed Transaction ID: tx000000000000000662162-005f2a2c2a-23478-default What am I missing? Thanks in advance! /Jason From mark at stackhpc.com Wed Aug 5 07:53:20 2020 From: mark at stackhpc.com (Mark Goddard) Date: Wed, 5 Aug 2020 08:53:20 +0100 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: On Tue, 4 Aug 2020 at 16:58, Monika Samal wrote: > Hello Guys, > > With Michaels help I was able to solve the problem but now there is > another error I was able to create my network on vlan but still error > persist. PFB the logs: > > http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ > > Kindly help > > regards, > Monika > ------------------------------ > *From:* Michael Johnson > *Sent:* Monday, August 3, 2020 9:10 PM > *To:* Fabian Zimmermann > *Cc:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Yeah, it looks like nova is failing to boot the instance. > > Check this setting in your octavia.conf files: > https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id > > Also, if kolla-ansible didn't set both of these values correctly, please > open bug reports for kolla-ansible. These all should have been configured > by the deployment tool. > > I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael > > On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: > > Seems like the flavor is missing or empty '' - check for typos and enable > debug. > > Check if the nova req contains valid information/flavor. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 15:46: > > It's registered > > Get Outlook for Android > ------------------------------ > *From:* Fabian Zimmermann > *Sent:* Monday, August 3, 2020 7:08:21 PM > *To:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Did you check the (nova) flavor you use in octavia. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 10:53: > > After Michael suggestion I was able to create load balancer but there is > error in status. > > > > PFB the error link: > > http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ > ------------------------------ > *From:* Monika Samal > *Sent:* Monday, August 3, 2020 2:08 PM > *To:* Michael Johnson > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Thanks a ton Michael for helping me out > ------------------------------ > *From:* Michael Johnson > *Sent:* Friday, July 31, 2020 3:57 AM > *To:* Monika Samal > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Just to close the loop on this, the octavia.conf file had > "project_name = admin" instead of "project_name = service" in the > [service_auth] section. This was causing the keystone errors when > Octavia was communicating with neutron. > > I don't know if that is a bug in kolla-ansible or was just a local > configuration issue. > > Michael > > On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > > > Hello Fabian,, > > > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > > > Regards, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > Hi, > > > > just to debug, could you replace the auth_type password with v3password? > > > > And do a curl against your :5000 and :35357 urls and paste the output. > > > > Fabian > > > > Monika Samal schrieb am Do., 30. Juli 2020, > 22:15: > > > > Hello Fabian, > > > > http://paste.openstack.org/show/796477/ > > > > Thanks, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > The sections should be > > > > service_auth > > keystone_authtoken > > > > if i read the docs correctly. Maybe you can just paste your config > (remove/change passwords) to paste.openstack.org and post the link? > > > > Fabian > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emiller at genesishosting.com Wed Aug 5 09:41:10 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 5 Aug 2020 04:41:10 -0500 Subject: [cinder][nova] Local storage in compute node Message-ID: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> Hi, I'm research methods to get around high storage latency for some applications where redundancy does not matter, so using local NVMe drives in compute nodes seems to be the practical choice. However, there does not appear to be a good solution from what I have read. For example, BlockDeviceDriver has been deprecated/removed, LVM is only supported via iSCSI (which is slow) and localization of LVM volumes onto the same compute node as VMs is impossible, and other methods (PCI pass-through, etc.) would require direct access to the local drives, where device cleansing would need to occur after a device was removed from a VM, and I don't believe there is a hook for this. Ephemeral storage appears to be an option, but I believe it has the same issue as PCI pass-through, in that there is no abiilty to automatically cleanse a device after it has been used. In our default configuration, ephemeral storage is redirected to use Ceph, which solves the cleansing issue, but isn't suitable due to its high latency. Also, ephemeral storage appears as a second device, not the root disk, so that complicates a few configurations we have. Is there any other way to write an operating system image onto a local drive and boot from it? Or preferably assign an LVM /dev/mapper path as a device in libvirt (no iSCSI) after configuring a logical volume? or am I missing something? Thanks! Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From emiller at genesishosting.com Wed Aug 5 10:03:29 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 5 Aug 2020 05:03:29 -0500 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> In case this is the answer, I found that in nova.conf, under the [libvirt] stanza, images_type can be set to "lvm". This looks like it may do the trick - using the compute node's LVM to provision and mount a logical volume, for either persistent or ephemeral storage defined in the flavor. Can anyone validate that this is the right approach according to our needs? Also, I have read about the LVM device filters - which is important to avoid the host's LVM from seeing the guest's volumes, in case anyone else finds this message. Thanks! Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From lyarwood at redhat.com Wed Aug 5 11:19:34 2020 From: lyarwood at redhat.com (Lee Yarwood) Date: Wed, 5 Aug 2020 12:19:34 +0100 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> Message-ID: <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> On 05-08-20 05:03:29, Eric K. Miller wrote: > In case this is the answer, I found that in nova.conf, under the > [libvirt] stanza, images_type can be set to "lvm". This looks like it > may do the trick - using the compute node's LVM to provision and mount a > logical volume, for either persistent or ephemeral storage defined in > the flavor. > > Can anyone validate that this is the right approach according to our > needs? I'm not sure if it is given your initial requirements. Do you need full host block devices to be provided to the instance? The LVM imagebackend will just provision LVs on top of the provided VG so there's no direct mapping to a full host block device with this approach. That said there's no real alternative available at the moment. > Also, I have read about the LVM device filters - which is important to > avoid the host's LVM from seeing the guest's volumes, in case anyone > else finds this message. Yeah that's a common pitfall when using LVM based ephemeral disks that contain additional LVM PVs/VGs/LVs etc. You need to ensure that the host is configured to not scan these LVs in order for their PVs/VGs/LVs etc to remain hidden from the host: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_filters -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From smooney at redhat.com Wed Aug 5 11:40:34 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 05 Aug 2020 12:40:34 +0100 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> Message-ID: <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> On Wed, 2020-08-05 at 12:19 +0100, Lee Yarwood wrote: > On 05-08-20 05:03:29, Eric K. Miller wrote: > > In case this is the answer, I found that in nova.conf, under the > > [libvirt] stanza, images_type can be set to "lvm". This looks like it > > may do the trick - using the compute node's LVM to provision and mount a > > logical volume, for either persistent or ephemeral storage defined in > > the flavor. > > > > Can anyone validate that this is the right approach according to our > > needs? > > I'm not sure if it is given your initial requirements. > > Do you need full host block devices to be provided to the instance? > > The LVM imagebackend will just provision LVs on top of the provided VG > so there's no direct mapping to a full host block device with this > approach. > > That said there's no real alternative available at the moment. well one alternitive to nova providing local lvm storage is to use the cinder lvm driver but install it on all compute nodes then use the cidner InstanceLocalityFilter to ensure the volume is alocated form the host the vm is on. https://docs.openstack.org/cinder/latest/configuration/block-storage/scheduler-filters.html#instancelocalityfilter on drawback to this is that if the if the vm is moved i think you would need to also migrate the cinder volume seperatly afterwards. > > > Also, I have read about the LVM device filters - which is important to > > avoid the host's LVM from seeing the guest's volumes, in case anyone > > else finds this message. > > > Yeah that's a common pitfall when using LVM based ephemeral disks that > contain additional LVM PVs/VGs/LVs etc. You need to ensure that the host > is configured to not scan these LVs in order for their PVs/VGs/LVs etc > to remain hidden from the host: > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_filters  > > From smooney at redhat.com Wed Aug 5 11:45:47 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 05 Aug 2020 12:45:47 +0100 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> Message-ID: <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> On Wed, 2020-08-05 at 12:40 +0100, Sean Mooney wrote: > On Wed, 2020-08-05 at 12:19 +0100, Lee Yarwood wrote: > > On 05-08-20 05:03:29, Eric K. Miller wrote: > > > In case this is the answer, I found that in nova.conf, under the > > > [libvirt] stanza, images_type can be set to "lvm". This looks like it > > > may do the trick - using the compute node's LVM to provision and mount a > > > logical volume, for either persistent or ephemeral storage defined in > > > the flavor. > > > > > > Can anyone validate that this is the right approach according to our > > > needs? > > > > I'm not sure if it is given your initial requirements. > > > > Do you need full host block devices to be provided to the instance? > > > > The LVM imagebackend will just provision LVs on top of the provided VG > > so there's no direct mapping to a full host block device with this > > approach. > > > > That said there's no real alternative available at the moment. > > well one alternitive to nova providing local lvm storage is to use > the cinder lvm driver but install it on all compute nodes then > use the cidner InstanceLocalityFilter to ensure the volume is alocated form the host > the vm is on. > https://docs.openstack.org/cinder/latest/configuration/block-storage/scheduler-filters.html#instancelocalityfilter > on drawback to this is that if the if the vm is moved i think you would need to also migrate the cinder volume > seperatly afterwards. by the way if you were to take this approch i think there is an nvmeof driver so you can use nvme over rdma instead of iscsi. > > > > > > Also, I have read about the LVM device filters - which is important to > > > avoid the host's LVM from seeing the guest's volumes, in case anyone > > > else finds this message. > > > > > > Yeah that's a common pitfall when using LVM based ephemeral disks that > > contain additional LVM PVs/VGs/LVs etc. You need to ensure that the host > > is configured to not scan these LVs in order for their PVs/VGs/LVs etc > > to remain hidden from the host: > > > > > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_filters > > > > > > > From donny at fortnebula.com Wed Aug 5 12:36:18 2020 From: donny at fortnebula.com (Donny Davis) Date: Wed, 5 Aug 2020 08:36:18 -0400 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> Message-ID: I use local nvme to drive the CI workload for the openstack community for the last year or so. It seems to work pretty well. I just created a filesystem (xfs) and mounted it to /var/lib/nova/instances I moved glance to using my swift backend and it really made the download of the images much faster. It depends on if the workload is going to handle HA or you are expecting to migrate machines. If the workload is ephemeral or HA can be handled in the app I think local storage is still a very viable option. Simpler is better IMO On Wed, Aug 5, 2020 at 7:48 AM Sean Mooney wrote: > On Wed, 2020-08-05 at 12:40 +0100, Sean Mooney wrote: > > On Wed, 2020-08-05 at 12:19 +0100, Lee Yarwood wrote: > > > On 05-08-20 05:03:29, Eric K. Miller wrote: > > > > In case this is the answer, I found that in nova.conf, under the > > > > [libvirt] stanza, images_type can be set to "lvm". This looks like > it > > > > may do the trick - using the compute node's LVM to provision and > mount a > > > > logical volume, for either persistent or ephemeral storage defined in > > > > the flavor. > > > > > > > > Can anyone validate that this is the right approach according to our > > > > needs? > > > > > > I'm not sure if it is given your initial requirements. > > > > > > Do you need full host block devices to be provided to the instance? > > > > > > The LVM imagebackend will just provision LVs on top of the provided VG > > > so there's no direct mapping to a full host block device with this > > > approach. > > > > > > That said there's no real alternative available at the moment. > > > > well one alternitive to nova providing local lvm storage is to use > > the cinder lvm driver but install it on all compute nodes then > > use the cidner InstanceLocalityFilter to ensure the volume is alocated > form the host > > the vm is on. > > > https://docs.openstack.org/cinder/latest/configuration/block-storage/scheduler-filters.html#instancelocalityfilter > > on drawback to this is that if the if the vm is moved i think you would > need to also migrate the cinder volume > > seperatly afterwards. > by the way if you were to take this approch i think there is an nvmeof > driver so you can use nvme over rdma > instead of iscsi. > > > > > > > > > Also, I have read about the LVM device filters - which is important > to > > > > avoid the host's LVM from seeing the guest's volumes, in case anyone > > > > else finds this message. > > > > > > > > > Yeah that's a common pitfall when using LVM based ephemeral disks that > > > contain additional LVM PVs/VGs/LVs etc. You need to ensure that the > host > > > is configured to not scan these LVs in order for their PVs/VGs/LVs etc > > > to remain hidden from the host: > > > > > > > > > > > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_filters > > > > > > > > > > > > > > > -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Wed Aug 5 13:01:12 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 05 Aug 2020 14:01:12 +0100 Subject: [cinder][nova] Local storage in compute node In-Reply-To: References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> Message-ID: <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> On Wed, 2020-08-05 at 08:36 -0400, Donny Davis wrote: > I use local nvme to drive the CI workload for the openstack community for > the last year or so. It seems to work pretty well. I just created a > filesystem (xfs) and mounted it to /var/lib/nova/instances > I moved glance to using my swift backend and it really made the download of > the images much faster. > > It depends on if the workload is going to handle HA or you are expecting to > migrate machines. If the workload is ephemeral or HA can be handled in the > app I think local storage is still a very viable option. > > Simpler is better IMO yes that works well with the default flat/qcow file format i assume there was a reason this was not the starting point. the nova lvm backend i think does not supprot thin provisioning so fi you did the same thing creating the volume group on the nvme deivce you would technically get better write performance after the vm is booted but the vm spwan is slower since we cant take advantage of thin providioning and each root disk need to be copided form the cahced image. so just monting the nova data directory on an nvme driver or a raid of nvme drives works well and is simple to do. i take a slightly more complex approach from my home cluster wehre i put the nova data directory on a bcache block device which puts an nvme pci ssd as a cache infront of my raid 10 fo HDDs to acclerate it. from nova point of view there is nothing special about this setup it just works. the draw back to this is you cant change teh stroage avaiable to a vm without creating a new flaovr. exposing the nvme deivce or subsection of them via cinder has the advantage of allowing you to use teh vloume api to tailor the amount of storage per vm rather then creating a bunch of different flavors but with the over head fo needing to connect to the storage over a network protocol. so there are trade off with both appoches. generally i recommend using local sotrage e.g. the vm root disk or ephemeral disk for fast scratchpad space to work on data bug persitie all relevent data permently via cinder volumes. that requires you to understand which block devices a local and which are remote but it give you the best of both worlds. > > > > On Wed, Aug 5, 2020 at 7:48 AM Sean Mooney wrote: > > > On Wed, 2020-08-05 at 12:40 +0100, Sean Mooney wrote: > > > On Wed, 2020-08-05 at 12:19 +0100, Lee Yarwood wrote: > > > > On 05-08-20 05:03:29, Eric K. Miller wrote: > > > > > In case this is the answer, I found that in nova.conf, under the > > > > > [libvirt] stanza, images_type can be set to "lvm". This looks like > > > > it > > > > > may do the trick - using the compute node's LVM to provision and > > > > mount a > > > > > logical volume, for either persistent or ephemeral storage defined in > > > > > the flavor. > > > > > > > > > > Can anyone validate that this is the right approach according to our > > > > > needs? > > > > > > > > I'm not sure if it is given your initial requirements. > > > > > > > > Do you need full host block devices to be provided to the instance? > > > > > > > > The LVM imagebackend will just provision LVs on top of the provided VG > > > > so there's no direct mapping to a full host block device with this > > > > approach. > > > > > > > > That said there's no real alternative available at the moment. > > > > > > well one alternitive to nova providing local lvm storage is to use > > > the cinder lvm driver but install it on all compute nodes then > > > use the cidner InstanceLocalityFilter to ensure the volume is alocated > > > > form the host > > > the vm is on. > > > > > > > https://docs.openstack.org/cinder/latest/configuration/block-storage/scheduler-filters.html#instancelocalityfilter > > > on drawback to this is that if the if the vm is moved i think you would > > > > need to also migrate the cinder volume > > > seperatly afterwards. > > > > by the way if you were to take this approch i think there is an nvmeof > > driver so you can use nvme over rdma > > instead of iscsi. > > > > > > > > > > > > Also, I have read about the LVM device filters - which is important > > > > to > > > > > avoid the host's LVM from seeing the guest's volumes, in case anyone > > > > > else finds this message. > > > > > > > > > > > > Yeah that's a common pitfall when using LVM based ephemeral disks that > > > > contain additional LVM PVs/VGs/LVs etc. You need to ensure that the > > > > host > > > > is configured to not scan these LVs in order for their PVs/VGs/LVs etc > > > > to remain hidden from the host: > > > > > > > > > > > > > > > > > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_filters > > > > > > > > > > > > > > > > > > > > > > > > > From donny at fortnebula.com Wed Aug 5 13:22:58 2020 From: donny at fortnebula.com (Donny Davis) Date: Wed, 5 Aug 2020 09:22:58 -0400 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> Message-ID: On Wed, Aug 5, 2020 at 9:01 AM Sean Mooney wrote: > On Wed, 2020-08-05 at 08:36 -0400, Donny Davis wrote: > > I use local nvme to drive the CI workload for the openstack community for > > the last year or so. It seems to work pretty well. I just created a > > filesystem (xfs) and mounted it to /var/lib/nova/instances > > I moved glance to using my swift backend and it really made the download > of > > the images much faster. > > > > It depends on if the workload is going to handle HA or you are expecting > to > > migrate machines. If the workload is ephemeral or HA can be handled in > the > > app I think local storage is still a very viable option. > > > > Simpler is better IMO > yes that works well with the default flat/qcow file format > i assume there was a reason this was not the starting point. > the nova lvm backend i think does not supprot thin provisioning > so fi you did the same thing creating the volume group on the nvme deivce > you would technically get better write performance after the vm is booted > but > the vm spwan is slower since we cant take advantage of thin providioning > and > each root disk need to be copided form the cahced image. > > so just monting the nova data directory on an nvme driver or a raid of > nvme drives > works well and is simple to do. > > i take a slightly more complex approach from my home cluster wehre i put > the > nova data directory on a bcache block device which puts an nvme pci ssd as > a cache > infront of my raid 10 fo HDDs to acclerate it. from nova point of view > there is nothing special > about this setup it just works. > > the draw back to this is you cant change teh stroage avaiable to a vm > without creating a new flaovr. > exposing the nvme deivce or subsection of them via cinder has the > advantage of allowing you to use > teh vloume api to tailor the amount of storage per vm rather then creating > a bunch of different flavors > but with the over head fo needing to connect to the storage over a network > protocol. > > so there are trade off with both appoches. > generally i recommend using local sotrage e.g. the vm root disk or > ephemeral disk for fast scratchpad space > to work on data bug persitie all relevent data permently via cinder > volumes. that requires you to understand which block > devices a local and which are remote but it give you the best of both > worlds. > > > > > > > > > On Wed, Aug 5, 2020 at 7:48 AM Sean Mooney wrote: > > > > > On Wed, 2020-08-05 at 12:40 +0100, Sean Mooney wrote: > > > > On Wed, 2020-08-05 at 12:19 +0100, Lee Yarwood wrote: > > > > > On 05-08-20 05:03:29, Eric K. Miller wrote: > > > > > > In case this is the answer, I found that in nova.conf, under the > > > > > > [libvirt] stanza, images_type can be set to "lvm". This looks > like > > > > > > it > > > > > > may do the trick - using the compute node's LVM to provision and > > > > > > mount a > > > > > > logical volume, for either persistent or ephemeral storage > defined in > > > > > > the flavor. > > > > > > > > > > > > Can anyone validate that this is the right approach according to > our > > > > > > needs? > > > > > > > > > > I'm not sure if it is given your initial requirements. > > > > > > > > > > Do you need full host block devices to be provided to the instance? > > > > > > > > > > The LVM imagebackend will just provision LVs on top of the > provided VG > > > > > so there's no direct mapping to a full host block device with this > > > > > approach. > > > > > > > > > > That said there's no real alternative available at the moment. > > > > > > > > well one alternitive to nova providing local lvm storage is to use > > > > the cinder lvm driver but install it on all compute nodes then > > > > use the cidner InstanceLocalityFilter to ensure the volume is > alocated > > > > > > form the host > > > > the vm is on. > > > > > > > > > > > https://docs.openstack.org/cinder/latest/configuration/block-storage/scheduler-filters.html#instancelocalityfilter > > > > on drawback to this is that if the if the vm is moved i think you > would > > > > > > need to also migrate the cinder volume > > > > seperatly afterwards. > > > > > > by the way if you were to take this approch i think there is an nvmeof > > > driver so you can use nvme over rdma > > > instead of iscsi. > > > > > > > > > > > > > > > Also, I have read about the LVM device filters - which is > important > > > > > > to > > > > > > avoid the host's LVM from seeing the guest's volumes, in case > anyone > > > > > > else finds this message. > > > > > > > > > > > > > > > Yeah that's a common pitfall when using LVM based ephemeral disks > that > > > > > contain additional LVM PVs/VGs/LVs etc. You need to ensure that the > > > > > > host > > > > > is configured to not scan these LVs in order for their PVs/VGs/LVs > etc > > > > > to remain hidden from the host: > > > > > > > > > > > > > > > > > > > > > > > > > > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_filters > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have been through just about every possible nvme backend option for nova and the one that has turned up to be the most reliable and predictable has been simple defaults so far. Right now I am giving an nvme + nfs backend a spin. It doesn't perform badly, but it is not a local nvme. One of the things I have found with nvme is the mdadm raid driver is just not fast enough to keep up if you use anything other than raid0/1 (10) - I have a raid5 array I have got working pretty good - but its still limited. I don't have any vroc capable equipment, so maybe that will make a difference if implemented. I also have an all nvme ceph cluster I plan to test using cephfs (i know rbd is an option, but where is the fun in that). From my experience over the last two years in working with nvme only things, it seems that nothing comes close to matching the performance of what a couple local nvme drives in raid0 can do. NVME is so fast that the rest of my (old) equipment just can't keep up, it really does push things to the limits of what is possible. The all nvme ceph cluster does push my 40G network to its limits, but I had to create multiple OSD's per nvme to get there - for my gear (intel DC p3600's) I ended up at 3 OSD's per nvme. It seems to me to be limited by network performance. If you have any other questions I am happy to help where I can - I have been working with all nvme stuff for the last couple years and have gotten something into prod for about 1 year with it (maybe a little longer). >From what I can tell, getting max performance from nvme for an instance is a non-trivial task because it's just so much faster than the rest of the stack and careful considerations must be taken to get the most out of it. I am curious to see where you take this Eric -- ~/DonnyD C: 805 814 6800 "No mission too difficult. No sacrifice too great. Duty First" -------------- next part -------------- An HTML attachment was scrubbed... URL: From openstack at gmx.net Wed Aug 5 14:33:22 2020 From: openstack at gmx.net (Marc Vorwerk) Date: Wed, 05 Aug 2020 16:33:22 +0200 Subject: [nova] Change Volume Type Properties Message-ID: <24E5E9E3-6BF6-492C-BBBB-670DC070CF15@gmx.net> Hi, I'm looking for a way to add the property volume_backend_name to an existing Volume Type which is in use. If I try to change this, I got the following error: root at control01rt:~# openstack volume type show test-type +--------------------+--------------------------------------+ | Field              | Value                                | +--------------------+--------------------------------------+ | access_project_ids | None                                 | | description        | None                                 | | id                 | 68febdad-e7b1-4d41-ba11-72d0e1a1cce0 | | is_public          | True                                 | | name               | test-type                            | | properties         |                                      | | qos_specs_id       | None                                 | +--------------------+--------------------------------------+ root at control01rt:~# openstack volume type set --property volume_backend_name=ceph test-type Failed to set volume type property: Volume Type is currently in use. (HTTP 400) (Request-ID: req-2b8f3829-5c16-42c3-ac57-01199688bd58) Command Failed: One or more of the operations failed root at control01rt:~# Problem what I see is, that there are instances/volumes which use this volume type. Have anybody an idea, how I can add the volume_backend_name property to the existing Volume Type? thanks in advance! Regards Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Wed Aug 5 15:14:49 2020 From: mark at stackhpc.com (Mark Goddard) Date: Wed, 5 Aug 2020 16:14:49 +0100 Subject: =?UTF-8?Q?Re=3A_=5Bkolla=5D_Proposing_Micha=C5=82_Nasiadka_for_kayobe=2Dco?= =?UTF-8?Q?re?= In-Reply-To: References: Message-ID: On Tue, 28 Jul 2020 at 16:08, Doug Szumski wrote: > > > On 28/07/2020 15:50, Mark Goddard wrote: > > Hi, > > > > I'd like to propose adding Michał Nasiadka to the kayobe-core group. > > Michał is a valued member of the Kolla core team, and has been > > providing some good patches and reviews for Kayobe too. > > > > Kayobians, please respond with +1/-1. It's been a week, with only approvals - welcome to the core team Michał! > Sounds excellent, +1 for Michał! > > > > Cheers, > > Mark > > From johnsomor at gmail.com Wed Aug 5 15:16:23 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Wed, 5 Aug 2020 08:16:23 -0700 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard wrote: > > > On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: > >> Hello Guys, >> >> With Michaels help I was able to solve the problem but now there is >> another error I was able to create my network on vlan but still error >> persist. PFB the logs: >> >> http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ >> >> Kindly help >> >> regards, >> Monika >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Monday, August 3, 2020 9:10 PM >> *To:* Fabian Zimmermann >> *Cc:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Yeah, it looks like nova is failing to boot the instance. >> >> Check this setting in your octavia.conf files: >> https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id >> >> Also, if kolla-ansible didn't set both of these values correctly, please >> open bug reports for kolla-ansible. These all should have been configured >> by the deployment tool. >> >> > I wasn't following this thread due to no [kolla] tag, but here are the > recently added docs for Octavia in kolla [1]. Note > the octavia_service_auth_project variable which was added to migrate from > the admin project to the service project for octavia resources. We're > lacking proper automation for the flavor, image etc, but it is being worked > on in Victoria [2]. > > [1] > https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html > [2] https://review.opendev.org/740180 > > Michael >> >> On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann >> wrote: >> >> Seems like the flavor is missing or empty '' - check for typos and enable >> debug. >> >> Check if the nova req contains valid information/flavor. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 15:46: >> >> It's registered >> >> Get Outlook for Android >> ------------------------------ >> *From:* Fabian Zimmermann >> *Sent:* Monday, August 3, 2020 7:08:21 PM >> *To:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Did you check the (nova) flavor you use in octavia. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 10:53: >> >> After Michael suggestion I was able to create load balancer but there is >> error in status. >> >> >> >> PFB the error link: >> >> http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Monday, August 3, 2020 2:08 PM >> *To:* Michael Johnson >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Thanks a ton Michael for helping me out >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Friday, July 31, 2020 3:57 AM >> *To:* Monika Samal >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Just to close the loop on this, the octavia.conf file had >> "project_name = admin" instead of "project_name = service" in the >> [service_auth] section. This was causing the keystone errors when >> Octavia was communicating with neutron. >> >> I don't know if that is a bug in kolla-ansible or was just a local >> configuration issue. >> >> Michael >> >> On Thu, Jul 30, 2020 at 1:39 PM Monika Samal >> wrote: >> > >> > Hello Fabian,, >> > >> > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ >> > >> > Regards, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:57 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > Hi, >> > >> > just to debug, could you replace the auth_type password with v3password? >> > >> > And do a curl against your :5000 and :35357 urls and paste the output. >> > >> > Fabian >> > >> > Monika Samal schrieb am Do., 30. Juli 2020, >> 22:15: >> > >> > Hello Fabian, >> > >> > http://paste.openstack.org/show/796477/ >> > >> > Thanks, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:38 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > The sections should be >> > >> > service_auth >> > keystone_authtoken >> > >> > if i read the docs correctly. Maybe you can just paste your config >> (remove/change passwords) to paste.openstack.org and post the link? >> > >> > Fabian >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnasiadka at gmail.com Wed Aug 5 15:28:52 2020 From: mnasiadka at gmail.com (=?utf-8?Q?Micha=C5=82_Nasiadka?=) Date: Wed, 5 Aug 2020 17:28:52 +0200 Subject: =?utf-8?Q?Re=3A_=5Bkolla=5D_Proposing_Micha=C5=82_Nasiadka_for_ka?= =?utf-8?Q?yobe-core?= In-Reply-To: References: Message-ID: Hi, Thanks for being a part of such a great team! Best regards, Michal > On 5 Aug 2020, at 17:14, Mark Goddard wrote: > > On Tue, 28 Jul 2020 at 16:08, Doug Szumski wrote: >> >> >> On 28/07/2020 15:50, Mark Goddard wrote: >>> Hi, >>> >>> I'd like to propose adding Michał Nasiadka to the kayobe-core group. >>> Michał is a valued member of the Kolla core team, and has been >>> providing some good patches and reviews for Kayobe too. >>> >>> Kayobians, please respond with +1/-1. > It's been a week, with only approvals - welcome to the core team Michał! >> Sounds excellent, +1 for Michał! >>> >>> Cheers, >>> Mark >>> > From jasowang at redhat.com Wed Aug 5 02:22:15 2020 From: jasowang at redhat.com (Jason Wang) Date: Wed, 5 Aug 2020 10:22:15 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200804183503.39f56516.cohuck@redhat.com> References: <20200713232957.GD5955@joy-OptiPlex-7040> <9bfa8700-91f5-ebb4-3977-6321f0487a63@redhat.com> <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> Message-ID: On 2020/8/5 上午12:35, Cornelia Huck wrote: > [sorry about not chiming in earlier] > > On Wed, 29 Jul 2020 16:05:03 +0800 > Yan Zhao wrote: > >> On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: > (...) > >>> Based on the feedback we've received, the previously proposed interface >>> is not viable. I think there's agreement that the user needs to be >>> able to parse and interpret the version information. Using json seems >>> viable, but I don't know if it's the best option. Is there any >>> precedent of markup strings returned via sysfs we could follow? > I don't think encoding complex information in a sysfs file is a viable > approach. Quoting Documentation/filesystems/sysfs.rst: > > "Attributes should be ASCII text files, preferably with only one value > per file. It is noted that it may not be efficient to contain only one > value per file, so it is socially acceptable to express an array of > values of the same type. > > Mixing types, expressing multiple lines of data, and doing fancy > formatting of data is heavily frowned upon." > > Even though this is an older file, I think these restrictions still > apply. +1, that's another reason why devlink(netlink) is better. Thanks From yan.y.zhao at intel.com Wed Aug 5 02:16:54 2020 From: yan.y.zhao at intel.com (Yan Zhao) Date: Wed, 5 Aug 2020 10:16:54 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: References: <20200713232957.GD5955@joy-OptiPlex-7040> <9bfa8700-91f5-ebb4-3977-6321f0487a63@redhat.com> <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> Message-ID: <20200805021654.GB30485@joy-OptiPlex-7040> On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: > > On 2020/8/5 上午12:35, Cornelia Huck wrote: > > [sorry about not chiming in earlier] > > > > On Wed, 29 Jul 2020 16:05:03 +0800 > > Yan Zhao wrote: > > > > > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: > > (...) > > > > > > Based on the feedback we've received, the previously proposed interface > > > > is not viable. I think there's agreement that the user needs to be > > > > able to parse and interpret the version information. Using json seems > > > > viable, but I don't know if it's the best option. Is there any > > > > precedent of markup strings returned via sysfs we could follow? > > I don't think encoding complex information in a sysfs file is a viable > > approach. Quoting Documentation/filesystems/sysfs.rst: > > > > "Attributes should be ASCII text files, preferably with only one value > > per file. It is noted that it may not be efficient to contain only one > > value per file, so it is socially acceptable to express an array of > > values of the same type. > > Mixing types, expressing multiple lines of data, and doing fancy > > formatting of data is heavily frowned upon." > > > > Even though this is an older file, I think these restrictions still > > apply. > > > +1, that's another reason why devlink(netlink) is better. > hi Jason, do you have any materials or sample code about devlink, so we can have a good study of it? I found some kernel docs about it but my preliminary study didn't show me the advantage of devlink. Thanks Yan From jasowang at redhat.com Wed Aug 5 02:41:54 2020 From: jasowang at redhat.com (Jason Wang) Date: Wed, 5 Aug 2020 10:41:54 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200805021654.GB30485@joy-OptiPlex-7040> References: <20200713232957.GD5955@joy-OptiPlex-7040> <9bfa8700-91f5-ebb4-3977-6321f0487a63@redhat.com> <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> Message-ID: <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> On 2020/8/5 上午10:16, Yan Zhao wrote: > On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: >> On 2020/8/5 上午12:35, Cornelia Huck wrote: >>> [sorry about not chiming in earlier] >>> >>> On Wed, 29 Jul 2020 16:05:03 +0800 >>> Yan Zhao wrote: >>> >>>> On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: >>> (...) >>> >>>>> Based on the feedback we've received, the previously proposed interface >>>>> is not viable. I think there's agreement that the user needs to be >>>>> able to parse and interpret the version information. Using json seems >>>>> viable, but I don't know if it's the best option. Is there any >>>>> precedent of markup strings returned via sysfs we could follow? >>> I don't think encoding complex information in a sysfs file is a viable >>> approach. Quoting Documentation/filesystems/sysfs.rst: >>> >>> "Attributes should be ASCII text files, preferably with only one value >>> per file. It is noted that it may not be efficient to contain only one >>> value per file, so it is socially acceptable to express an array of >>> values of the same type. >>> Mixing types, expressing multiple lines of data, and doing fancy >>> formatting of data is heavily frowned upon." >>> >>> Even though this is an older file, I think these restrictions still >>> apply. >> >> +1, that's another reason why devlink(netlink) is better. >> > hi Jason, > do you have any materials or sample code about devlink, so we can have a good > study of it? > I found some kernel docs about it but my preliminary study didn't show me the > advantage of devlink. CC Jiri and Parav for a better answer for this. My understanding is that the following advantages are obvious (as I replied in another thread): - existing users (NIC, crypto, SCSI, ib), mature and stable - much better error reporting (ext_ack other than string or errno) - namespace aware - do not couple with kobject Thanks > > Thanks > Yan > From jiri at mellanox.com Wed Aug 5 07:56:47 2020 From: jiri at mellanox.com (Jiri Pirko) Date: Wed, 5 Aug 2020 09:56:47 +0200 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> References: <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> Message-ID: <20200805075647.GB2177@nanopsycho> Wed, Aug 05, 2020 at 04:41:54AM CEST, jasowang at redhat.com wrote: > >On 2020/8/5 上午10:16, Yan Zhao wrote: >> On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: >> > On 2020/8/5 上午12:35, Cornelia Huck wrote: >> > > [sorry about not chiming in earlier] >> > > >> > > On Wed, 29 Jul 2020 16:05:03 +0800 >> > > Yan Zhao wrote: >> > > >> > > > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: >> > > (...) >> > > >> > > > > Based on the feedback we've received, the previously proposed interface >> > > > > is not viable. I think there's agreement that the user needs to be >> > > > > able to parse and interpret the version information. Using json seems >> > > > > viable, but I don't know if it's the best option. Is there any >> > > > > precedent of markup strings returned via sysfs we could follow? >> > > I don't think encoding complex information in a sysfs file is a viable >> > > approach. Quoting Documentation/filesystems/sysfs.rst: >> > > >> > > "Attributes should be ASCII text files, preferably with only one value >> > > per file. It is noted that it may not be efficient to contain only one >> > > value per file, so it is socially acceptable to express an array of >> > > values of the same type. >> > > Mixing types, expressing multiple lines of data, and doing fancy >> > > formatting of data is heavily frowned upon." >> > > >> > > Even though this is an older file, I think these restrictions still >> > > apply. >> > >> > +1, that's another reason why devlink(netlink) is better. >> > >> hi Jason, >> do you have any materials or sample code about devlink, so we can have a good >> study of it? >> I found some kernel docs about it but my preliminary study didn't show me the >> advantage of devlink. > > >CC Jiri and Parav for a better answer for this. > >My understanding is that the following advantages are obvious (as I replied >in another thread): > >- existing users (NIC, crypto, SCSI, ib), mature and stable >- much better error reporting (ext_ack other than string or errno) >- namespace aware >- do not couple with kobject Jason, what is your use case? > >Thanks > > >> >> Thanks >> Yan >> > From jasowang at redhat.com Wed Aug 5 08:02:48 2020 From: jasowang at redhat.com (Jason Wang) Date: Wed, 5 Aug 2020 16:02:48 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200805075647.GB2177@nanopsycho> References: <20200716083230.GA25316@joy-OptiPlex-7040> <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> Message-ID: On 2020/8/5 下午3:56, Jiri Pirko wrote: > Wed, Aug 05, 2020 at 04:41:54AM CEST, jasowang at redhat.com wrote: >> On 2020/8/5 上午10:16, Yan Zhao wrote: >>> On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: >>>> On 2020/8/5 上午12:35, Cornelia Huck wrote: >>>>> [sorry about not chiming in earlier] >>>>> >>>>> On Wed, 29 Jul 2020 16:05:03 +0800 >>>>> Yan Zhao wrote: >>>>> >>>>>> On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: >>>>> (...) >>>>> >>>>>>> Based on the feedback we've received, the previously proposed interface >>>>>>> is not viable. I think there's agreement that the user needs to be >>>>>>> able to parse and interpret the version information. Using json seems >>>>>>> viable, but I don't know if it's the best option. Is there any >>>>>>> precedent of markup strings returned via sysfs we could follow? >>>>> I don't think encoding complex information in a sysfs file is a viable >>>>> approach. Quoting Documentation/filesystems/sysfs.rst: >>>>> >>>>> "Attributes should be ASCII text files, preferably with only one value >>>>> per file. It is noted that it may not be efficient to contain only one >>>>> value per file, so it is socially acceptable to express an array of >>>>> values of the same type. >>>>> Mixing types, expressing multiple lines of data, and doing fancy >>>>> formatting of data is heavily frowned upon." >>>>> >>>>> Even though this is an older file, I think these restrictions still >>>>> apply. >>>> +1, that's another reason why devlink(netlink) is better. >>>> >>> hi Jason, >>> do you have any materials or sample code about devlink, so we can have a good >>> study of it? >>> I found some kernel docs about it but my preliminary study didn't show me the >>> advantage of devlink. >> >> CC Jiri and Parav for a better answer for this. >> >> My understanding is that the following advantages are obvious (as I replied >> in another thread): >> >> - existing users (NIC, crypto, SCSI, ib), mature and stable >> - much better error reporting (ext_ack other than string or errno) >> - namespace aware >> - do not couple with kobject > Jason, what is your use case? I think the use case is to report device compatibility for live migration. Yan proposed a simple sysfs based migration version first, but it looks not sufficient and something based on JSON is discussed. Yan, can you help to summarize the discussion so far for Jiri as a reference? Thanks > > > >> Thanks >> >> >>> Thanks >>> Yan >>> From dgilbert at redhat.com Wed Aug 5 09:44:23 2020 From: dgilbert at redhat.com (Dr. David Alan Gilbert) Date: Wed, 5 Aug 2020 10:44:23 +0100 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200804083708.GA30485@joy-OptiPlex-7040> References: <20200717101258.65555978@x1.home> <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200729131255.68730f68@x1.home> <20200730034104.GB32327@joy-OptiPlex-7040> <20200730112930.6f4c5762@x1.home> <20200804083708.GA30485@joy-OptiPlex-7040> Message-ID: <20200805094423.GB3004@work-vm> * Yan Zhao (yan.y.zhao at intel.com) wrote: > > > yes, include a device_api field is better. > > > for mdev, "device_type=vfio-mdev", is it right? > > > > No, vfio-mdev is not a device API, it's the driver that attaches to the > > mdev bus device to expose it through vfio. The device_api exposes the > > actual interface of the vfio device, it's also vfio-pci for typical > > mdev devices found on x86, but may be vfio-ccw, vfio-ap, etc... See > > VFIO_DEVICE_API_PCI_STRING and friends. > > > ok. got it. > > > > > > > device_id=8086591d > > > > > > > > Is device_id interpreted relative to device_type? How does this > > > > relate to mdev_type? If we have an mdev_type, doesn't that fully > > > > defined the software API? > > > > > > > it's parent pci id for mdev actually. > > > > If we need to specify the parent PCI ID then something is fundamentally > > wrong with the mdev_type. The mdev_type should define a unique, > > software compatible interface, regardless of the parent device IDs. If > > a i915-GVTg_V5_2 means different things based on the parent device IDs, > > then then different mdev_types should be reported for those parent > > devices. > > > hmm, then do we allow vendor specific fields? > or is it a must that a vendor specific field should have corresponding > vendor attribute? > > another thing is that the definition of mdev_type in GVT only corresponds > to vGPU computing ability currently, > e.g. i915-GVTg_V5_2, is 1/2 of a gen9 IGD, i915-GVTg_V4_2 is 1/2 of a > gen8 IGD. > It is too coarse-grained to live migration compatibility. Can you explain why that's too coarse? Is this because it's too specific (i.e. that a i915-GVTg_V4_2 could be migrated to a newer device?), or that it's too specific on the exact sizings (i.e. that there may be multiple different sizes of a gen9)? Dave > Do you think we need to update GVT's definition of mdev_type? > And is there any guide in mdev_type definition? > > > > > > > mdev_type=i915-GVTg_V5_2 > > > > > > > > And how are non-mdev devices represented? > > > > > > > non-mdev can opt to not include this field, or as you said below, a > > > vendor signature. > > > > > > > > > aggregator=1 > > > > > > pv_mode="none+ppgtt+context" > > > > > > > > These are meaningless vendor specific matches afaict. > > > > > > > yes, pv_mode and aggregator are vendor specific fields. > > > but they are important to decide whether two devices are compatible. > > > pv_mode means whether a vGPU supports guest paravirtualized api. > > > "none+ppgtt+context" means guest can not use pv, or use ppgtt mode pv or > > > use context mode pv. > > > > > > > > > interface_version=3 > > > > > > > > Not much granularity here, I prefer Sean's previous > > > > .[.bugfix] scheme. > > > > > > > yes, .[.bugfix] scheme may be better, but I'm not sure if > > > it works for a complicated scenario. > > > e.g for pv_mode, > > > (1) initially, pv_mode is not supported, so it's pv_mode=none, it's 0.0.0, > > > (2) then, pv_mode=ppgtt is supported, pv_mode="none+ppgtt", it's 0.1.0, > > > indicating pv_mode=none can migrate to pv_mode="none+ppgtt", but not vice versa. > > > (3) later, pv_mode=context is also supported, > > > pv_mode="none+ppgtt+context", so it's 0.2.0. > > > > > > But if later, pv_mode=ppgtt is removed. pv_mode="none+context", how to > > > name its version? "none+ppgtt" (0.1.0) is not compatible to > > > "none+context", but "none+ppgtt+context" (0.2.0) is compatible to > > > "none+context". > > > > If pv_mode=ppgtt is removed, then the compatible versions would be > > 0.0.0 or 1.0.0, ie. the major version would be incremented due to > > feature removal. > > > > > Maintain such scheme is painful to vendor driver. > > > > Migration compatibility is painful, there's no way around that. I > > think the version scheme is an attempt to push some of that low level > > burden on the vendor driver, otherwise the management tools need to > > work on an ever growing matrix of vendor specific features which is > > going to become unwieldy and is largely meaningless outside of the > > vendor driver. Instead, the vendor driver can make strategic decisions > > about where to continue to maintain a support burden and make explicit > > decisions to maintain or break compatibility. The version scheme is a > > simplification and abstraction of vendor driver features in order to > > create a small, logical compatibility matrix. Compromises necessarily > > need to be made for that to occur. > > > ok. got it. > > > > > > > COMPATIBLE: > > > > > > device_type=pci > > > > > > device_id=8086591d > > > > > > mdev_type=i915-GVTg_V5_{val1:int:1,2,4,8} > > > > > this mixed notation will be hard to parse so i would avoid that. > > > > > > > > Some background, Intel has been proposing aggregation as a solution to > > > > how we scale mdev devices when hardware exposes large numbers of > > > > assignable objects that can be composed in essentially arbitrary ways. > > > > So for instance, if we have a workqueue (wq), we might have an mdev > > > > type for 1wq, 2wq, 3wq,... Nwq. It's not really practical to expose a > > > > discrete mdev type for each of those, so they want to define a base > > > > type which is composable to other types via this aggregation. This is > > > > what this substitution and tagging is attempting to accomplish. So > > > > imagine this set of values for cases where it's not practical to unroll > > > > the values for N discrete types. > > > > > > > > > > aggregator={val1}/2 > > > > > > > > So the {val1} above would be substituted here, though an aggregation > > > > factor of 1/2 is a head scratcher... > > > > > > > > > > pv_mode={val2:string:"none+ppgtt","none+context","none+ppgtt+context"} > > > > > > > > I'm lost on this one though. I think maybe it's indicating that it's > > > > compatible with any of these, so do we need to list it? Couldn't this > > > > be handled by Sean's version proposal where the minor version > > > > represents feature compatibility? > > > yes, it's indicating that it's compatible with any of these. > > > Sean's version proposal may also work, but it would be painful for > > > vendor driver to maintain the versions when multiple similar features > > > are involved. > > > > This is something vendor drivers need to consider when adding and > > removing features. > > > > > > > > interface_version={val3:int:2,3} > > > > > > > > What does this turn into in a few years, 2,7,12,23,75,96,... > > > > > > > is a range better? > > > > I was really trying to point out that sparseness becomes an issue if > > the vendor driver is largely disconnected from how their feature > > addition and deprecation affects migration support. Thanks, > > > ok. we'll use the x.y.z scheme then. > > Thanks > Yan > -- Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK From yan.y.zhao at intel.com Wed Aug 5 09:33:38 2020 From: yan.y.zhao at intel.com (Yan Zhao) Date: Wed, 5 Aug 2020 17:33:38 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: References: <20200721005113.GA10502@joy-OptiPlex-7040> <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> Message-ID: <20200805093338.GC30485@joy-OptiPlex-7040> On Wed, Aug 05, 2020 at 04:02:48PM +0800, Jason Wang wrote: > > On 2020/8/5 下午3:56, Jiri Pirko wrote: > > Wed, Aug 05, 2020 at 04:41:54AM CEST, jasowang at redhat.com wrote: > > > On 2020/8/5 上午10:16, Yan Zhao wrote: > > > > On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: > > > > > On 2020/8/5 上午12:35, Cornelia Huck wrote: > > > > > > [sorry about not chiming in earlier] > > > > > > > > > > > > On Wed, 29 Jul 2020 16:05:03 +0800 > > > > > > Yan Zhao wrote: > > > > > > > > > > > > > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: > > > > > > (...) > > > > > > > > > > > > > > Based on the feedback we've received, the previously proposed interface > > > > > > > > is not viable. I think there's agreement that the user needs to be > > > > > > > > able to parse and interpret the version information. Using json seems > > > > > > > > viable, but I don't know if it's the best option. Is there any > > > > > > > > precedent of markup strings returned via sysfs we could follow? > > > > > > I don't think encoding complex information in a sysfs file is a viable > > > > > > approach. Quoting Documentation/filesystems/sysfs.rst: > > > > > > > > > > > > "Attributes should be ASCII text files, preferably with only one value > > > > > > per file. It is noted that it may not be efficient to contain only one > > > > > > value per file, so it is socially acceptable to express an array of > > > > > > values of the same type. > > > > > > Mixing types, expressing multiple lines of data, and doing fancy > > > > > > formatting of data is heavily frowned upon." > > > > > > > > > > > > Even though this is an older file, I think these restrictions still > > > > > > apply. > > > > > +1, that's another reason why devlink(netlink) is better. > > > > > > > > > hi Jason, > > > > do you have any materials or sample code about devlink, so we can have a good > > > > study of it? > > > > I found some kernel docs about it but my preliminary study didn't show me the > > > > advantage of devlink. > > > > > > CC Jiri and Parav for a better answer for this. > > > > > > My understanding is that the following advantages are obvious (as I replied > > > in another thread): > > > > > > - existing users (NIC, crypto, SCSI, ib), mature and stable > > > - much better error reporting (ext_ack other than string or errno) > > > - namespace aware > > > - do not couple with kobject > > Jason, what is your use case? > > > I think the use case is to report device compatibility for live migration. > Yan proposed a simple sysfs based migration version first, but it looks not > sufficient and something based on JSON is discussed. > > Yan, can you help to summarize the discussion so far for Jiri as a > reference? > yes. we are currently defining an device live migration compatibility interface in order to let user space like openstack and libvirt knows which two devices are live migration compatible. currently the devices include mdev (a kernel emulated virtual device) and physical devices (e.g. a VF of a PCI SRIOV device). the attributes we want user space to compare including common attribues: device_api: vfio-pci, vfio-ccw... mdev_type: mdev type of mdev or similar signature for physical device It specifies a device's hardware capability. e.g. i915-GVTg_V5_4 means it's of 1/4 of a gen9 Intel graphics device. software_version: device driver's version. in .[.bugfix] scheme, where there is no compatibility across major versions, minor versions have forward compatibility (ex. 1-> 2 is ok, 2 -> 1 is not) and bugfix version number indicates some degree of internal improvement that is not visible to the user in terms of features or compatibility, vendor specific attributes: each vendor may define different attributes device id : device id of a physical devices or mdev's parent pci device. it could be equal to pci id for pci devices aggregator: used together with mdev_type. e.g. aggregator=2 together with i915-GVTg_V5_4 means 2*1/4=1/2 of a gen9 Intel graphics device. remote_url: for a local NVMe VF, it may be configured with a remote url of a remote storage and all data is stored in the remote side specified by the remote url. ... Comparing those attributes by user space alone is not an easy job, as it can't simply assume an equal relationship between source attributes and target attributes. e.g. for a source device of mdev_type=i915-GVTg_V5_4,aggregator=2, (1/2 of gen9), it actually could find a compatible device of mdev_type=i915-GVTg_V5_8,aggregator=4 (also 1/2 of gen9), if mdev_type of i915-GVTg_V5_4 is not available in the target machine. So, in our current proposal, we want to create two sysfs attributes under a device sysfs node. /sys//migration/self /sys//migration/compatible #cat /sys//migration/self device_type=vfio_pci mdev_type=i915-GVTg_V5_4 device_id=8086591d aggregator=2 software_version=1.0.0 #cat /sys//migration/compatible device_type=vfio_pci mdev_type=i915-GVTg_V5_{val1:int:2,4,8} device_id=8086591d aggregator={val1}/2 software_version=1.0.0 The /sys//migration/self specifies self attributes of a device. The /sys//migration/compatible specifies the list of compatible devices of a device. as in the example, compatible devices could have device_type == vfio_pci && device_id == 8086591d && software_version == 1.0.0 && ( (mdev_type of i915-GVTg_V5_2 && aggregator==1) || (mdev_type of i915-GVTg_V5_4 && aggregator==2) || (mdev_type of i915-GVTg_V5_8 && aggregator=4) ) by comparing whether a target device is in compatible list of source device, the user space can know whether a two devices are live migration compatible. Additional notes: 1)software_version in the compatible list may not be necessary as it already has a major.minor.bugfix scheme. 2)for vendor attribute like remote_url, it may not be statically assigned and could be changed with a device interface. So, as Cornelia pointed that it's not good to use complex format in a sysfs attribute, we'd like to know whether there're other good ways to our use case, e.g. splitting a single attribute to multiple simple sysfs attributes as what Cornelia suggested or devlink that Jason has strongly recommended. Thanks Yan From jiri at mellanox.com Wed Aug 5 10:53:19 2020 From: jiri at mellanox.com (Jiri Pirko) Date: Wed, 5 Aug 2020 12:53:19 +0200 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200805093338.GC30485@joy-OptiPlex-7040> References: <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> <20200805093338.GC30485@joy-OptiPlex-7040> Message-ID: <20200805105319.GF2177@nanopsycho> Wed, Aug 05, 2020 at 11:33:38AM CEST, yan.y.zhao at intel.com wrote: >On Wed, Aug 05, 2020 at 04:02:48PM +0800, Jason Wang wrote: >> >> On 2020/8/5 下午3:56, Jiri Pirko wrote: >> > Wed, Aug 05, 2020 at 04:41:54AM CEST, jasowang at redhat.com wrote: >> > > On 2020/8/5 上午10:16, Yan Zhao wrote: >> > > > On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: >> > > > > On 2020/8/5 上午12:35, Cornelia Huck wrote: >> > > > > > [sorry about not chiming in earlier] >> > > > > > >> > > > > > On Wed, 29 Jul 2020 16:05:03 +0800 >> > > > > > Yan Zhao wrote: >> > > > > > >> > > > > > > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: >> > > > > > (...) >> > > > > > >> > > > > > > > Based on the feedback we've received, the previously proposed interface >> > > > > > > > is not viable. I think there's agreement that the user needs to be >> > > > > > > > able to parse and interpret the version information. Using json seems >> > > > > > > > viable, but I don't know if it's the best option. Is there any >> > > > > > > > precedent of markup strings returned via sysfs we could follow? >> > > > > > I don't think encoding complex information in a sysfs file is a viable >> > > > > > approach. Quoting Documentation/filesystems/sysfs.rst: >> > > > > > >> > > > > > "Attributes should be ASCII text files, preferably with only one value >> > > > > > per file. It is noted that it may not be efficient to contain only one >> > > > > > value per file, so it is socially acceptable to express an array of >> > > > > > values of the same type. >> > > > > > Mixing types, expressing multiple lines of data, and doing fancy >> > > > > > formatting of data is heavily frowned upon." >> > > > > > >> > > > > > Even though this is an older file, I think these restrictions still >> > > > > > apply. >> > > > > +1, that's another reason why devlink(netlink) is better. >> > > > > >> > > > hi Jason, >> > > > do you have any materials or sample code about devlink, so we can have a good >> > > > study of it? >> > > > I found some kernel docs about it but my preliminary study didn't show me the >> > > > advantage of devlink. >> > > >> > > CC Jiri and Parav for a better answer for this. >> > > >> > > My understanding is that the following advantages are obvious (as I replied >> > > in another thread): >> > > >> > > - existing users (NIC, crypto, SCSI, ib), mature and stable >> > > - much better error reporting (ext_ack other than string or errno) >> > > - namespace aware >> > > - do not couple with kobject >> > Jason, what is your use case? >> >> >> I think the use case is to report device compatibility for live migration. >> Yan proposed a simple sysfs based migration version first, but it looks not >> sufficient and something based on JSON is discussed. >> >> Yan, can you help to summarize the discussion so far for Jiri as a >> reference? >> >yes. >we are currently defining an device live migration compatibility >interface in order to let user space like openstack and libvirt knows >which two devices are live migration compatible. >currently the devices include mdev (a kernel emulated virtual device) >and physical devices (e.g. a VF of a PCI SRIOV device). > >the attributes we want user space to compare including >common attribues: > device_api: vfio-pci, vfio-ccw... > mdev_type: mdev type of mdev or similar signature for physical device > It specifies a device's hardware capability. e.g. > i915-GVTg_V5_4 means it's of 1/4 of a gen9 Intel graphics > device. > software_version: device driver's version. > in .[.bugfix] scheme, where there is no > compatibility across major versions, minor versions have > forward compatibility (ex. 1-> 2 is ok, 2 -> 1 is not) and > bugfix version number indicates some degree of internal > improvement that is not visible to the user in terms of > features or compatibility, > >vendor specific attributes: each vendor may define different attributes > device id : device id of a physical devices or mdev's parent pci device. > it could be equal to pci id for pci devices > aggregator: used together with mdev_type. e.g. aggregator=2 together > with i915-GVTg_V5_4 means 2*1/4=1/2 of a gen9 Intel > graphics device. > remote_url: for a local NVMe VF, it may be configured with a remote > url of a remote storage and all data is stored in the > remote side specified by the remote url. > ... > >Comparing those attributes by user space alone is not an easy job, as it >can't simply assume an equal relationship between source attributes and >target attributes. e.g. >for a source device of mdev_type=i915-GVTg_V5_4,aggregator=2, (1/2 of >gen9), it actually could find a compatible device of >mdev_type=i915-GVTg_V5_8,aggregator=4 (also 1/2 of gen9), >if mdev_type of i915-GVTg_V5_4 is not available in the target machine. > >So, in our current proposal, we want to create two sysfs attributes >under a device sysfs node. >/sys//migration/self >/sys//migration/compatible > >#cat /sys//migration/self >device_type=vfio_pci >mdev_type=i915-GVTg_V5_4 >device_id=8086591d >aggregator=2 >software_version=1.0.0 > >#cat /sys//migration/compatible >device_type=vfio_pci >mdev_type=i915-GVTg_V5_{val1:int:2,4,8} >device_id=8086591d >aggregator={val1}/2 >software_version=1.0.0 > >The /sys//migration/self specifies self attributes of >a device. >The /sys//migration/compatible specifies the list of >compatible devices of a device. as in the example, compatible devices >could have > device_type == vfio_pci && > device_id == 8086591d && > software_version == 1.0.0 && > ( > (mdev_type of i915-GVTg_V5_2 && aggregator==1) || > (mdev_type of i915-GVTg_V5_4 && aggregator==2) || > (mdev_type of i915-GVTg_V5_8 && aggregator=4) > ) > >by comparing whether a target device is in compatible list of source >device, the user space can know whether a two devices are live migration >compatible. > >Additional notes: >1)software_version in the compatible list may not be necessary as it >already has a major.minor.bugfix scheme. >2)for vendor attribute like remote_url, it may not be statically >assigned and could be changed with a device interface. > >So, as Cornelia pointed that it's not good to use complex format in >a sysfs attribute, we'd like to know whether there're other good ways to >our use case, e.g. splitting a single attribute to multiple simple sysfs >attributes as what Cornelia suggested or devlink that Jason has strongly >recommended. Hi Yan. Thanks for the explanation, I'm still fuzzy about the details. Anyway, I suggest you to check "devlink dev info" command we have implemented for multiple drivers. You can try netdevsim to test this. I think that the info you need to expose might be put there. Devlink creates instance per-device. Specific device driver calls into devlink core to create the instance. What device do you have? What driver is it handled by? > >Thanks >Yan > > > From smooney at redhat.com Wed Aug 5 11:35:01 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 05 Aug 2020 12:35:01 +0100 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200805105319.GF2177@nanopsycho> References: <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> <20200805093338.GC30485@joy-OptiPlex-7040> <20200805105319.GF2177@nanopsycho> Message-ID: <4cf2824c803c96496e846c5b06767db305e9fb5a.camel@redhat.com> On Wed, 2020-08-05 at 12:53 +0200, Jiri Pirko wrote: > Wed, Aug 05, 2020 at 11:33:38AM CEST, yan.y.zhao at intel.com wrote: > > On Wed, Aug 05, 2020 at 04:02:48PM +0800, Jason Wang wrote: > > > > > > On 2020/8/5 下午3:56, Jiri Pirko wrote: > > > > Wed, Aug 05, 2020 at 04:41:54AM CEST, jasowang at redhat.com wrote: > > > > > On 2020/8/5 上午10:16, Yan Zhao wrote: > > > > > > On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: > > > > > > > On 2020/8/5 上午12:35, Cornelia Huck wrote: > > > > > > > > [sorry about not chiming in earlier] > > > > > > > > > > > > > > > > On Wed, 29 Jul 2020 16:05:03 +0800 > > > > > > > > Yan Zhao wrote: > > > > > > > > > > > > > > > > > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: > > > > > > > > > > > > > > > > (...) > > > > > > > > > > > > > > > > > > Based on the feedback we've received, the previously proposed interface > > > > > > > > > > is not viable. I think there's agreement that the user needs to be > > > > > > > > > > able to parse and interpret the version information. Using json seems > > > > > > > > > > viable, but I don't know if it's the best option. Is there any > > > > > > > > > > precedent of markup strings returned via sysfs we could follow? > > > > > > > > > > > > > > > > I don't think encoding complex information in a sysfs file is a viable > > > > > > > > approach. Quoting Documentation/filesystems/sysfs.rst: > > > > > > > > > > > > > > > > "Attributes should be ASCII text files, preferably with only one value > > > > > > > > per file. It is noted that it may not be efficient to contain only one > > > > > > > > value per file, so it is socially acceptable to express an array of > > > > > > > > values of the same type. > > > > > > > > Mixing types, expressing multiple lines of data, and doing fancy > > > > > > > > formatting of data is heavily frowned upon." > > > > > > > > > > > > > > > > Even though this is an older file, I think these restrictions still > > > > > > > > apply. > > > > > > > > > > > > > > +1, that's another reason why devlink(netlink) is better. > > > > > > > > > > > > > > > > > > > hi Jason, > > > > > > do you have any materials or sample code about devlink, so we can have a good > > > > > > study of it? > > > > > > I found some kernel docs about it but my preliminary study didn't show me the > > > > > > advantage of devlink. > > > > > > > > > > CC Jiri and Parav for a better answer for this. > > > > > > > > > > My understanding is that the following advantages are obvious (as I replied > > > > > in another thread): > > > > > > > > > > - existing users (NIC, crypto, SCSI, ib), mature and stable > > > > > - much better error reporting (ext_ack other than string or errno) > > > > > - namespace aware > > > > > - do not couple with kobject > > > > > > > > Jason, what is your use case? > > > > > > > > > I think the use case is to report device compatibility for live migration. > > > Yan proposed a simple sysfs based migration version first, but it looks not > > > sufficient and something based on JSON is discussed. > > > > > > Yan, can you help to summarize the discussion so far for Jiri as a > > > reference? > > > > > > > yes. > > we are currently defining an device live migration compatibility > > interface in order to let user space like openstack and libvirt knows > > which two devices are live migration compatible. > > currently the devices include mdev (a kernel emulated virtual device) > > and physical devices (e.g. a VF of a PCI SRIOV device). > > > > the attributes we want user space to compare including > > common attribues: > > device_api: vfio-pci, vfio-ccw... > > mdev_type: mdev type of mdev or similar signature for physical device > > It specifies a device's hardware capability. e.g. > > i915-GVTg_V5_4 means it's of 1/4 of a gen9 Intel graphics > > device. by the way this nameing sceam works the opisite of how it would have expected i woudl have expected to i915-GVTg_V5 to be the same as i915-GVTg_V5_1 and i915-GVTg_V5_4 to use 4 times the amount of resouce as i915-GVTg_V5_1 not 1 quarter. i would much rather see i915-GVTg_V5_4 express as aggreataor:i915-GVTg_V5=4 e.g. that it is 4 of the basic i915-GVTg_V5 type the invertion of the relationship makes this much harder to resonabout IMO. if i915-GVTg_V5_8 and i915-GVTg_V5_4 are both actully claiming the same resouce and both can be used at the same time with your suggested nameing scemem i have have to fine the mdevtype with the largest value and store that then do math by devidign it by the suffix of the requested type every time i want to claim the resouce in our placement inventoies. if we represent it the way i suggest we dont if it i915-GVTg_V5_8 i know its using 8 of i915-GVTg_V5 it makes it significantly simpler. > > software_version: device driver's version. > > in .[.bugfix] scheme, where there is no > > compatibility across major versions, minor versions have > > forward compatibility (ex. 1-> 2 is ok, 2 -> 1 is not) and > > bugfix version number indicates some degree of internal > > improvement that is not visible to the user in terms of > > features or compatibility, > > > > vendor specific attributes: each vendor may define different attributes > > device id : device id of a physical devices or mdev's parent pci device. > > it could be equal to pci id for pci devices > > aggregator: used together with mdev_type. e.g. aggregator=2 together > > with i915-GVTg_V5_4 means 2*1/4=1/2 of a gen9 Intel > > graphics device. > > remote_url: for a local NVMe VF, it may be configured with a remote > > url of a remote storage and all data is stored in the > > remote side specified by the remote url. > > ... just a minor not that i find ^ much more simmple to understand then the current proposal with self and compatiable. if i have well defiend attibute that i can parse and understand that allow me to calulate the what is and is not compatible that is likely going to more useful as you wont have to keep maintianing a list of other compatible devices every time a new sku is released. in anycase thank for actully shareing ^ as it make it simpler to reson about what you have previously proposed. > > > > Comparing those attributes by user space alone is not an easy job, as it > > can't simply assume an equal relationship between source attributes and > > target attributes. e.g. > > for a source device of mdev_type=i915-GVTg_V5_4,aggregator=2, (1/2 of > > gen9), it actually could find a compatible device of > > mdev_type=i915-GVTg_V5_8,aggregator=4 (also 1/2 of gen9), > > if mdev_type of i915-GVTg_V5_4 is not available in the target machine. > > > > So, in our current proposal, we want to create two sysfs attributes > > under a device sysfs node. > > /sys//migration/self > > /sys//migration/compatible > > > > #cat /sys//migration/self > > device_type=vfio_pci > > mdev_type=i915-GVTg_V5_4 > > device_id=8086591d > > aggregator=2 > > software_version=1.0.0 > > > > #cat /sys//migration/compatible > > device_type=vfio_pci > > mdev_type=i915-GVTg_V5_{val1:int:2,4,8} > > device_id=8086591d > > aggregator={val1}/2 > > software_version=1.0.0 > > > > The /sys//migration/self specifies self attributes of > > a device. > > The /sys//migration/compatible specifies the list of > > compatible devices of a device. as in the example, compatible devices > > could have > > device_type == vfio_pci && > > device_id == 8086591d && > > software_version == 1.0.0 && > > ( > > (mdev_type of i915-GVTg_V5_2 && aggregator==1) || > > (mdev_type of i915-GVTg_V5_4 && aggregator==2) || > > (mdev_type of i915-GVTg_V5_8 && aggregator=4) > > ) > > > > by comparing whether a target device is in compatible list of source > > device, the user space can know whether a two devices are live migration > > compatible. > > > > Additional notes: > > 1)software_version in the compatible list may not be necessary as it > > already has a major.minor.bugfix scheme. > > 2)for vendor attribute like remote_url, it may not be statically > > assigned and could be changed with a device interface. > > > > So, as Cornelia pointed that it's not good to use complex format in > > a sysfs attribute, we'd like to know whether there're other good ways to > > our use case, e.g. splitting a single attribute to multiple simple sysfs > > attributes as what Cornelia suggested or devlink that Jason has strongly > > recommended. > > Hi Yan. > > Thanks for the explanation, I'm still fuzzy about the details. > Anyway, I suggest you to check "devlink dev info" command we have > implemented for multiple drivers. is devlink exposed as a filesytem we can read with just open? openstack will likely try to leverage libvirt to get this info but when we cant its much simpler to read sysfs then it is to take a a depenency on a commandline too and have to fork shell to execute it and parse the cli output. pyroute2 which we use in some openstack poject has basic python binding for devlink but im not sure how complete it is as i think its relitivly new addtion. if we need to take a dependcy we will but that would be a drawback fo devlink not that that is a large one just something to keep in mind. > You can try netdevsim to test this. > I think that the info you need to expose might be put there. > > Devlink creates instance per-device. Specific device driver calls into > devlink core to create the instance. What device do you have? What > driver is it handled by? > > > > > > Thanks > > Yan > > > > > > > > From whayutin at redhat.com Wed Aug 5 16:23:46 2020 From: whayutin at redhat.com (Wesley Hayutin) Date: Wed, 5 Aug 2020 10:23:46 -0600 Subject: [tripleo][ci] container pulls failing In-Reply-To: References: Message-ID: On Wed, Jul 29, 2020 at 4:48 PM Wesley Hayutin wrote: > > > On Wed, Jul 29, 2020 at 4:33 PM Alex Schultz wrote: > >> On Wed, Jul 29, 2020 at 7:13 AM Wesley Hayutin >> wrote: >> > >> > >> > >> > On Wed, Jul 29, 2020 at 2:25 AM Bogdan Dobrelya >> wrote: >> >> >> >> On 7/28/20 6:09 PM, Wesley Hayutin wrote: >> >> > >> >> > >> >> > On Tue, Jul 28, 2020 at 7:24 AM Emilien Macchi > >> > > wrote: >> >> > >> >> > >> >> > >> >> > On Tue, Jul 28, 2020 at 9:20 AM Alex Schultz < >> aschultz at redhat.com >> >> > > wrote: >> >> > >> >> > On Tue, Jul 28, 2020 at 7:13 AM Emilien Macchi >> >> > > wrote: >> >> > > >> >> > > >> >> > > >> >> > > On Mon, Jul 27, 2020 at 5:27 PM Wesley Hayutin >> >> > > wrote: >> >> > >> >> >> > >> FYI... >> >> > >> >> >> > >> If you find your jobs are failing with an error similar >> to >> >> > [1], you have been rate limited by docker.io < >> http://docker.io> >> >> > via the upstream mirror system and have hit [2]. I've been >> >> > discussing the issue w/ upstream infra, rdo-infra and a few >> CI >> >> > engineers. >> >> > >> >> >> > >> There are a few ways to mitigate the issue however I >> don't >> >> > see any of the options being completed very quickly so I'm >> >> > asking for your patience while this issue is socialized and >> >> > resolved. >> >> > >> >> >> > >> For full transparency we're considering the following >> options. >> >> > >> >> >> > >> 1. move off of docker.io to quay.io >> >> > >> >> > > >> >> > > >> >> > > quay.io also has API rate limit: >> >> > > https://docs.quay.io/issues/429.html >> >> > > >> >> > > Now I'm not sure about how many requests per seconds one >> can >> >> > do vs the other but this would need to be checked with the >> quay >> >> > team before changing anything. >> >> > > Also quay.io had its big downtimes as >> well, >> >> > SLA needs to be considered. >> >> > > >> >> > >> 2. local container builds for each job in master, >> possibly >> >> > ussuri >> >> > > >> >> > > >> >> > > Not convinced. >> >> > > You can look at CI logs: >> >> > > - pulling / updating / pushing container images from >> >> > docker.io to local registry takes ~10 >> min on >> >> > standalone (OVH) >> >> > > - building containers from scratch with updated repos and >> >> > pushing them to local registry takes ~29 min on standalone >> (OVH). >> >> > > >> >> > >> >> >> > >> 3. parent child jobs upstream where rpms and containers >> will >> >> > be build and host artifacts for the child jobs >> >> > > >> >> > > >> >> > > Yes, we need to investigate that. >> >> > > >> >> > >> >> >> > >> 4. remove some portion of the upstream jobs to lower the >> >> > impact we have on 3rd party infrastructure. >> >> > > >> >> > > >> >> > > I'm not sure I understand this one, maybe you can give an >> >> > example of what could be removed? >> >> > >> >> > We need to re-evaulate our use of scenarios (e.g. we have two >> >> > scenario010's both are non-voting). There's a reason we >> >> > historically >> >> > didn't want to add more jobs because of these types of >> resource >> >> > constraints. I think we've added new jobs recently and >> likely >> >> > need to >> >> > reduce what we run. Additionally we might want to look into >> reducing >> >> > what we run on stable branches as well. >> >> > >> >> > >> >> > Oh... removing jobs (I thought we would remove some steps of the >> jobs). >> >> > Yes big +1, this should be a continuous goal when working on CI, >> and >> >> > always evaluating what we need vs what we run now. >> >> > >> >> > We should look at: >> >> > 1) services deployed in scenarios that aren't worth testing (e.g. >> >> > deprecated or unused things) (and deprecate the unused things) >> >> > 2) jobs themselves (I don't have any example beside scenario010 >> but >> >> > I'm sure there are more). >> >> > -- >> >> > Emilien Macchi >> >> > >> >> > >> >> > Thanks Alex, Emilien >> >> > >> >> > +1 to reviewing the catalog and adjusting things on an ongoing basis. >> >> > >> >> > All.. it looks like the issues with docker.io >> were >> >> > more of a flare up than a change in docker.io >> policy >> >> > or infrastructure [2]. The flare up started on July 27 8am utc and >> >> > ended on July 27 17:00 utc, see screenshots. >> >> >> >> The numbers of image prepare workers and its exponential fallback >> >> intervals should be also adjusted. I've analysed the log snippet [0] >> for >> >> the connection reset counts by workers versus the times the rate >> >> limiting was triggered. See the details in the reported bug [1]. >> >> >> >> tl;dr -- for an example 5 sec interval 03:55:31,379 - 03:55:36,110: >> >> >> >> Conn Reset Counts by a Worker PID: >> >> 3 58412 >> >> 2 58413 >> >> 3 58415 >> >> 3 58417 >> >> >> >> which seems too much of (workers*reconnects) and triggers rate limiting >> >> immediately. >> >> >> >> [0] >> >> >> https://13b475d7469ed7126ee9-28d4ad440f46f2186fe3f98464e57890.ssl.cf1.rackcdn.com/741228/6/check/tripleo-ci-centos-8-undercloud-containers/8e47836/logs/undercloud/var/log/tripleo-container-image-prepare.log >> >> >> >> [1] https://bugs.launchpad.net/tripleo/+bug/1889372 >> >> >> >> -- >> >> Best regards, >> >> Bogdan Dobrelya, >> >> Irc #bogdando >> >> >> > >> > FYI.. >> > >> > The issue w/ "too many requests" is back. Expect delays and failures >> in attempting to merge your patches upstream across all branches. The >> issue is being tracked as a critical issue. >> >> Working with the infra folks and we have identified the authorization >> header as causing issues when we're rediected from docker.io to >> cloudflare. I'll throw up a patch tomorrow to handle this case which >> should improve our usage of the cache. It needs some testing against >> other registries to ensure that we don't break authenticated fetching >> of resources. >> >> Thanks Alex! > FYI.. we have been revisited by the container pull issue, "too many requests". Alex has some fresh patches on it: https://review.opendev.org/#/q/status:open+project:openstack/tripleo-common+topic:bug/1889122 expect trouble in check and gate: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22429%20Client%20Error%3A%20Too%20Many%20Requests%20for%20url%3A%5C%22%20AND%20voting%3A1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From yumeng_bao at yahoo.com Wed Aug 5 17:06:03 2020 From: yumeng_bao at yahoo.com (yumeng bao) Date: Thu, 6 Aug 2020 01:06:03 +0800 Subject: [nova] If any spec freeze exception now? References: Message-ID: Hi gibi and all, I wanna mention the SRIOV SmartNIC Support Spec https://review.opendev.org/#/c/742785 This spec is proposed based on feedback from our PTG discussion, yet there are still open questions need to be nailed down. Since this spec involves nova neutron and cyborg, it will probably take a long time to get ideas from different aspects and reach an agreement. Can we keep this as an exception and keep review it to reach closer to an agreement? Hopefully we can reach an agreement in Victoria, and start to land in W. Xinran and I were trying to attend nova’s weekly meeting to discuss this spec, but the time too late for us. :( We will find if there is any other way to sync and response more actively to all your comments and feedback. And just to point out, nova operations support are still one of cyborg’s high priority goals in Victoria, we will keep focus on it and won’t sacrifice time of this goal. Regards， Yumeng -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Wed Aug 5 17:10:28 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 5 Aug 2020 12:10:28 -0500 Subject: [cinder] Change Volume Type Properties In-Reply-To: <24E5E9E3-6BF6-492C-BBBB-670DC070CF15@gmx.net> References: <24E5E9E3-6BF6-492C-BBBB-670DC070CF15@gmx.net> Message-ID: (updated subject with correct project name) On 8/5/20 9:33 AM, Marc Vorwerk wrote: > > Hi, > > I'm looking fora way to add the property /volume_backend_name/ to an > existing Volume Type which is in use. > > If I try to change this, I got the following error: > > root at control01rt:~# openstack volume type show test-type > > +--------------------+--------------------------------------+ > > | Field              | Value                                | > > +--------------------+--------------------------------------+ > > | access_project_ids | None                                 | > > | description        | None                                 | > > | id                 | 68febdad-e7b1-4d41-ba11-72d0e1a1cce0 | > > | is_public          | True                                 | > > | name               | test-type                            | > > | properties |                                      | > > | qos_specs_id       | None                         | > > +--------------------+--------------------------------------+ > > root at control01rt:~# openstack volume type set --property > volume_backend_name=ceph test-type > > Failed to set volume type property: Volume Type is currently in use. > (HTTP 400) (Request-ID: req-2b8f3829-5c16-42c3-ac57-01199688bd58) > > Command Failed: One or more of the operations failed > > root at control01rt:~# > > Problem what I see is, that there are instances/volumes which use this > volume type. > > Have anybody an idea, how I can add the /volume_backend_name/ property > to the existing Volume Type? > This is not allowed since the scheduler may have already scheduled these volumes to a different backend than the one you are now specifying in the extra specs. That would lead to a mismatch between the volumes and their volume type that isn't obvious. To get around this, you will need to create a new volume type with the volume_backend_name you want specified first. You can then retype your existing volumes to this new volume type. Assuming most or all of these volumes are already on that backend, the retype operation should just be a quick database update. If needed, you can then delete the original volume type that is no longer being used, then rename the new volume type to get back to using the same type name. This part isn't necessary, but you may need that if you've configured the old name as the default volume type in your cinder.conf file. Hope that helps. Sean -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Wed Aug 5 17:18:03 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 05 Aug 2020 18:18:03 +0100 Subject: [nova] If any spec freeze exception now? In-Reply-To: References: Message-ID: <3c98b824f04d8fa6fd07addd956a097db1aa20ba.camel@redhat.com> On Thu, 2020-08-06 at 01:06 +0800, yumeng bao wrote: > Hi gibi and all, > > I wanna mention the SRIOV SmartNIC Support Spec https://review.opendev.org/#/c/742785 > > This spec is proposed based on feedback from our PTG discussion, yet there are still open questions need to be nailed > down. Since this spec involves nova neutron and cyborg, it will probably take a long time to get ideas from different > aspects and reach an agreement. Can we keep this as an exception and keep review it to reach closer to an agreement? > Hopefully we can reach an agreement in Victoria, and start to land in W. well you dont need to close it without an exception the way exception work we normlly give a dealin of 1 week to finalise the spec after its granted so basiclaly unless you think we can fully agreee all the outstanding items before thursday week and merge it then you should just retarget the spec to the backlog or W release and keep working on it rather then ask for an excption. exception are only for thing that we expect to merge in victoria including the code. > > Xinran and I were trying to attend nova’s weekly meeting to discuss this spec, but the time too late for us. :( We > will find if there is any other way to sync and response more actively to all your comments and feedback. > > And just to point out, nova operations support are still one of cyborg’s high priority goals in Victoria, we will > keep focus on it and won’t sacrifice time of this goal. > > > Regards， > Yumeng From balazs.gibizer at est.tech Wed Aug 5 17:31:44 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Wed, 05 Aug 2020 19:31:44 +0200 Subject: [nova] If any spec freeze exception now? In-Reply-To: <3c98b824f04d8fa6fd07addd956a097db1aa20ba.camel@redhat.com> References: <3c98b824f04d8fa6fd07addd956a097db1aa20ba.camel@redhat.com> Message-ID: On Wed, Aug 5, 2020 at 18:18, Sean Mooney wrote: > On Thu, 2020-08-06 at 01:06 +0800, yumeng bao wrote: >> Hi gibi and all, >> >> I wanna mention the SRIOV SmartNIC Support Spec >> https://review.opendev.org/#/c/742785 >> >> This spec is proposed based on feedback from our PTG discussion, >> yet there are still open questions need to be nailed >> down. Since this spec involves nova neutron and cyborg, it will >> probably take a long time to get ideas from different >> aspects and reach an agreement. Can we keep this as an exception >> and keep review it to reach closer to an agreement? >> Hopefully we can reach an agreement in Victoria, and start to land >> in W. > well you dont need to close it without an exception > the way exception work we normlly give a dealin of 1 week to finalise > the spec after its granted > so basiclaly unless you think we can fully agreee all the outstanding > items before thursday week and merge it > then you should just retarget the spec to the backlog or W release > and keep working on it rather then ask for an > excption. exception are only for thing that we expect to merge in > victoria including the code. Agree with Sean. No need for an exception to continue discussing the spec during the V cycle. Having the spec freeze only means that now we know that the SmartNIC spec is not going to be implemented in V. Cheers, gibi >> >> Xinran and I were trying to attend nova’s weekly meeting to >> discuss this spec, but the time too late for us. :( We >> will find if there is any other way to sync and response more >> actively to all your comments and feedback. >> >> And just to point out, nova operations support are still one of >> cyborg’s high priority goals in Victoria, we will >> keep focus on it and won’t sacrifice time of this goal. >> >> >> Regards， >> Yumeng > From gouthampravi at gmail.com Wed Aug 5 18:54:41 2020 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Wed, 5 Aug 2020 11:54:41 -0700 Subject: [manila] Doc-a-thon event coming up next Thursday (Aug 6th) In-Reply-To: References: Message-ID: Thank you so much for putting this together Victoria, and Vida! As a reminder, we will not be meeting on IRC tomorrow (6th August 2020), but instead will be in https://meetpad.opendev.org/ManilaV-ReleaseDocAThon You can get to the etherpad link for the meeting by going to etherpad.opendev.org instead of meetpad.opendev.org: https://etherpad.opendev.org/p/ManilaV-ReleaseDocAThon Please bring any documentation issues to that meeting Hoping to see you all there! On Mon, Aug 3, 2020 at 12:20 PM Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> wrote: > Hi everybody, > > An update on this. We decided to take over the upstream meeting directly > and start *at* the slot of the Manila weekly meeting. We will join the > Jitsi bridge [0] at 3pm UTC time and start going through the list of bugs > we have in [1]. There is no finish time, you can join and leave the bridge > freely. We will also use IRC Freenode channel #openstack-manila if needed. > > If the time slot doesn't work for you (we are aware this is not a friendly > slot for EMEA/APAC), you can still go through the bug list in [1], claim a > bug and work on it. > > If things go well, we plan to do this again in a different slot so > everybody that wants to collaborate can do it. > > Looking forward to see you there, > > Cheers, > > V > > [0] https://meetpad.opendev.org/ManilaV-ReleaseDocAThon > [1] https://ethercalc.openstack.org/ur17jprbprxx > > On Fri, Jul 31, 2020 at 2:05 PM Victoria Martínez de la Cruz < > victoria at vmartinezdelacruz.com> wrote: > >> Hi folks, >> >> We will be organizing a doc-a-thon next Thursday, August 6th, with the >> main goal of improving our docs for the next release. We will be gathering >> on our Freenode channel #openstack-manila after our weekly meeting (3pm >> UTC) and also using a videoconference tool (exact details TBC) to go over a >> curated list of opened doc bugs we have here [0]. >> >> *Your* participation is truly valued, being you an already Manila >> contributor or if you are interested in contributing and you didn't know >> how, so looking forward to seeing you there :) >> >> Cheers, >> >> Victoria >> >> [0] https://ethercalc.openstack.org/ur17jprbprxx >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Wed Aug 5 19:45:28 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 5 Aug 2020 15:45:28 -0400 Subject: [tc] monthly meeting Message-ID: Hi everyone, Here’s the agenda for our monthly TC meeting. It will happen tomorrow (Thursday the 6th) at 1400 UTC in #openstack-tc and I will be your chair. If you can’t attend, please put your name in the “Apologies for Absence” section. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting * ACTIVE INITIATIVES - Follow up on past action items - OpenStack User-facing APIs and CLIs (belmoreira) - W cycle goal selection start - Completion of retirement cleanup (gmann) https://etherpad.opendev.org/p/tc-retirement-cleanup Thank you, Mohammed -- Mohammed Naser VEXXHOST, Inc. From skaplons at redhat.com Wed Aug 5 20:26:01 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 5 Aug 2020 22:26:01 +0200 Subject: [neutron] Drivers team meetings 7.08.2020 and 14.08.2020 Message-ID: Hi, I have internal training this Friday and I will be on PTO next Friday. Because of that I will not be able to chair neutron drivers meetings those 2 weeks. Currently we don’t have any new RFES to discuss so lets cancel those meetings and focus on implementation of RFEs already accepted. See You on drivers meeting on Friday, 21.08.2020 — Slawek Kaplonski Principal software engineer Red Hat From nate.johnston at redhat.com Wed Aug 5 22:02:23 2020 From: nate.johnston at redhat.com (Nate Johnston) Date: Wed, 5 Aug 2020 18:02:23 -0400 Subject: [tc][ptl] Proposal: Distributed Project Leadership Message-ID: <20200805220223.2elo2nrauzr575al@firewall> The governing structure for OpenStack projects has long been for a Project Technical Lead (PTL) to be elected to serve as a singular focus for that project. While the PTL role varies significantly from project to project, the PTL has many responsibilities for managing the development and release process for a project as well as representing the project both internally and externally. There have been a number of projects that have expressed an interest in functioning in a mode that would not require a PTL, but would rather devolve the responsibilities of the PTL into various liaison roles, that could in turn be held by one or more contributors. This topic was discussed by the TC at the recent virtual PTG, and we now have a proposal to put forth to the community for comment. Jean-Phillipe Evrard and I worked up a more detailed proposal for everyone to comment on and review. We are calling this the 'distributed project leadership model'. Most importantly, this is an opt-in process where projects interested in pursuing a distributed project leadership model should opt in to it, but for projects satisfied with the status quo nothing would change. I encourage everyone who is interested to examine the proposal and comment: https://review.opendev.org/744995 Thank you, Nate Johnston From jasonanderson at uchicago.edu Wed Aug 5 23:18:28 2020 From: jasonanderson at uchicago.edu (Jason Anderson) Date: Wed, 5 Aug 2020 23:18:28 +0000 Subject: [swift][ceph] Container ACLs don't seem to be respected on Ceph RGW In-Reply-To: <757BCAB6-CA22-439E-9C0C-BE4DEC7B7927@uchicago.edu> References: <757BCAB6-CA22-439E-9C0C-BE4DEC7B7927@uchicago.edu> Message-ID: <48C1BF75-211F-4F7C-ABCA-D59777C469A8@uchicago.edu> As an update, I think one of my problems was the dangling space after “_member_” in my ACL list, which was quite painful to discover. I think it was breaking the matching of my user, which had the role _member_ assigned. And, it does look like read ACLs must be of the form “.r:*”, despite the Ceph docs. With this in place, public read ACL works. I still can’t get write ACLs to work though, and from looking at the code[1] I’m not sure how it’s supposed to work. /Jason [1]: https://github.com/ceph/ceph/blob/f52fb99f011d9b124ed91f3d001d3551e9a10c8d/src/rgw/rgw_acl_swift.cc > On Aug 4, 2020, at 10:49 PM, Jason Anderson wrote: > > Hi all, > > Just scratching my head at this for a while and though I’d ask here in case it saves some time. I’m running a Ceph cluster on the Nautilus release and it’s running Swift via the rgw. I have Keystone authentication turned on. Everything works fine in the normal case of creating containers, uploading files, listing containers, etc. > > However, I notice that ACLs don’t seem to work. I am not overriding "rgw enforce swift acls”, so it is set to the default of true. I can’t seem to share a container or make it public. > > (Side note, confusingly, the Ceph implementation has a different syntax for public read/write containers, ‘*’ as opposed to ‘*:*’ for public write for example.) > > Here’s what I’m doing > > (as admin) > swift post —write-acl ‘*’ —read-acl ‘*’ public-container > swift stat public-container > Account: v1 > Container: public-container > Objects: 1 > Bytes: 5801 > Read ACL: * > Write ACL: * > Sync To: > Sync Key: > X-Timestamp: 1595883106.23179 > X-Container-Bytes-Used-Actual: 8192 > X-Storage-Policy: default-placement > X-Storage-Class: STANDARD > Last-Modified: Wed, 05 Aug 2020 03:42:11 GMT > X-Trans-Id: tx000000000000000662156-005f2a2bea-23478-default > X-Openstack-Request-Id: tx000000000000000662156-005f2a2bea-23478-default > Accept-Ranges: bytes > Content-Type: text/plain; charset=utf-8 > > (as non-admin) > swift upload public-container test.txt > Warning: failed to create container 'public-container': 409 Conflict: BucketAlreadyExists > Object HEAD failed: https://ceph.example.org:7480/swift/v1/public-container/README.md 403 Forbidden > > swift list public-container > Container GET failed: https://ceph.example.org:7480/swift/v1/public-container?format=json 403 Forbidden [first 60 chars of response] b'{"Code":"AccessDenied","BucketName”:”public-container","RequestId":"tx0' > Failed Transaction ID: tx000000000000000662162-005f2a2c2a-23478-default > > What am I missing? Thanks in advance! > > /Jason From jasonanderson at uchicago.edu Wed Aug 5 23:19:53 2020 From: jasonanderson at uchicago.edu (Jason Anderson) Date: Wed, 5 Aug 2020 23:19:53 +0000 Subject: [swift][ceph] Container ACLs don't seem to be respected on Ceph RGW In-Reply-To: <48C1BF75-211F-4F7C-ABCA-D59777C469A8@uchicago.edu> References: <757BCAB6-CA22-439E-9C0C-BE4DEC7B7927@uchicago.edu> <48C1BF75-211F-4F7C-ABCA-D59777C469A8@uchicago.edu> Message-ID: On Aug 5, 2020, at 6:18 PM, Jason Anderson > wrote: As an update, I think one of my problems was the dangling space after “_member_” in my ACL list, which was quite painful to discover. I think it was breaking the matching of my user, which had the role _member_ assigned. Sorry, I meant in my Ceph configuration, which had this line in the rgw section: rgw keystone accepted roles = _member_ , Member, admin And, it does look like read ACLs must be of the form “.r:*”, despite the Ceph docs. With this in place, public read ACL works. I still can’t get write ACLs to work though, and from looking at the code[1] I’m not sure how it’s supposed to work. /Jason [1]: https://github.com/ceph/ceph/blob/f52fb99f011d9b124ed91f3d001d3551e9a10c8d/src/rgw/rgw_acl_swift.cc On Aug 4, 2020, at 10:49 PM, Jason Anderson > wrote: Hi all, Just scratching my head at this for a while and though I’d ask here in case it saves some time. I’m running a Ceph cluster on the Nautilus release and it’s running Swift via the rgw. I have Keystone authentication turned on. Everything works fine in the normal case of creating containers, uploading files, listing containers, etc. However, I notice that ACLs don’t seem to work. I am not overriding "rgw enforce swift acls”, so it is set to the default of true. I can’t seem to share a container or make it public. (Side note, confusingly, the Ceph implementation has a different syntax for public read/write containers, ‘*’ as opposed to ‘*:*’ for public write for example.) Here’s what I’m doing (as admin) swift post —write-acl ‘*’ —read-acl ‘*’ public-container swift stat public-container Account: v1 Container: public-container Objects: 1 Bytes: 5801 Read ACL: * Write ACL: * Sync To: Sync Key: X-Timestamp: 1595883106.23179 X-Container-Bytes-Used-Actual: 8192 X-Storage-Policy: default-placement X-Storage-Class: STANDARD Last-Modified: Wed, 05 Aug 2020 03:42:11 GMT X-Trans-Id: tx000000000000000662156-005f2a2bea-23478-default X-Openstack-Request-Id: tx000000000000000662156-005f2a2bea-23478-default Accept-Ranges: bytes Content-Type: text/plain; charset=utf-8 (as non-admin) swift upload public-container test.txt Warning: failed to create container 'public-container': 409 Conflict: BucketAlreadyExists Object HEAD failed: https://ceph.example.org:7480/swift/v1/public-container/README.md 403 Forbidden swift list public-container Container GET failed: https://ceph.example.org:7480/swift/v1/public-container?format=json 403 Forbidden [first 60 chars of response] b'{"Code":"AccessDenied","BucketName”:”public-container","RequestId":"tx0' Failed Transaction ID: tx000000000000000662162-005f2a2c2a-23478-default What am I missing? Thanks in advance! /Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: From yumeng_bao at yahoo.com Thu Aug 6 02:11:19 2020 From: yumeng_bao at yahoo.com (yumeng bao) Date: Thu, 6 Aug 2020 10:11:19 +0800 Subject: [nova] If any spec freeze exception now? In-Reply-To: References: Message-ID: <681905FE-A165-42B2-9D67-0AF259F62E5A@yahoo.com> Ok. That make sense to me! Thanks gibi and Sean! Regards， Yumeng > On Aug 6, 2020, at 1:31 AM, Balázs Gibizer wrote: > >  > >> On Wed, Aug 5, 2020 at 18:18, Sean Mooney wrote: >>> On Thu, 2020-08-06 at 01:06 +0800, yumeng bao wrote: >>> Hi gibi and all, >>> I wanna mention the SRIOV SmartNIC Support Spec https://review.opendev.org/#/c/742785 >>> This spec is proposed based on feedback from our PTG discussion, yet there are still open questions need to be nailed >>> down. Since this spec involves nova neutron and cyborg, it will probably take a long time to get ideas from different >>> aspects and reach an agreement. Can we keep this as an exception and keep review it to reach closer to an agreement? >>> Hopefully we can reach an agreement in Victoria, and start to land in W. >> well you dont need to close it without an exception >> the way exception work we normlly give a dealin of 1 week to finalise the spec after its granted >> so basiclaly unless you think we can fully agreee all the outstanding items before thursday week and merge it >> then you should just retarget the spec to the backlog or W release and keep working on it rather then ask for an >> excption. exception are only for thing that we expect to merge in victoria including the code. > > Agree with Sean. No need for an exception to continue discussing the spec during the V cycle. Having the spec freeze only means that now we know that the SmartNIC spec is not going to be implemented in V. > > Cheers, > gibi > >>> Xinran and I were trying to attend nova’s weekly meeting to discuss this spec, but the time too late for us. :( We >>> will find if there is any other way to sync and response more actively to all your comments and feedback. >>> And just to point out, nova operations support are still one of cyborg’s high priority goals in Victoria, we will >>> keep focus on it and won’t sacrifice time of this goal. >>> Regards， >>> Yumeng > > From emiller at genesishosting.com Thu Aug 6 04:28:28 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 5 Aug 2020 23:28:28 -0500 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0481447A@gmsxchsvr01.thecreation.com> > Do you need full host block devices to be provided to the instance? No - a thin-provisioned LV in LVM would be best. > The LVM imagebackend will just provision LVs on top of the provided VG so > there's no direct mapping to a full host block device with this approach. That's perfect! > Yeah that's a common pitfall when using LVM based ephemeral disks that > contain additional LVM PVs/VGs/LVs etc. You need to ensure that the host is > configured to not scan these LVs in order for their PVs/VGs/LVs etc to remain > hidden from the host: Thanks for the link! I will let everyone know how testing goes. Eric From emiller at genesishosting.com Thu Aug 6 04:30:01 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 5 Aug 2020 23:30:01 -0500 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0481447B@gmsxchsvr01.thecreation.com> > That said there's no real alternative available at the moment. > well one alternitive to nova providing local lvm storage is to use > the cinder lvm driver but install it on all compute nodes then > use the cidner InstanceLocalityFilter to ensure the volume is alocated form > the host > the vm is on. > https://docs.openstack.org/cinder/latest/configuration/block- > storage/scheduler-filters.html#instancelocalityfilter > on drawback to this is that if the if the vm is moved i think you would need to > also migrate the cinder volume > seperatly afterwards. I wasn't aware of the InstanceLocalityFilter, so thank you for mentioning it! Eric From emiller at genesishosting.com Thu Aug 6 04:39:50 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 5 Aug 2020 23:39:50 -0500 Subject: [cinder][nova] Local storage in compute node In-Reply-To: References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0481447C@gmsxchsvr01.thecreation.com> From: Donny Davis [mailto:donny at fortnebula.com] Sent: Wednesday, August 05, 2020 8:23 AM > If you have any other questions I am happy to help where I can - I have been working with all nvme stuff for the last couple years and have gotten something into prod for about 1 year with it (maybe a little longer).  > From what I can tell, getting max performance from nvme for an instance is a non-trivial task because it's just so much faster than the rest of the stack and careful considerations must be taken to get the most out of it.  > I am curious to see where you take this Eric Thanks for the response! We also use Ceph with NVMe SSDs, with many NVMe namespaces with one OSD per namespace, to fully utilize the SSDs. You are right - they are so fast that they are literally faster than any application can use. They are great for multi-tenant environments, though, where it's usually better to have more hardware than people can utilize. My first test is to try using the Libvirt "images_type=lvm" method to see how well it works. I will report back... Eric From emiller at genesishosting.com Thu Aug 6 04:53:49 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 5 Aug 2020 23:53:49 -0500 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0481447D@gmsxchsvr01.thecreation.com> > From: Sean Mooney [mailto:smooney at redhat.com] > Sent: Wednesday, August 05, 2020 8:01 AM > yes that works well with the default flat/qcow file format > i assume there was a reason this was not the starting point. > the nova lvm backend i think does not supprot thin provisioning > so fi you did the same thing creating the volume group on the nvme deivce > you would technically get better write performance after the vm is booted > but > the vm spwan is slower since we cant take advantage of thin providioning > and > each root disk need to be copided form the cahced image. I wasn't aware that the nova LVM backend ([libvirt]/images_type = lvm) didn't support thin provisioned LV's. However, I do see that the "sparse_logical_volumes" parameter indicates it has been deprecated: https://docs.openstack.org/nova/rocky/configuration/config.html#libvirt.sparse_logical_volumes That would definitely be a downer. > so just monting the nova data directory on an nvme driver or a raid of nvme > drives > works well and is simple to do. Maybe we should consider doing this instead. I'll test with the Nova LVM backend first. > so there are trade off with both appoches. > generally i recommend using local sotrage e.g. the vm root disk or ephemeral > disk for fast scratchpad space > to work on data bug persitie all relevent data permently via cinder volumes. > that requires you to understand which block > devices a local and which are remote but it give you the best of both worlds. Our use case simply requires high-speed non-redundant storage for self-replicating applications like Couchbase, Cassandra, MongoDB, etc. or very inexpensive VMs that are backed-up often and can withstand the downtime when restoring from backup. That will be one more requirement (or rather a very nice to have), is to be able to create images (backups) of the local storage onto object storage, so hopefully "openstack server backup create" works like it does with rbd-backed Nova-managed persistent storage. I will let you know what I find out! Thanks everyone! Eric From marino.mrc at gmail.com Thu Aug 6 07:32:21 2020 From: marino.mrc at gmail.com (Marco Marino) Date: Thu, 6 Aug 2020 09:32:21 +0200 Subject: [tripleo] Deploy overcloud without provisioning Message-ID: Hi, I'm trying to deploy an overcloud using tripleo with pre provisioned nodes. My configuration is quite simple: - 1 controller and 1 compute nodes on which I already installed CentOS 8.2 - Both nodes have a dedicated idrac interface with an ip in 192.168.199.0/24. Please note that this interface is not visible with "ip a" or "ifconfig". It's a dedicated IDRAC interface - Both nodes have a NIC configured in the subnet 192.168.199.0/24 (192.168.199.200 and 192.168.199.201) - Undercloud uses 192.168.199.0/24 as pxe/provisioning network (but I don't need provisioning) Question: should I import nodes with "openstack overcloud node import nodes.yaml" even if I don't need the provisioning step? Furthermore, on the undercloud I created one file: /home/stack/templates/node-info.yaml with the following content parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 1 ComputeCount: 1 Question: How can I specify that "node X with ip Y should be used as a controller and node Z with ip K should be used as a compute"?? Should I set the property with the following command? openstack baremetal node set --property capabilities='profile:control' controller1 Thank you, Marco -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Thu Aug 6 07:46:01 2020 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 6 Aug 2020 08:46:01 +0100 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: On Wed, 5 Aug 2020 at 16:16, Michael Johnson wrote: > Looking at that error, it appears that the lb-mgmt-net is not setup > correctly. The Octavia controller containers are not able to reach the > amphora instances on the lb-mgmt-net subnet. > > I don't know how kolla is setup to connect the containers to the neutron > lb-mgmt-net network. Maybe the above documents will help with that. > Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. > Michael > > On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard wrote: > >> >> >> On Tue, 4 Aug 2020 at 16:58, Monika Samal >> wrote: >> >>> Hello Guys, >>> >>> With Michaels help I was able to solve the problem but now there is >>> another error I was able to create my network on vlan but still error >>> persist. PFB the logs: >>> >>> http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ >>> >>> Kindly help >>> >>> regards, >>> Monika >>> ------------------------------ >>> *From:* Michael Johnson >>> *Sent:* Monday, August 3, 2020 9:10 PM >>> *To:* Fabian Zimmermann >>> *Cc:* Monika Samal ; openstack-discuss < >>> openstack-discuss at lists.openstack.org> >>> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >>> balancer >>> >>> Yeah, it looks like nova is failing to boot the instance. >>> >>> Check this setting in your octavia.conf files: >>> https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id >>> >>> Also, if kolla-ansible didn't set both of these values correctly, please >>> open bug reports for kolla-ansible. These all should have been configured >>> by the deployment tool. >>> >>> >> I wasn't following this thread due to no [kolla] tag, but here are the >> recently added docs for Octavia in kolla [1]. Note >> the octavia_service_auth_project variable which was added to migrate from >> the admin project to the service project for octavia resources. We're >> lacking proper automation for the flavor, image etc, but it is being worked >> on in Victoria [2]. >> >> [1] >> https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html >> [2] https://review.opendev.org/740180 >> >> Michael >>> >>> On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann >>> wrote: >>> >>> Seems like the flavor is missing or empty '' - check for typos and >>> enable debug. >>> >>> Check if the nova req contains valid information/flavor. >>> >>> Fabian >>> >>> Monika Samal schrieb am Mo., 3. Aug. 2020, >>> 15:46: >>> >>> It's registered >>> >>> Get Outlook for Android >>> ------------------------------ >>> *From:* Fabian Zimmermann >>> *Sent:* Monday, August 3, 2020 7:08:21 PM >>> *To:* Monika Samal ; openstack-discuss < >>> openstack-discuss at lists.openstack.org> >>> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >>> balancer >>> >>> Did you check the (nova) flavor you use in octavia. >>> >>> Fabian >>> >>> Monika Samal schrieb am Mo., 3. Aug. 2020, >>> 10:53: >>> >>> After Michael suggestion I was able to create load balancer but there is >>> error in status. >>> >>> >>> >>> PFB the error link: >>> >>> http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ >>> ------------------------------ >>> *From:* Monika Samal >>> *Sent:* Monday, August 3, 2020 2:08 PM >>> *To:* Michael Johnson >>> *Cc:* Fabian Zimmermann ; Amy Marrich < >>> amy at demarco.com>; openstack-discuss < >>> openstack-discuss at lists.openstack.org>; community at lists.openstack.org < >>> community at lists.openstack.org> >>> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >>> balancer >>> >>> Thanks a ton Michael for helping me out >>> ------------------------------ >>> *From:* Michael Johnson >>> *Sent:* Friday, July 31, 2020 3:57 AM >>> *To:* Monika Samal >>> *Cc:* Fabian Zimmermann ; Amy Marrich < >>> amy at demarco.com>; openstack-discuss < >>> openstack-discuss at lists.openstack.org>; community at lists.openstack.org < >>> community at lists.openstack.org> >>> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >>> balancer >>> >>> Just to close the loop on this, the octavia.conf file had >>> "project_name = admin" instead of "project_name = service" in the >>> [service_auth] section. This was causing the keystone errors when >>> Octavia was communicating with neutron. >>> >>> I don't know if that is a bug in kolla-ansible or was just a local >>> configuration issue. >>> >>> Michael >>> >>> On Thu, Jul 30, 2020 at 1:39 PM Monika Samal >>> wrote: >>> > >>> > Hello Fabian,, >>> > >>> > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ >>> > >>> > Regards, >>> > Monika >>> > ________________________________ >>> > From: Fabian Zimmermann >>> > Sent: Friday, July 31, 2020 1:57 AM >>> > To: Monika Samal >>> > Cc: Michael Johnson ; Amy Marrich < >>> amy at demarco.com>; openstack-discuss < >>> openstack-discuss at lists.openstack.org>; community at lists.openstack.org < >>> community at lists.openstack.org> >>> > Subject: Re: [openstack-community] Octavia :; Unable to create load >>> balancer >>> > >>> > Hi, >>> > >>> > just to debug, could you replace the auth_type password with >>> v3password? >>> > >>> > And do a curl against your :5000 and :35357 urls and paste the output. >>> > >>> > Fabian >>> > >>> > Monika Samal schrieb am Do., 30. Juli >>> 2020, 22:15: >>> > >>> > Hello Fabian, >>> > >>> > http://paste.openstack.org/show/796477/ >>> > >>> > Thanks, >>> > Monika >>> > ________________________________ >>> > From: Fabian Zimmermann >>> > Sent: Friday, July 31, 2020 1:38 AM >>> > To: Monika Samal >>> > Cc: Michael Johnson ; Amy Marrich < >>> amy at demarco.com>; openstack-discuss < >>> openstack-discuss at lists.openstack.org>; community at lists.openstack.org < >>> community at lists.openstack.org> >>> > Subject: Re: [openstack-community] Octavia :; Unable to create load >>> balancer >>> > >>> > The sections should be >>> > >>> > service_auth >>> > keystone_authtoken >>> > >>> > if i read the docs correctly. Maybe you can just paste your config >>> (remove/change passwords) to paste.openstack.org and post the link? >>> > >>> > Fabian >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From e0ne at e0ne.info Thu Aug 6 07:46:25 2020 From: e0ne at e0ne.info (Ivan Kolodyazhny) Date: Thu, 6 Aug 2020 10:46:25 +0300 Subject: [horizon] Victoria virtual mid-cycle poll In-Reply-To: References: Message-ID: Hi everybody, According to our poll [2] we'll have a one-hour mid-cycle poll today at 13.00 UTC. I'll share a Zoom link before the meeting today. We're going to discuss current release priorities [3] and our future plans. [2] https://doodle.com/poll/dkmsai49v4zzpca2 [3] https://etherpad.opendev.org/p/horizon-release-priorities Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Thu, Jul 30, 2020 at 1:00 PM Ivan Kolodyazhny wrote: > Hi team, > > If something can go wrong, it will definitely go wrong. > It means that I did a mistake in my original mail and sent you > completely wrong dates:(. > > Horizon Virtual mid-cycle is supposed to be next week Aug 5-7. I'm > planning to have a single one-hour session. > In case, if we've got a lot of participants and topic to discuss, we can > schedule one more session a week or two weeks later. > > Here is a correct poll: https://doodle.com/poll/dkmsai49v4zzpca2 > > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > > > On Wed, Jul 22, 2020 at 10:26 AM Ivan Kolodyazhny wrote: > >> Hi team, >> >> As discussed at Horizon's Virtual PTG [1], we'll have a virtual mid-cycle >> meeting around Victoria-2 milestone. >> >> We'll discuss Horizon current cycle development priorities and the future >> of Horizon with modern JS frameworks. >> >> Please indicate your availability to meet for the first session, which >> will be held during the week of July 27-31: >> >> https://doodle.com/poll/3neps94amcreaw8q >> >> Please respond before 12:00 UTC on Tuesday 4 August. >> >> [1] https://etherpad.opendev.org/p/horizon-v-ptg >> >> Regards, >> Ivan Kolodyazhny, >> http://blog.e0ne.info/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From e0ne at e0ne.info Thu Aug 6 07:48:39 2020 From: e0ne at e0ne.info (Ivan Kolodyazhny) Date: Thu, 6 Aug 2020 10:48:39 +0300 Subject: [horizon] patternfly? In-Reply-To: <20200628140127.GA502608@straylight.m.ringlet.net> References: <20200623005343.rkgtee524s5tl7kx@yuggoth.org> <115da5a2-0bf1-4ec0-8ba6-0b3d1f3b9ab7@debian.org> <7406ea49-37ed-da56-24c5-786c342e632e@catalyst.net.nz> <20200628140127.GA502608@straylight.m.ringlet.net> Message-ID: Hi, We can discuss Horizon v next today during our mid-cycle call: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016346.html Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Sun, Jun 28, 2020 at 5:03 PM Peter Pentchev wrote: > On Wed, Jun 24, 2020 at 02:07:06PM +1200, Adrian Turjak wrote: > > On 24/06/20 1:05 am, Thomas Goirand wrote: > > > Anyone dismissing how huge of a problem this is, isn't doing serious > > > programming, for serious production use. That person would just be > doing > > > script kiddy work in the playground. Yes, it works, yes, it's shiny and > > > all. The upstream code may even be super nice, well written and all. > But > > > it's *NOT* serious to put such JS bundling approach in production. > > And yet people are running huge projects in production like this just > > fine. So clearly people are finding sane practices around it that give > > them enough security to feel safe that don't involve packaging each npm > > requirement as an OS package. How exactly are all the huge powerhouses > > doing it then when most of the internet's front end is giant js bundles > > built from npm dependencies? How does gitlab do it for their omnibus? > > From a cursory glance it did seem like they did use npm, and had a rake > > job to compile the js. Gitlab most definitely isn't "script kiddy work". > > > > I'm mostly a python dev, so I don't deal with npm often. When it comes > > to python though, other than underlying OS packages for python/pip > > itself, I use pip for installing my versions (in a venv or container). > > I've had too many painful cases of weird OS package versions, and I > > dislike the idea of relying on the OS when there is a perfectly valid > > and working package management system for my application requirements. I > > can audit the versions installed against known CVEs, and because I > > control the requirements, I can ensure I'm never using out of date > > libraries. > > > > Javascript and npm is only different because the sheer number of > > dependencies. Which is terrifying, don't get me wrong, but you can lock > > versions, you can audit them against CVEs, you can be warned if they are > > out of date. How other than by sheer scale is it really worse than pip > > if you follow some standards and a consistent process? > > What Thomas is trying to say, and I think other people in this thread > also agreed with, is that it's not "only" because of the sheer number of > dependencies. My personal opinion is that the Javascript ecosystem is > currently where Perl/CPAN was 25 years ago, Python was between 15 and 20 > years ago, and Ruby was 10-15 years ago: quite popular, attracting many > people who "just want to write a couple of lines of code to solve this > simple task", and, as a very logical consequence, full of small > libraries that various people developed to fix their own itches and just > released out into the wild without very much thought of long-term > maintenance. Now, this has several consequences (most of them have been > pointed out already): > > - there are many (not all, but many) developers who do not even try to > keep their own libraries backwards-compatible > > - there are many (not all, but many) developers who, once they have > written a piece of code that uses three libraries from other people, > do not really bother to follow the development of those libraries and > try to make their own piece of code compatible with their new versions > (this holds even more if there are not three, but fifteen libraries > from other people; it can be a bit hard to keep up with them all if > their authors do not care about API stability) > > - there are many libraries that lock the versions of their dependencies, > thus bringing back what was once known as "DLL hell", over and over > and over again (and yes, this happens in other languages, too) > > - there are many, many, *many* libraries that solve the same problems > over and over again in subtly different ways, either because their > authors were not aware of the other implementations or because said > other implementations could not exactly scratch the author's itch and > it was easier to write their own instead of spend some more time > trying to adapt the other one and propose changes to its author > (and, yes, I myself have been guilty of this in C, Perl, and Python > projects in the past; NIH is a very, very easy slope to slide down > along) > > I *think* that, with time, many Javascript developers will realize that > this situation is unsustainable in the long term, and, one by one, they > will start doing what C/C++, Perl, Python, and Ruby people have been > doing for some time now: > > - start thinking about backwards compatibility, think really hard before > making an incompatible change and, if they really have to, use > something like semantic versioning (not necessarily exactly semver, > but something similar) to signal the API breakage > > - once the authors of the libraries they depend on start doing this, > start encoding loose version requirements (not strictly pinned), such > as "dep >= 1.2.1, dep < 3". This is already done in many Python > packages, and OpenStack's upper-constraints machinery is a wonderful > example of how this can be maintained in a conservative manner that > virtually guarantees that the end result will work. > > - start wondering whether it is really worth it to maintain their own > pet implementation instead of extending a more-widely-used one, thus > eventually having the community settle on a well-known set of > more-or-less comprehensive and very widely tested packages for most > tasks. Once this happens, the authors of these widely-used libraries > absolutely *have* to keep some degree of backwards compatibility and > some kind of reasonable versioning scheme to signal changes. > > So, I'm kind of optimistic and I believe that, with time, the Javascript > ecosystem will become better. Unfortunately, this process has taken many > years for the other languages I've mentioned, and is not really fully > complete in any of them: any module repository has its share of > mostly-maintained reimplementations of various shapes and sizes of the > wheel. So I guess the point of all this was mostly to explain the > problem (once again) more than propose any short-term solutions :/ > > G'luck, > Peter > > -- > Peter Pentchev roam at ringlet.net roam at debian.org pp at storpool.com > PGP key: http://people.FreeBSD.org/~roam/roam.key.asc > Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ramishra at redhat.com Thu Aug 6 07:51:53 2020 From: ramishra at redhat.com (Rabi Mishra) Date: Thu, 6 Aug 2020 13:21:53 +0530 Subject: [tripleo] Deploy overcloud without provisioning In-Reply-To: References: Message-ID: On Thu, Aug 6, 2020 at 1:07 PM Marco Marino wrote: > Hi, > I'm trying to deploy an overcloud using tripleo with pre provisioned > nodes. My configuration is quite simple: > - 1 controller and 1 compute nodes on which I already installed CentOS 8.2 > - Both nodes have a dedicated idrac interface with an ip in > 192.168.199.0/24. Please note that this interface is not visible with "ip > a" or "ifconfig". It's a dedicated IDRAC interface > - Both nodes have a NIC configured in the subnet 192.168.199.0/24 > (192.168.199.200 and 192.168.199.201) > - Undercloud uses 192.168.199.0/24 as pxe/provisioning network (but I > don't need provisioning) > > Question: should I import nodes with "openstack overcloud node import > nodes.yaml" even if I don't need the provisioning step? > > Furthermore, on the undercloud I created one file: > /home/stack/templates/node-info.yaml with the following content > > parameter_defaults: > OvercloudControllerFlavor: control > OvercloudComputeFlavor: compute > ControllerCount: 1 > ComputeCount: 1 > > Question: How can I specify that "node X with ip Y should be used as a > controller and node Z with ip K should be used as a compute"?? > With pre-provisioned nodes (DeployedServer), you would need to specify HostnameMap and DeployedServerPortMap parameters that would map the pre-provisioned hosts and ctlplane ips. Please check documentation[1] for more details. [1] https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deployed_server.html#deployed-server-with-config-download > Should I set the property with the following command? > openstack baremetal node set --property capabilities='profile:control' > controller1 > > Thank you, > Marco > > -- Regards, Rabi Mishra -------------- next part -------------- An HTML attachment was scrubbed... URL: From emiller at genesishosting.com Thu Aug 6 07:57:54 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Thu, 6 Aug 2020 02:57:54 -0500 Subject: [cinder][nova] Local storage in compute node References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0481447F@gmsxchsvr01.thecreation.com> > No - a thin-provisioned LV in LVM would be best. From testing, it looks like thick-provisioned is the only choice at this stage. That's fine. > I will let everyone know how testing goes. So far, everything is working perfectly with Nova using LVM. It was a quick configuration and it did exactly what I expected, which is always nice. :) As far as performance goes, it is decent, but not stellar. Of course, I'm comparing crazy fast native NVMe storage in RAID 0 across 4 x Micron 9300 SSDs (using md as the underlying physical volume in LVM) to virtualized storage. Some numbers from fio, just to get an idea for how good/bad the IOPS will be: Configuration: 32 core EPYC 7502P with 512GiB of RAM - CentOS 7 latest updates - Kolla Ansible (Stein) deployment 32 vCPU VM with 64GiB of RAM 32 x 10GiB test files (I'm using file tests, not raw device tests, so not optimal, but easiest when the VM root disk is the test disk) iodepth=10 numofjobs=32 time=30 (seconds) The VM was deployed using a qcow2 image, then deployed as a raw image, to see the difference in performance. There was none, which makes sense, since I'm pretty sure the qcow2 image was decompressed and stored in the LVM logical volume - so both tests were measuring the same thing. Bare metal (random 4KiB reads): 8066MiB/sec 154.34 microsecond avg latency 2.065 million IOPS VM qcow2 (random 4KiB reads): 589MiB/sec 2122.10 microsecond avg latency 151k IOPS Bare metal (random 4KiB writes): 4940MiB/sec 252.44 microsecond avg latency 1.265 million IOPS VM qcow2 (random 4KiB writes): 589MiB/sec 2119.16 microsecond avg latency 151k IOPS Since the read and write VM results are nearly identical, my assumption is that the emulation layer is the bottleneck. CPUs in the VM were all at 55% utilization (all kernel usage). The qemu process on the bare metal machine indicated 1600% (or so) CPU utilization. Below are runs with sequential 1MiB block tests Bare metal (sequential 1MiB reads): 13.3GiB/sec 23446.43 microsecond avg latency 13.7k IOPS VM qcow2 (sequential 1MiB reads): 8378MiB/sec 38164.52 microsecond avg latency 8377 IOPS Bare metal (sequential 1MiB writes): 8098MiB/sec 39488.00 microsecond avg latency 8097 million IOPS VM qcow2 (sequential 1MiB writes): 8087MiB/sec 39534.96 microsecond avg latency 8087 IOPS Amazing that a VM can move 8GiB/sec to/from storage. :) However, IOPS limits are a bit disappointing when compared to bare metal (but this is relative since 151k IOPS is quite a bit!). Not sure if additional "iothreads" QEMU would help, but that is not set in the Libvirt XML file, and I don't see any way to use Nova to set it. The Libvirt XML for the disk appears as:
Any suggestions for improvement? I "think" that the "images_type = flat" option in nova.conf indicates that images are stored in the /var/lib/nova/instances/* directories? If so, that might be an option, but since we're using Kolla, that directory (or rather /var/lib/nova) is currently a docker volume. So, it might be necessary to mount the NVMe storage at its respective /var/lib/docker/volumes/nova_compute/_data/instances directory. Not sure if the "flat" option will be any faster, especially since Docker would be another layer to go through. Any opinions? Thanks! Eric From marino.mrc at gmail.com Thu Aug 6 08:05:21 2020 From: marino.mrc at gmail.com (Marco Marino) Date: Thu, 6 Aug 2020 10:05:21 +0200 Subject: [tripleo] Deploy overcloud without provisioning In-Reply-To: References: Message-ID: Thank you Rabi, but node import is mandatory or not? Bare Metal part is useful for power management and I'd like to maintain this feature. Marco Il giorno gio 6 ago 2020 alle ore 09:52 Rabi Mishra ha scritto: > > > On Thu, Aug 6, 2020 at 1:07 PM Marco Marino wrote: > >> Hi, >> I'm trying to deploy an overcloud using tripleo with pre provisioned >> nodes. My configuration is quite simple: >> - 1 controller and 1 compute nodes on which I already installed CentOS 8.2 >> - Both nodes have a dedicated idrac interface with an ip in >> 192.168.199.0/24. Please note that this interface is not visible with >> "ip a" or "ifconfig". It's a dedicated IDRAC interface >> - Both nodes have a NIC configured in the subnet 192.168.199.0/24 >> (192.168.199.200 and 192.168.199.201) >> - Undercloud uses 192.168.199.0/24 as pxe/provisioning network (but I >> don't need provisioning) >> >> Question: should I import nodes with "openstack overcloud node import >> nodes.yaml" even if I don't need the provisioning step? >> >> Furthermore, on the undercloud I created one file: >> /home/stack/templates/node-info.yaml with the following content >> >> parameter_defaults: >> OvercloudControllerFlavor: control >> OvercloudComputeFlavor: compute >> ControllerCount: 1 >> ComputeCount: 1 >> >> > Question: How can I specify that "node X with ip Y should be used as a >> controller and node Z with ip K should be used as a compute"?? >> > > With pre-provisioned nodes (DeployedServer), you would need to specify > HostnameMap and DeployedServerPortMap parameters that would map the > pre-provisioned hosts and ctlplane ips. > > Please check documentation[1] for more details. > > [1] > https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deployed_server.html#deployed-server-with-config-download > > >> Should I set the property with the following command? >> openstack baremetal node set --property capabilities='profile:control' >> controller1 >> >> Thank you, >> Marco >> >> > > -- > Regards, > Rabi Mishra > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ramishra at redhat.com Thu Aug 6 08:23:32 2020 From: ramishra at redhat.com (Rabi Mishra) Date: Thu, 6 Aug 2020 13:53:32 +0530 Subject: [tripleo] Deploy overcloud without provisioning In-Reply-To: References: Message-ID: On Thu, Aug 6, 2020 at 1:35 PM Marco Marino wrote: > Thank you Rabi, > but node import is mandatory or not? Bare Metal part is useful for power > management and I'd like to maintain this feature. > > AFAIK, the intent of using pre-provisioned nodes is to create an overcloud without power management control, among other things. Node import is not required when Tripleo(Ironic) is not doing the provisioning. I don't know if there are ways for Ironic to do power management of pre-provisioned nodes. Someone else may have a better answer. > Marco > > Il giorno gio 6 ago 2020 alle ore 09:52 Rabi Mishra > ha scritto: > >> >> >> On Thu, Aug 6, 2020 at 1:07 PM Marco Marino wrote: >> >>> Hi, >>> I'm trying to deploy an overcloud using tripleo with pre provisioned >>> nodes. My configuration is quite simple: >>> - 1 controller and 1 compute nodes on which I already installed CentOS >>> 8.2 >>> - Both nodes have a dedicated idrac interface with an ip in >>> 192.168.199.0/24. Please note that this interface is not visible with >>> "ip a" or "ifconfig". It's a dedicated IDRAC interface >>> - Both nodes have a NIC configured in the subnet 192.168.199.0/24 >>> (192.168.199.200 and 192.168.199.201) >>> - Undercloud uses 192.168.199.0/24 as pxe/provisioning network (but I >>> don't need provisioning) >>> >>> Question: should I import nodes with "openstack overcloud node import >>> nodes.yaml" even if I don't need the provisioning step? >>> >>> Furthermore, on the undercloud I created one file: >>> /home/stack/templates/node-info.yaml with the following content >>> >>> parameter_defaults: >>> OvercloudControllerFlavor: control >>> OvercloudComputeFlavor: compute >>> ControllerCount: 1 >>> ComputeCount: 1 >>> >>> >> Question: How can I specify that "node X with ip Y should be used as a >>> controller and node Z with ip K should be used as a compute"?? >>> >> >> With pre-provisioned nodes (DeployedServer), you would need to specify >> HostnameMap and DeployedServerPortMap parameters that would map the >> pre-provisioned hosts and ctlplane ips. >> >> Please check documentation[1] for more details. >> >> [1] >> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deployed_server.html#deployed-server-with-config-download >> >> >>> Should I set the property with the following command? >>> openstack baremetal node set --property capabilities='profile:control' >>> controller1 >>> >>> Thank you, >>> Marco >>> >>> >> >> -- >> Regards, >> Rabi Mishra >> >> -- Regards, Rabi Mishra -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimo.sgaravatto at gmail.com Thu Aug 6 10:02:26 2020 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Thu, 6 Aug 2020 12:02:26 +0200 Subject: [ops] [keystone] "Roles are not immutable" Message-ID: After updating to Train from rocky (on stein we just performed the db-sync), we tried the new "keystone-status update check" command which says that the admin role is not immutable [*]. As far as I understand this is something that was done to prevent deleting/modifying the default roles (that could cause major problems). But how am I supposed to fix this? The "--immutable" option for the "openstack role set" command, documented at: https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/role-v3.html is not available in Train. Thanks, Massimo [*] +-------------------------------------------+ | Check: Check default roles are immutable | | Result: Failure | | Details: Roles are not immutable: admin | +-------------------------------------------+ -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Thu Aug 6 10:13:36 2020 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Thu, 6 Aug 2020 12:13:36 +0200 Subject: [ironic] [infra] bifrost-integration-tinyipa-opensuse-15 broken Message-ID: Hi folks, Our openSUSE CI job has been broken for a few days [1]. It fails on the early bindep stage with [2] File './suse/x86_64/libJudy1-1.0.5-lp151.2.2.x86_64.rpm' not found on medium ' https://mirror.mtl01.inap.opendev.org/opensuse/distribution/leap/15.1/repo/oss/&apos ; I've raised it on #openstack-infra, but I'm not sure if there has been any follow up. Help is appreciated Dmitry [1] https://zuul.openstack.org/builds?job_name=bifrost-integration-tinyipa-opensuse-15 [2] https://zuul.openstack.org/build/f4c7d174d171482394d1d0754c863ae1/console -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Aug 6 10:15:26 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 06 Aug 2020 11:15:26 +0100 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <046E9C0290DD9149B106B72FC9156BEA0481447D@gmsxchsvr01.thecreation.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> <046E9C0290DD9149B106B72FC9156BEA0481447D@gmsxchsvr01.thecreation.com> Message-ID: <5136cf19c506c5fa8b0293a0b5f4f15cb714ce3b.camel@redhat.com> On Wed, 2020-08-05 at 23:53 -0500, Eric K. Miller wrote: > > From: Sean Mooney [mailto:smooney at redhat.com] > > Sent: Wednesday, August 05, 2020 8:01 AM > > yes that works well with the default flat/qcow file format > > i assume there was a reason this was not the starting point. > > the nova lvm backend i think does not supprot thin provisioning > > so fi you did the same thing creating the volume group on the nvme deivce > > you would technically get better write performance after the vm is booted > > but > > the vm spwan is slower since we cant take advantage of thin providioning > > and > > each root disk need to be copided form the cahced image. > > I wasn't aware that the nova LVM backend ([libvirt]/images_type = lvm) didn't support thin provisioned LV's. However, > I do see that the "sparse_logical_volumes" parameter indicates it has been deprecated: > https://docs.openstack.org/nova/rocky/configuration/config.html#libvirt.sparse_logical_volumes > > That would definitely be a downer. > > > so just monting the nova data directory on an nvme driver or a raid of nvme > > drives > > works well and is simple to do. > > Maybe we should consider doing this instead. I'll test with the Nova LVM backend first. > > > so there are trade off with both appoches. > > generally i recommend using local sotrage e.g. the vm root disk or ephemeral > > disk for fast scratchpad space > > to work on data bug persitie all relevent data permently via cinder volumes. > > that requires you to understand which block > > devices a local and which are remote but it give you the best of both worlds. > > Our use case simply requires high-speed non-redundant storage for self-replicating applications like Couchbase, > Cassandra, MongoDB, etc. or very inexpensive VMs that are backed-up often and can withstand the downtime when > restoring from backup. > > That will be one more requirement (or rather a very nice to have), is to be able to create images (backups) of the > local storage onto object storage, so hopefully "openstack server backup create" works like it does with rbd-backed > Nova-managed persistent storage. it wil snapshot the root disk if you use addtional ephmeeral disks i do not think they are included but if you create the vms wit a singel root disk that is big enaough for your needs and use swift as your glance backend then yes. it will store the backups in object storage and rotate up to N backups per instance. > > I will let you know what I find out! > > Thanks everyone! > > Eric From emiller at genesishosting.com Thu Aug 6 10:26:59 2020 From: emiller at genesishosting.com (Eric K. Miller) Date: Thu, 6 Aug 2020 05:26:59 -0500 Subject: [cinder][nova] Local storage in compute node In-Reply-To: <5136cf19c506c5fa8b0293a0b5f4f15cb714ce3b.camel@redhat.com> References: <046E9C0290DD9149B106B72FC9156BEA04814477@gmsxchsvr01.thecreation.com> <046E9C0290DD9149B106B72FC9156BEA04814478@gmsxchsvr01.thecreation.com> <20200805111934.77lesgmmdiqeo27m@lyarwood.usersys.redhat.com> <7b7f6e277f77423ae6502d81c6d778fd4249b99d.camel@redhat.com> <92839697a08966dc17cd5c4c181bb32e2d197f93.camel@redhat.com> <4f025d444406898903dabf3049ed021822cce19b.camel@redhat.com> <046E9C0290DD9149B106B72FC9156BEA0481447D@gmsxchsvr01.thecreation.com> <5136cf19c506c5fa8b0293a0b5f4f15cb714ce3b.camel@redhat.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA04814483@gmsxchsvr01.thecreation.com> > it wil snapshot the root disk > if you use addtional ephmeeral disks i do not think they are included > but if you create the vms wit a singel root disk that is big enaough for your > needs and use swift as your glance backend > then yes. it will store the backups in object storage and rotate up to N > backups per instance. Thanks Sean! I tested a VM with a single root disk (no ephemeral disks) and it worked as expected (how you described). From e0ne at e0ne.info Thu Aug 6 12:03:53 2020 From: e0ne at e0ne.info (Ivan Kolodyazhny) Date: Thu, 6 Aug 2020 15:03:53 +0300 Subject: [horizon] Victoria virtual mid-cycle poll In-Reply-To: References: Message-ID: Here is Zoom connection details: Topic: Horizon Virtual Mid-Cycle Time: Aug 6, 2020 01:00 PM Universal Time UTC Join Zoom Meeting https://zoom.us/j/94173501669?pwd=c3JuNnpJMnBvNzgzdVJ5NDRhMnlhQT09 Meeting ID: 941 7350 1669 Passcode: 710495 One tap mobile +16468769923,,94173501669#,,,,,,0#,,710495# US (New York) +16699006833,,94173501669#,,,,,,0#,,710495# US (San Jose) Dial by your location +1 646 876 9923 US (New York) +1 669 900 6833 US (San Jose) +1 253 215 8782 US (Tacoma) +1 301 715 8592 US (Germantown) +1 312 626 6799 US (Chicago) +1 346 248 7799 US (Houston) +1 408 638 0968 US (San Jose) Meeting ID: 941 7350 1669 Passcode: 710495 Find your local number: https://zoom.us/u/ah3SiLk1q Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Thu, Aug 6, 2020 at 10:46 AM Ivan Kolodyazhny wrote: > Hi everybody, > > According to our poll [2] we'll have a one-hour mid-cycle poll today at > 13.00 UTC. I'll share a Zoom link before the meeting today. > > We're going to discuss current release priorities [3] and our future plans. > > > [2] https://doodle.com/poll/dkmsai49v4zzpca2 > [3] https://etherpad.opendev.org/p/horizon-release-priorities > > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > > > On Thu, Jul 30, 2020 at 1:00 PM Ivan Kolodyazhny wrote: > >> Hi team, >> >> If something can go wrong, it will definitely go wrong. >> It means that I did a mistake in my original mail and sent you >> completely wrong dates:(. >> >> Horizon Virtual mid-cycle is supposed to be next week Aug 5-7. I'm >> planning to have a single one-hour session. >> In case, if we've got a lot of participants and topic to discuss, we can >> schedule one more session a week or two weeks later. >> >> Here is a correct poll: https://doodle.com/poll/dkmsai49v4zzpca2 >> >> Regards, >> Ivan Kolodyazhny, >> http://blog.e0ne.info/ >> >> >> On Wed, Jul 22, 2020 at 10:26 AM Ivan Kolodyazhny wrote: >> >>> Hi team, >>> >>> As discussed at Horizon's Virtual PTG [1], we'll have a virtual >>> mid-cycle meeting around Victoria-2 milestone. >>> >>> We'll discuss Horizon current cycle development priorities and the >>> future of Horizon with modern JS frameworks. >>> >>> Please indicate your availability to meet for the first session, which >>> will be held during the week of July 27-31: >>> >>> https://doodle.com/poll/3neps94amcreaw8q >>> >>> Please respond before 12:00 UTC on Tuesday 4 August. >>> >>> [1] https://etherpad.opendev.org/p/horizon-v-ptg >>> >>> Regards, >>> Ivan Kolodyazhny, >>> http://blog.e0ne.info/ >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Thu Aug 6 14:04:21 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Thu, 6 Aug 2020 14:04:21 +0000 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <88c24f3a-7d29-aa39-ed12-803279cc90c1@openstack.org> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <88c24f3a-7d29-aa39-ed12-803279cc90c1@openstack.org> Message-ID: <20200806140421.GN31915@sync> Hey all, Thanks for your replies. About the fact that nova already implement this, I will try again on my side, but maybe it was not yet implemented in newton (I only tried nova on newton version). Thank you for bringing that to me. About the healhcheck already done on nova side (and also on neutron). As far as I understand, it's done using a specific rabbit queue, which can work while others queues are not working. The purpose of adding ping endpoint here is to be able to ping in all topics, not only those used for healthcheck reports. Also, as mentionned by Thierry, what we need is a way to externally do pings toward neutron agents and nova computes. The patch itself is not going to add any load on rabbit. It really depends on the way the operator will use it. On my side, I built a small external oslo.messaging script which I can use to do such pings. Cheers, -- Arnaud Morin On 03.08.20 - 12:15, Thierry Carrez wrote: > Ken Giusti wrote: > > On Mon, Jul 27, 2020 at 1:18 PM Dan Smith > > wrote: > > > The primary concern was about something other than nova sitting on our > > > bus making calls to our internal services. I imagine that the proposal > > > to bake it into oslo.messaging is for the same purpose, and I'd probably > > > have the same concern. At the time I think we agreed that if we were > > > going to support direct-to-service health checks, they should be teensy > > > HTTP servers with oslo healthchecks middleware. Further loading down > > > rabbit with those pings doesn't seem like the best plan to > > > me. Especially since Nova (compute) services already check in over RPC > > > periodically and the success of that is discoverable en masse through > > > the API. > > > > While initially in favor of this feature Dan's concern has me > > reconsidering this. > > > > Now I believe that if the purpose of this feature is to check the > > operational health of a service _using_ oslo.messaging, then I'm against > > it.   A naked ping to a generic service point in an application doesn't > > prove the operating health of that application beyond its connection to > > rabbit. > > While I understand the need to further avoid loading down Rabbit, I like the > universality of this solution, solving a real operational issue. > > Obviously that creates a trade-off (further loading rabbit to get more > operational insights), but nobody forces you to run those ping calls, they > would be opt-in. So the proposed code in itself does not weigh down Rabbit, > or make anything sit on the bus. > > > Connectivity monitoring between an application and rabbit is done using > > the keepalive connection heartbeat mechanism built into the rabbit > > protocol, which O.M. supports today. > > I'll let Arnaud answer, but I suspect the operational need is code-external > checking of the rabbit->agent chain, not code-internal checking of the > agent->rabbit chain. The heartbeat mechanism is used by the agent to keep > the Rabbit connection alive, ensuring it works in most of the cases. The > check described above is to catch the corner cases where it still doesn't. > > -- > Thierry Carrez (ttx) > From marino.mrc at gmail.com Thu Aug 6 14:06:08 2020 From: marino.mrc at gmail.com (Marco Marino) Date: Thu, 6 Aug 2020 16:06:08 +0200 Subject: [tripleo] Overcloud without provisioning error: os-net-config command not found Message-ID: Hi, I'm trying to deploy an overcloud using pre-provisioned nodes. I have only 2 nodes, 1 compute and 1 controller and here is the command I'm using: openstack overcloud deploy --templates --disable-validations -e /usr/share/openstack-tripleo-heat-templates/environments/deployed-server-environment.yaml -e /home/stack/templates/node-info.yaml -e /home/stack/templates/ctlplane-assignments.yaml -e /home/stack/templates/hostname-map.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -n /home/stack/os-deploy-custom-config/network_data.yaml --overcloud-ssh-user stack --overcloud-ssh-key /home/stack/.ssh/id_rsa Here is the content of custom files: (undercloud) [stack at undercloud ~]$ cat templates/node-info.yaml parameter_defaults: OvercloudControllerFlavor: control OvercloudComputeFlavor: compute ControllerCount: 1 ComputeCount: 1 (undercloud) [stack at undercloud ~]$ cat templates/ctlplane-assignments.yaml resource_registry: OS::TripleO::DeployedServer::ControlPlanePort: /usr/share/openstack-tripleo-heat-templates/deployed-server/deployed-neutron-port.yaml parameter_defaults: DeployedServerPortMap: controller-0-ctlplane: fixed_ips: - ip_address: 192.168.199.200 subnets: - cidr: 192.168.199.0/24 network: tags: 192.168.199.0/24 compute-0-ctlplane: fixed_ips: - ip_address: 192.168.199.210 subnets: - cidr: 192.168.199.0/24 network: tags: 192.168.199.0/24 (undercloud) [stack at undercloud ~]$ cat templates/hostname-map.yaml parameter_defaults: HostnameMap: overcloud-controller-0: controller-0 overcloud-novacompute-0: compute-0 http://paste.openstack.org/show/796634/ <-- Here is the complete output for overcloud deploy command. It seems that the error is /var/lib/tripleo-config/scripts/run_os_net_config.sh: line 59: os-net-config: command not found" os-net-config is provided by "delorean-component-tripleo" repository. So my question is: should I pre install Openstack repositories on pre-provisioned nodes in addition to operating system installation and network configuration? Thank you, Marco -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnaud.morin at gmail.com Thu Aug 6 14:11:32 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Thu, 6 Aug 2020 14:11:32 +0000 Subject: [largescale-sig] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> Message-ID: <20200806141132.GO31915@sync> Hi Mohammed, 1 - That's something we would also like, but it's beyond the patch I propose. I need this patch not only for kubernetes, but also for monitoring my legagy openstack agents running outside of k8s. 2 - Yes, latest version of rabbitmq is better on that point, but we still see some weird issue (I will ask the community about it in another topic). 3 - Thanks for this operator, we'll take a look! By saying 1 rabbit per service, I understand 1 server, not 1 cluster, right? That sounds risky if you lose the server. I suppose you dont do that for the database? 4 - Nice, how to you monitor those consumptions? Using rabbit management API? Cheers, -- Arnaud Morin On 03.08.20 - 10:21, Mohammed Naser wrote: > I have a few operational suggestions on how I think we could do this best: > > 1. I think exposing a healthcheck endpoint that _actually_ runs the > ping and responds with a 200 OK makes a lot more sense in terms of > being able to run it inside something like Kubernetes, you end up with > a "who makes the ping and who responds to it" type of scenario which > can be tricky though I'm sure we can figure that out > 2. I've found that newer releases of RabbitMQ really help with those > un-usable queues after a split, I haven't had any issues at all with > newer releases, so that could be something to help your life be a lot > easier. > 3. You mentioned you're moving towards Kubernetes, we're doing the > same and building an operator: > https://opendev.org/vexxhost/openstack-operator -- Because the > operator manages the whole thing and Kubernetes does it's thing too, > we started moving towards 1 (single) rabbitmq per service, which > reaaaaaaally helped a lot in stabilizing things. Oslo messaging is a > lot better at recovering when a single service IP is pointing towards > it because it doesn't do weird things like have threads trying to > connect to other Rabbit ports. Just a thought. > 4. In terms of telemetry and making sure you avoid that issue, we > track the consumption rates of queues inside OpenStack. OpenStack > consumption rate should be constant and never growing, anytime it > grows, we instantly detect that something is fishy. However, the > other issue comes in that when you restart any openstack service, it > 'forgets' all it's existing queues and then you have a set of building > up queues until they automatically expire which happens around 30 > minutes-ish, so it makes that alarm of "things are not being consumed" > a little noisy if you're restarting services > > Sorry for the wall of super unorganized text, all over the place here > but thought I'd chime in with my 2 cents :) > > On Mon, Jul 27, 2020 at 6:04 AM Arnaud Morin wrote: > > > > Hey all, > > > > TLDR: I propose a change to oslo_messaging to allow doing a ping over RPC, > > this is useful to monitor liveness of agents. > > > > > > Few weeks ago, I proposed a patch to oslo_messaging [1], which is adding a > > ping endpoint to RPC dispatcher. > > It means that every openstack service which is using oslo_messaging RPC > > endpoints (almosts all OpenStack services and agents - e.g. neutron > > server + agents, nova + computes, etc.) will then be able to answer to a > > specific "ping" call over RPC. > > > > I decided to propose this patch in my company mainly for 2 reasons: > > 1 - we are struggling monitoring our nova compute and neutron agents in a > > correct way: > > > > 1.1 - sometimes our agents are disconnected from RPC, but the python process > > is still running. > > 1.2 - sometimes the agent is still connected, but the queue / binding on > > rabbit cluster is not working anymore (after a rabbit split for > > example). This one is very hard to debug, because the agent is still > > reporting health correctly on neutron server, but it's not able to > > receive messages anymore. > > > > > > 2 - we are trying to monitor agents running in k8s pods: > > when running a python agent (neutron l3-agent for example) in a k8s pod, we > > wanted to find a way to monitor if it is still live of not. > > > > > > Adding a RPC ping endpoint could help us solve both these issues. > > Note that we still need an external mechanism (out of OpenStack) to do this > > ping. > > We also think it could be nice for other OpenStackers, and especially > > large scale ops. > > > > Feel free to comment. > > > > > > [1] https://review.opendev.org/#/c/735385/ > > > > > > -- > > Arnaud Morin > > > > > > > -- > Mohammed Naser > VEXXHOST, Inc. From sgolovat at redhat.com Thu Aug 6 14:16:56 2020 From: sgolovat at redhat.com (Sergii Golovatiuk) Date: Thu, 6 Aug 2020 16:16:56 +0200 Subject: [tripleo][ci] Make tripleo-ci-centos-8-containerized-undercloud-upgrades voting again Message-ID: Hi, tripleo-ci-centos-8-containerized-undercloud-upgrades has been improved significantly in terms of stability [1]. To improve CI coverage for upgrades I propose to make it voting. That will help to make upgrades more stable and catch bugs as early as possible. To keep it stable, Upgrade team is going to add it to their own triage process and dedicate the engineer to fix it if it's red for 2-3 days in a row. [1] https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-containerized-undercloud-upgrades&project=openstack/tripleo-common -- Sergii Golovatiuk Senior Software Developer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz at redhat.com Thu Aug 6 14:31:24 2020 From: aschultz at redhat.com (Alex Schultz) Date: Thu, 6 Aug 2020 08:31:24 -0600 Subject: [tripleo] Overcloud without provisioning error: os-net-config command not found In-Reply-To: References: Message-ID: On Thu, Aug 6, 2020 at 8:13 AM Marco Marino wrote: > > Hi, I'm trying to deploy an overcloud using pre-provisioned nodes. I have only 2 nodes, 1 compute and 1 controller and here is the command I'm using: > > > openstack overcloud deploy --templates --disable-validations -e /usr/share/openstack-tripleo-heat-templates/environments/deployed-server-environment.yaml -e /home/stack/templates/node-info.yaml -e /home/stack/templates/ctlplane-assignments.yaml -e /home/stack/templates/hostname-map.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -n /home/stack/os-deploy-custom-config/network_data.yaml --overcloud-ssh-user stack --overcloud-ssh-key /home/stack/.ssh/id_rsa > > Here is the content of custom files: > > (undercloud) [stack at undercloud ~]$ cat templates/node-info.yaml > parameter_defaults: > OvercloudControllerFlavor: control > OvercloudComputeFlavor: compute > ControllerCount: 1 > ComputeCount: 1 > > (undercloud) [stack at undercloud ~]$ cat templates/ctlplane-assignments.yaml > resource_registry: > OS::TripleO::DeployedServer::ControlPlanePort: /usr/share/openstack-tripleo-heat-templates/deployed-server/deployed-neutron-port.yaml > > parameter_defaults: > DeployedServerPortMap: > controller-0-ctlplane: > fixed_ips: > - ip_address: 192.168.199.200 > subnets: > - cidr: 192.168.199.0/24 > network: > tags: > 192.168.199.0/24 > compute-0-ctlplane: > fixed_ips: > - ip_address: 192.168.199.210 > subnets: > - cidr: 192.168.199.0/24 > network: > tags: > 192.168.199.0/24 > > > (undercloud) [stack at undercloud ~]$ cat templates/hostname-map.yaml > parameter_defaults: > HostnameMap: > overcloud-controller-0: controller-0 > overcloud-novacompute-0: compute-0 > > > http://paste.openstack.org/show/796634/ <-- Here is the complete output for overcloud deploy command. > It seems that the error is > /var/lib/tripleo-config/scripts/run_os_net_config.sh: line 59: os-net-config: command not found" > > os-net-config is provided by "delorean-component-tripleo" repository. So my question is: should I pre install Openstack repositories on pre-provisioned nodes in addition to operating system installation and network configuration? > Yes per the documentation: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deployed_server.html#package-repositories > Thank you, > Marco > > > > From arnaud.morin at gmail.com Thu Aug 6 14:40:16 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Thu, 6 Aug 2020 14:40:16 +0000 Subject: [nova][neutron][oslo][ops] rabbit bindings issue Message-ID: <20200806144016.GP31915@sync> Hey all, I would like to ask the community about a rabbit issue we have from time to time. In our current architecture, we have a cluster of rabbits (3 nodes) for all our OpenStack services (mostly nova and neutron). When one node of this cluster is down, the cluster continue working (we use pause_minority strategy). But, sometimes, the third server is not able to recover automatically and need a manual intervention. After this intervention, we restart the rabbitmq-server process, which is then able to join the cluster back. At this time, the cluster looks ok, everything is fine. BUT, nothing works. Neutron and nova agents are not able to report back to servers. They appear dead. Servers seems not being able to consume messages. The exchanges, queues, bindings seems good in rabbit. What we see is that removing bindings (using rabbitmqadmin delete binding or the web interface) and recreate them again (using the same routing key) brings the service back up and running. Doing this for all queues is really painful. Our next plan is to automate it, but is there anyone in the community already saw this kind of issues? Our bug looks like the one described in [1]. Someone recommands to create an Alternate Exchange. Is there anyone already tried that? FYI, we are running rabbit 3.8.2 (with OpenStack Stein). We had the same kind of issues using older version of rabbit. Thanks for your help. [1] https://groups.google.com/forum/#!newtopic/rabbitmq-users/rabbitmq-users/zFhmpHF2aWk -- Arnaud Morin From bdobreli at redhat.com Thu Aug 6 15:02:34 2020 From: bdobreli at redhat.com (Bogdan Dobrelya) Date: Thu, 6 Aug 2020 17:02:34 +0200 Subject: [tripleo][ci] Make tripleo-ci-centos-8-containerized-undercloud-upgrades voting again In-Reply-To: References: Message-ID: +1 On 8/6/20 4:16 PM, Sergii Golovatiuk wrote: > Hi, > > tripleo-ci-centos-8-containerized-undercloud-upgrades has been improved > significantly in terms of stability [1]. To improve CI coverage for > upgrades I propose to make it voting. That will help to make upgrades > more stable and catch bugs as early as possible. To keep it stable, > Upgrade team is going to add it to their own triage process and > dedicate the engineer to fix it if it's red for 2-3 days in a row. > > [1] > https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-containerized-undercloud-upgrades&project=openstack/tripleo-common > > -- > SergiiGolovatiuk > > Senior Software Developer > > Red Hat > > > -- Best regards, Bogdan Dobrelya, Irc #bogdando From marios at redhat.com Thu Aug 6 15:51:07 2020 From: marios at redhat.com (Marios Andreou) Date: Thu, 6 Aug 2020 18:51:07 +0300 Subject: [tripleo][ci] Make tripleo-ci-centos-8-containerized-undercloud-upgrades voting again In-Reply-To: References: Message-ID: On Thu, Aug 6, 2020 at 5:19 PM Sergii Golovatiuk wrote: > Hi, > > tripleo-ci-centos-8-containerized-undercloud-upgrades has been improved > significantly in terms of stability [1]. To improve CI coverage for > upgrades I propose to make it voting. That will help to make upgrades more > stable and catch bugs as early as possible. To keep it stable, Upgrade team > is going to add it to their own triage process and dedicate the engineer to > fix it if it's red for 2-3 days in a row. > > o/ as discussed on irc, IMO we should make it voting "until we can't". Your triage process about 2/3 days sounds reasonable but it will be seen in practice how well that works. Which is the original reason there is push-back against master voting upgrades jobs - i.e. whilst developing for the cycle the upgrades jobs might break until all new features are merged and you can accommodate for them in the upgrade. So they break and this blocks master gates. They were made non voting during a time that they were broken often and for long periods. let's make them voting "until we can't". > [1] > https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-containerized-undercloud-upgrades&project=openstack/tripleo-common > > -- > Sergii Golovatiuk > > Senior Software Developer > > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zaitcev at redhat.com Thu Aug 6 16:41:25 2020 From: zaitcev at redhat.com (Pete Zaitcev) Date: Thu, 6 Aug 2020 11:41:25 -0500 Subject: [TripleO] "bundle install" in puppet-tripleo Message-ID: <20200806114125.0f0961a9@suzdal.zaitcev.lan> Hello: Due to some circumstances, I started looking at running unit tests in puppet-tripleo. The official document[1] tells me to start by running "bundle install". This results in: [zaitcev at suzdal puppet-tripleo-c744015]$ bundle install Fetching https://git.openstack.org/openstack/puppet-openstack_spec_helper Fetching gem metadata from https://rubygems.org/........ Resolving dependencies........ Fetching rake 13.0.1 ....... Installing netaddr 1.5.1 Using pathspec 0.2.1 <--------------------- in black, not green Fetching pry 0.12.2 ....... Installing webmock 3.8.3 Using puppet-openstack_spec_helper 17.0.0 from https://git.openstack.org/openstack/puppet-openstack_spec_helper (at master at 273d24f) Updating files in vendor/cache Could not find pathspec-0.2.1.gem for installation [zaitcev at suzdal puppet-tripleo-c744015]$ Anyone got an idea what the above means? -- Pete [1] https://docs.openstack.org/puppet-openstack-guide/latest/contributor/testing.html From caifti at gmail.com Thu Aug 6 07:00:44 2020 From: caifti at gmail.com (Doina Cristina Duma) Date: Thu, 6 Aug 2020 09:00:44 +0200 Subject: [TC] [PTG] Victoria vPTG Summary of Conversations and Action Items In-Reply-To: References: Message-ID: Hello everyone, On Tue, Aug 4, 2020 at 2:14 PM Belmiro Moreira < moreira.belmiro.email.lists at gmail.com> wrote: > Hi everyone, > the problem described in the "OpenStack User-facing APIs" is something > that we face daily in our deployment. Different CLIs for different > operations. > same for us, really frustrating, going around and see what is missing (what options) > I'm really interested in driving this action item. > I totally support your proposal! Cristina > > Belmiro > > On Fri, Jun 12, 2020 at 9:38 PM Kendall Nelson > wrote: > >> Hello Everyone! >> >> I hope you all had a productive and enjoyable PTG! While it’s still >> reasonably fresh, I wanted to take a moment to summarize discussions and >> actions that came out of TC discussions. >> >> If there is a particular action item you are interested in taking, please >> reply on this thread! >> >> For the long version, check out the etherpad from the PTG[1]. >> >> Tuesday >> >> ====== >> >> Ussuri Retrospective >> >> ---------------------------- >> >> As usual we accomplished a lot. Some of the things we accomplished were >> around enumerating operating systems per release (again), removing python2 >> support, and adding the ideas repository. Towards the end of the release, >> we had a lot of discussions around what to do with leaderless projects, the >> role of PTLs, and what to do with projects that were missing PTL candidates >> for the next release. We discussed office hours, their history and reason >> for existence, and clarified how we can strengthen communication amongst >> ourselves, the projects, and the larger community. >> >> TC Onboarding >> >> -------------------- >> >> It was brought up that those elected most recently (and even new members >> the election before) felt like there wasn’t enough onboarding into the TC. >> Through discussion about what we can do to better support returning members >> is to better document the daily, weekly and monthly tasks TC members are >> supposed to be doing. Kendall Nelson proposed a patch to start adding more >> detail to a guide for TC members already[2]. It was also proposed that we >> have a sort of mentorship or shadow program for people interested in >> joining the TC or new TC members by more experienced TC members. The >> discussion about the shadow/mentorship program is to be continued. >> >> TC/UC Merge >> >> ------------------ >> >> Thierry gave an update on the merge of the committees. The simplified >> version is that the current proposal is that UC members are picked from TC >> members, the UC operates within the TC, and that we are already setup for >> this given the number of TC members that have AUC status. None of this >> requires a by-laws change. One next step that has already begun is the >> merging of the openstack-users ML into openstack-discuss ML. Other next >> steps are to decide when to do the actual transition (disbanding the >> separate UC, probably at the next election?) and when to setup AUC’s to be >> defined as extra-ATC’s to be included in the electorate for elections. For >> more detail, check out the openstack-discuss ML thread[3]. >> >> Wednesday >> >> ========= >> >> Help Wanted List >> >> ----------------------- >> >> We settled on a format for the job postings and have several on the list. >> We talked about how often we want to look through, update or add to it. The >> proposal is to do this yearly. We need to continue pushing on the board to >> dedicate contributors at their companies to work on these items, and get >> them to understand that it's an investment that will take longer than a >> year in a lot of cases; interns are great, but not enough. >> >> TC Position on Foundation Member Community Contributions >> >> >> ---------------------------------------------------------------------------------- >> >> The discussion started with a state of things today - the expectations of >> platinum members, the benefits the members get being on the board and why >> they should donate contributor resources for these benefits, etc. A variety >> of proposals were made: either enforce or remove the minimum contribution >> level, give gold members the chance to have increased visibility (perhaps >> giving them some of the platinum member advantages) if they supplement >> their monetary contributions with contributor contributions, etc. The >> #ACTION that was decided was for Mohammed to take these ideas to the board >> and see what they think. >> >> OpenStack User-facing APIs >> >> -------------------------------------- >> >> Users are confused about the state of the user facing API’s; they’ve been >> told to use the OpenStackClient(OSC) but upon use, they discover that there >> are features missing that exist in the python-*clients. Partial >> implementation in the OSC is worse than if the service only used their >> specific CLI. Members of the OpenStackSDK joined discussions and explained >> that many of the barriers that projects used to have behind implementing >> certain commands have been resolved. The proposal is to create a pop up >> team and that they start with fully migrating Nova, documenting the process >> and collecting any other unresolved blocking issues with the hope that one >> day we can set the migration of the remaining projects as a community goal. >> Supplementally, a new idea was proposed- enforcing new functionality to >> services is only added to the SDK (and optionally the OSC) and not the >> project’s specific CLI to stop increasing the disparity between the two. >> The #ACTION here is to start the pop up team, if you are interested, please >> reply! Additionally, if you disagree with this kind of enforcement, please >> contact the TC as soon as possible and explain your concerns. >> >> PTL Role in OpenStack today & Leaderless Projects >> >> --------------------------------------------------------------------- >> >> This was a veeeeeeeerrrry long conversation that went in circles a few >> times. The very short version is that we, the TC, are willing to let >> project teams decide for themselves if they want to have a more >> deconstructed kind of PTL role by breaking it into someone responsible for >> releases and someone responsible for security issues. This new format also >> comes with setting the expectation that for things like project updates and >> signing up for PTG time, if someone on the team doesn’t actively take that >> on, the default assumption is that the project won’t participate. The >> #ACTION we need someone to take on is to write a resolution about how this >> will work and how it can be done. Ideally, this would be done before the >> next technical election, so that teams can choose it at that point. If you >> are interested in taking on the writing of this resolution, please speak up! >> >> Cross Project Work >> >> ------------------------- >> >> -Pop Up Teams- >> >> The two teams we have right now are Encryption and Secure Consistent >> Policy Groups. Both are making slow progress and will continue. >> >> >> >> -Reducing Community Goals Per Cycle- >> >> Historically we have had two goals per cycle, but for smaller teams this >> can be a HUGE lift. The #ACTION is to clearly outline the documentation for >> the goal proposal and selection process to clarify that selecting only one >> goal is fine. No one has claimed this action item yet. >> >> -Victoria Goal Finalization- >> >> Currently, we have three proposals and one accepted goal. If we are going >> to select a second goal, it needs to be done ASAP as Victoria development >> has already begun. All TC members should review the last proposal >> requesting selection[4]. >> >> -Wallaby Cycle Goal Discussion Kick Off- >> >> Firstly, there is a #ACTION that one or two TC members are needed to >> guide the W goal selection. If you are interested, please reply to this >> thread! There were a few proposed goals for VIctoria that didn’t make it >> that could be the starting point for W discussions, in particular, the >> rootwrap goal which would be good for operators. The OpenStackCLI might be >> another goal to propose for Wallaby. >> >> Detecting Unmaintained Projects Early >> >> --------------------------------------------------- >> >> The TC liaisons program had been created a few releases ago, but the >> initial load on TC members was large. We discussed bringing this program >> back and making the project health checks happen twice a release, either >> the start or end of the release and once in the middle. TC liaisons will >> look at previously proposed releases, release activity of the team, the >> state of tempest plugins, if regular meetings are happening, if there are >> patches in progress and how busy the project’s IRC channel is to make a >> determination. Since more than one liaison will be assigned to each >> project, those liaisons can divvy up the work how they see fit. The other >> aspect that still needs to be decided is where the health checks will be >> recorded- in a wiki? In a meeting and meeting logs? That decision is still >> to be continued. The current #ACTION currently unassigned is that we need >> to assign liaisons for the Victoria cycle and decide when to do the first >> health check. >> >> Friday >> >> ===== >> >> Reducing Systems and Friction to Drive Change >> >> ---------------------------------------------------------------- >> >> This was another conversation that went in circles a bit before realizing >> that we should make a list of the more specific problems we want to address >> and then brainstorm solutions for them. The list we created (including >> things already being worked) are as follows: >> >> - >> >> TC separate from UC (solution in progress) >> - >> >> Stable releases being approved by a separate team (solution in >> progress) >> - >> >> Making repository creation faster (especially for established project >> teams) >> - >> >> Create a process blueprint for project team mergers >> - >> >> Requirements Team being one person >> - >> >> Stable Team >> - >> >> Consolidate the agent experience >> - >> >> Figure out how to improve project <--> openstack client/sdk >> interaction. >> >> If you feel compelled to pick one of these things up and start proposing >> solutions or add to the list, please do! >> >> Monitoring in OpenStack (Ceilometer + Telemetry + Gnocchi State) >> >> >> ----------------------------------------------------------------------------------------- >> >> This conversation is also ongoing, but essentially we talked about the >> state of things right now- largely they are not well maintained and there >> is added complexity with Ceilometers being partially dependent on Gnocchi. >> There are a couple of ideas to look into like using oslo.metrics for the >> interface between all the tools or using Ceilometer without Gnocchi if we >> can clean up those dependencies. No specific action items here, just please >> share your thoughts if you have them. >> >> Ideas Repo Next Steps >> >> ------------------------------- >> >> Out of the Ussuri retrospective, it was brought up that we probably >> needed to talk a little more about what we wanted for this repo. >> Essentially we just want it to be a place to collect ideas into without >> worrying about the how. It should be a place to document ideas we have had >> (old and new) and keep all the discussion in one place as opposed to >> historic email threads, meetings logs, other IRC logs, etc. We decided it >> would be good to periodically go through this repo, likely as a forum >> session at a summit to see if there is any updating that could happen or >> promotion of ideas to community goals, etc. >> >> ‘tc:approved-release’ Tag >> >> --------------------------------- >> >> This topic was proposed by the Manila team from a discussion they had >> earlier in the week. We talked about the history of the tag and how usage >> of tags has evolved. At this point, the proposal is to remove the tag as >> anything in the releases repo is essentially tc-approved. Ghanshyam has >> volunteered to document this and do the removal. The board also needs to be >> notified of this and to look at projects.yaml in the governance repo as the >> source of truth for TC approved projects. The unassigned #ACTION item is to >> review remaining tags and see if there are others that need to be >> modified/removed/added to drive common behavior across OpenSack >> components. >> >> Board Proposals >> >> ---------------------- >> >> This was a pretty quick summary of all discussions we had that had any >> impact on the board and largely decided who would mention them. >> >> >> >> Session Feedback >> >> ------------------------ >> >> This was also a pretty quick topic compared to many of the others, we >> talked about how things went across all our discussions (largely we called >> the PTG a success) logistically. We tried to make good use of the raising >> hands feature which mostly worked, but it lacks context and its possible >> that the conversation has moved on by the time it’s your turn (if you even >> remember what you want to say). >> >> OpenStack 2.0: k8s Native >> >> ----------------------------------- >> >> This topic was brought up at the end of our time so we didn’t have time >> to discuss it really. Basically Mohammed wanted to start the conversation >> about adding k8s as a base service[5] and what we would do if a project >> proposed required k8s. Adding services that work with k8s could open a door >> to new innovation in OpenStack. Obviously this topic will need to be >> discussed further as we barely got started before we had to wrap things up. >> >> >> So. >> >> >> The tldr; >> >> >> Here are the #ACTION items we need owners for: >> >> - >> >> Start the User Facing API Pop Up Team >> - >> >> Write a resolution about how the deconstructed PTL roles will work >> - >> >> Update Goal Selection docs to explain that one or more goals is fine; >> it doesn’t have to be more than one >> - >> >> Two volunteers to start the W goal selection process >> - >> >> Assign two TC liaisons per project >> - >> >> Review Tags to make sure they are still good for driving common >> behavior across all openstack projects >> >> >> Here are the things EVERYONE needs to do: >> >> - >> >> Review last goal proposal so that we can decide to accept or reject >> it for the V release[4] >> - >> >> Add systems that are barriers to progress in openstack to the >> Reducing Systems and Friction list >> - >> >> Continue conversations you find important >> >> >> >> Thanks everyone for your hard work and great conversations :) >> >> Enjoy the attached (photoshopped) team photo :) >> >> -Kendall (diablo_rojo) >> >> >> >> [1] TC PTG Etherpad: https://etherpad.opendev.org/p/tc-victoria-ptg >> >> [2] TC Guide Patch: https://review.opendev.org/#/c/732983/ >> >> [3] UC TC Merge Thread: >> http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014736.html >> >> >> [4] Proposed V Goal: https://review.opendev.org/#/c/731213/ >> >> [5] Base Service Description: >> https://governance.openstack.org/tc/reference/base-services.html >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From monika.samal at outlook.com Thu Aug 6 13:38:30 2020 From: monika.samal at outlook.com (Monika Samal) Date: Thu, 6 Aug 2020 13:38:30 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , Message-ID: Thanks for responding ? ________________________________ From: Mark Goddard Sent: Thursday, August 6, 2020 1:16 PM To: Michael Johnson Cc: Monika Samal ; Fabian Zimmermann ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer On Wed, 5 Aug 2020 at 16:16, Michael Johnson > wrote: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard > wrote: On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson > Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann > Cc: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From lijie at unitedstack.com Thu Aug 6 16:59:52 2020 From: lijie at unitedstack.com (=?utf-8?B?UmFtYm8=?=) Date: Fri, 7 Aug 2020 00:59:52 +0800 Subject: [cinder] Could you help me review the reimage feature? Message-ID: Hi,all:         I have a spec which is support volume backed server rebuild[0].This spec was accepted in Stein, but some of the work did not finish, so repropose it for Victoria.  I sincerely wish this spec will approved in Victoria, so I make an exception for this, and the Nova team will approved this if the cinder reimage question is solved this week[1].  This spec is depend on the cinder reimage api [2], and the reimage api has a question. We just need to know if cinder are ok with the change in polling to event like the volume extend. More clearly, Cinder reimage should add a new 'volume-reimage' external event like the volume extend, so that nova can wait for cinder to complete the reimage[3].        The Cinder code is[4], if you have some ideas, you can comments on it.Thank you very much! Ref: [0]:https://blueprints.launchpad.net/nova/+spec/volume-backed-server-rebuild [1]:http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-3/%23openstack-meeting-3.2020-08-06.log.html#t2020-08-06T16:18:22-2 [2]:https://blueprints.launchpad.net/cinder/+spec/add-volume-re-image-api [3]:https://review.opendev.org/#/c/454287/ [4]:https://review.opendev.org/#/c/606346/ Best Regards Rambo -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Thu Aug 6 17:08:29 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 6 Aug 2020 17:08:29 +0000 Subject: [ironic] [infra] bifrost-integration-tinyipa-opensuse-15 broken In-Reply-To: References: Message-ID: <20200806170829.rcotrtmneyeyktbn@yuggoth.org> On 2020-08-06 12:13:36 +0200 (+0200), Dmitry Tantsur wrote: > Our openSUSE CI job has been broken for a few days [1]. It fails on the > early bindep stage with [2] > > percent="-1" rate="-1"/> > rate="-1" done="0"/> > File > './suse/x86_64/libJudy1-1.0.5-lp151.2.2.x86_64.rpm' not found on > medium ' > https://mirror.mtl01.inap.opendev.org/opensuse/distribution/leap/15.1/repo/oss/&apos > ; > > I've raised it on #openstack-infra, but I'm not sure if there has been any > follow up. [...] Yes, we discussed it at some length immediately after you mentioned it: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2020-08-05.log.html#t2020-08-05T15:43:03 In short, the packages are in /opensuse/distribution/leap/15.1/repo/oss/x86_64/ not /opensuse/distribution/leap/15.1/repo/oss/suse/x86_64/ and the INDEX.gz files seem to point to the correct location for them. It's not clear to us why zypper is looking in the latter path; help from someone with more familiarity with openSUSE and zypper would be much appreciated. Our mirrors match the official mirrors in this regard, and our base jobs configure only the first part of the repository path: It's not clear to any of us what's adding the "/suse" to the URLs zypper is requesting. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From rosmaita.fossdev at gmail.com Thu Aug 6 21:00:28 2020 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Thu, 6 Aug 2020 17:00:28 -0400 Subject: [cinder] Could you help me review the reimage feature? In-Reply-To: References: Message-ID: <36500a46-7fcc-4fa0-fc09-d7235a833c9f@gmail.com> On 8/6/20 12:59 PM, Rambo wrote: > Hi,all: >         I have a spec which is support volume backed server > rebuild[0].This spec was accepted in Stein, but some of the work did not > finish, so repropose it for Victoria.  I sincerely wish this spec will > approved in Victoria, so I make an exception for this, and the Nova team > will approved this *if the cinder reimage question is solved this > week*[1].  This spec is depend on the cinder reimage api [2], and the > reimage api has a question. We just need to know if cinder are ok with > the change in polling to event like the volume extend. More clearly, > Cinder reimage should add a new 'volume-reimage' external event like the > volume extend, so that nova can wait for cinder to complete the reimage[3]. >        The Cinder code is[4], if you have some ideas, you can comments > on it.Thank you very much! The Cinder team is not going to approve this proposal this week, but we encourage you to continue working on it for Wallaby. The spec was approved for Stein and then re-targeted to Train. Until July 30, the last activity on the patch was April 1, 2019, so this has not been on the Cinder team's radar at all this development cycle. Because the spec is outdated, it should be proposed for Wallaby so the current Cinder team can review it and assess how it fits into the current project plans. I've already penciled you in for next week's midcycle so we can discuss this in more depth. But I am against making a snap decision in the next two days. cheers, brian > > > Ref: > > [0]:https://blueprints.launchpad.net/nova/+spec/volume-backed-server-rebuild > > [1]:http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-3/%23openstack-meeting-3.2020-08-06.log.html#t2020-08-06T16:18:22-2 > > [2]:https://blueprints.launchpad.net/cinder/+spec/add-volume-re-image-api > [3]:https://review.opendev.org/#/c/454287/ > [4]:https://review.opendev.org/#/c/606346/ > Best Regards > Rambo From cboylan at sapwetik.org Thu Aug 6 21:44:54 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 06 Aug 2020 14:44:54 -0700 Subject: [ironic] [infra] bifrost-integration-tinyipa-opensuse-15 broken In-Reply-To: <20200806170829.rcotrtmneyeyktbn@yuggoth.org> References: <20200806170829.rcotrtmneyeyktbn@yuggoth.org> Message-ID: <4e21ac6a-e287-4d3b-b0e6-c581c631e992@www.fastmail.com> On Thu, Aug 6, 2020, at 10:08 AM, Jeremy Stanley wrote: > On 2020-08-06 12:13:36 +0200 (+0200), Dmitry Tantsur wrote: > > Our openSUSE CI job has been broken for a few days [1]. It fails on the > > early bindep stage with [2] > > > > > percent="-1" rate="-1"/> > > > rate="-1" done="0"/> > > File > > './suse/x86_64/libJudy1-1.0.5-lp151.2.2.x86_64.rpm' not found on > > medium ' > > https://mirror.mtl01.inap.opendev.org/opensuse/distribution/leap/15.1/repo/oss/&apos > > ; > > > > I've raised it on #openstack-infra, but I'm not sure if there has been any > > follow up. > [...] > > Yes, we discussed it at some length immediately after you mentioned > it: > > http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2020-08-05.log.html#t2020-08-05T15:43:03 > > In short, the packages are in > /opensuse/distribution/leap/15.1/repo/oss/x86_64/ not > /opensuse/distribution/leap/15.1/repo/oss/suse/x86_64/ and the > INDEX.gz files seem to point to the correct location for them. It's > not clear to us why zypper is looking in the latter path; help from > someone with more familiarity with openSUSE and zypper would be much > appreciated. Our mirrors match the official mirrors in this regard, > and our base jobs configure only the first part of the repository > path: > > https://opendev.org/zuul/zuul-jobs/src/commit/1ba95015acc977dea8269889235434d052c736e2/roles/configure-mirrors/tasks/mirror/Suse.yaml#L3 > > > It's not clear to any of us what's adding the "/suse" to the URLs > zypper is requesting. https://review.opendev.org/745225 has landed and seems to fix this issue. Our hunch is that the type of the repo changed upstream of us which we then mirrored. Once this happened our repo configs were no longer correct. Zypper man pages and docs say repos should have their type auto-detected anyway so we've dropped the type specification entirely. This fixed things in testing. If anyone understands this better that info would be appreciated, but I expect the ironic jobs to be happier now too. Clark From kennelson11 at gmail.com Fri Aug 7 00:00:13 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 6 Aug 2020 17:00:13 -0700 Subject: [TC] New Office Hour Plans Message-ID: Hello! After taking a look at the poll results, Mohammed and I have two proposed plans for office hours: Plan A: Two office hours instead of three. This gives us slightly more coverage than one office hour without overextending ourselves to cover three office hours. Mohammed and I were thinking that one of the reasons why three office hours wasn't working was that it was kind of a big time commitment and TC members could easily rationalize not going to ones later in the week if they had already attended one earlier in the week. The two times that enable most TC members to attend at least one, if not both, would be Monday @14:00 UTC (TC members available: Belmiro, Rico, Kristi, Jay, Mohammed, myself, and Nate + non members) and Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). Plan B: Move to a single office hour on Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). Having only one office hour gives it more weight and importance and that should hopefully encourage more attendance from both community members and TC members alike. I guess Plan C is to go ahead with Plan A and then if we don't see activity during the Monday time slot, to reduce down to one office hour and go with Plan B. Please check out the patches Mohammed posted [1][2] and vote on what you'd prefer! -Kendall (diablo_rojo) [1] Dual Office Hour: https://review.opendev.org/#/c/745201/ [2] Single Office Hour: https://review.opendev.org/#/c/745200/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From monika.samal at outlook.com Thu Aug 6 23:11:52 2020 From: monika.samal at outlook.com (Monika Samal) Date: Thu, 6 Aug 2020 23:11:52 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , Message-ID: I tried following above document still facing same Octavia connection error with amphora image. Regards, Monika ________________________________ From: Mark Goddard Sent: Thursday, August 6, 2020 1:16:01 PM To: Michael Johnson Cc: Monika Samal ; Fabian Zimmermann ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer On Wed, 5 Aug 2020 at 16:16, Michael Johnson > wrote: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard > wrote: On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson > Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann > Cc: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From berndbausch at gmail.com Fri Aug 7 00:29:11 2020 From: berndbausch at gmail.com (Bernd Bausch) Date: Fri, 7 Aug 2020 09:29:11 +0900 Subject: [openstack-community] [infra] Problem with ask.openstack.org Message-ID: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> While ask.openstack.org is not necessarily loved by many, it continues to be used, and there are still people who answer questions. Recently, one of its features ceased working. I am talking about the "responses" page that lists all responses to questions that I have answered or commented on. This makes it very hard to follow up on such questions; I don't have a tool to see if somebody anwered my question or is the person who asked a question has provided updates. Is there anybody who can fix this? I know that some people would like to do away with ask.openstack.org entirely, since the software is bug-ridden and nobody manages the site. My personal opinion is that the current situation is worse than no "ask" site at all, since people might ask questions, get partial answers and no follow-up. This can create a negative view of the OpenStack community. In short, either fix it or remove it. Unfortunately I don't have the means to do either. Bernd. From laurentfdumont at gmail.com Fri Aug 7 00:36:26 2020 From: laurentfdumont at gmail.com (Laurent Dumont) Date: Thu, 6 Aug 2020 20:36:26 -0400 Subject: [openstack-community] [infra] Problem with ask.openstack.org In-Reply-To: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> References: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> Message-ID: It's definitely a tough sell. I'm not sure if it's worse to not have a community driven "a-la-Stackoverflow" style or one that is not super in shape. I would rather see go into archive mode only if it's too much of an Operational burden to keep running :( On Thu, Aug 6, 2020 at 8:33 PM Bernd Bausch wrote: > While ask.openstack.org is not necessarily loved by many, it continues > to be used, and there are still people who answer questions. > > Recently, one of its features ceased working. I am talking about the > "responses" page that lists all responses to questions that I have > answered or commented on. This makes it very hard to follow up on such > questions; I don't have a tool to see if somebody anwered my question or > is the person who asked a question has provided updates. > > Is there anybody who can fix this? > > I know that some people would like to do away with ask.openstack.org > entirely, since the software is bug-ridden and nobody manages the site. > My personal opinion is that the current situation is worse than no "ask" > site at all, since people might ask questions, get partial answers and > no follow-up. This can create a negative view of the OpenStack community. > > In short, either fix it or remove it. Unfortunately I don't have the > means to do either. > > Bernd. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Fri Aug 7 00:41:36 2020 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 06 Aug 2020 17:41:36 -0700 Subject: [openstack-community] [infra] Problem with ask.openstack.org In-Reply-To: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> References: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> Message-ID: <9b6967b9-5cb7-47f2-bacd-87d3304a3428@www.fastmail.com> On Thu, Aug 6, 2020, at 5:29 PM, Bernd Bausch wrote: > While ask.openstack.org is not necessarily loved by many, it continues > to be used, and there are still people who answer questions. > > Recently, one of its features ceased working. I am talking about the > "responses" page that lists all responses to questions that I have > answered or commented on. This makes it very hard to follow up on such > questions; I don't have a tool to see if somebody anwered my question or > is the person who asked a question has provided updates. > > Is there anybody who can fix this? > > I know that some people would like to do away with ask.openstack.org > entirely, since the software is bug-ridden and nobody manages the site. > My personal opinion is that the current situation is worse than no "ask" > site at all, since people might ask questions, get partial answers and > no follow-up. This can create a negative view of the OpenStack community. > > In short, either fix it or remove it. Unfortunately I don't have the > means to do either. I'm not able to debug the issue at this moment, but did want to point out that all of our config management is collaboratively managed in Git repos code reviewed in Gerrit. This means that if you know what the problem is you absolutely can fix it. Or if you'd prefer to turn off the service you can write a change for that as well. The biggest gap is in identifying the issue without access to server logs. Depending on the issue figuring out what is going on may require access. Relevant bits of code: https://opendev.org/opendev/system-config/src/branch/master/manifests/site.pp#L525-L538 https://opendev.org/opendev/system-config/src/branch/master/modules/openstack_project/manifests/ask.pp https://opendev.org/opendev/puppet-askbot Finally, we also expose server and service statistics via cacti and graphite. These can be useful for checking service health: http://cacti.openstack.org/cacti/graph_view.php https://grafana.opendev.org/?orgId=1 Clark From yasufum.o at gmail.com Fri Aug 7 08:37:16 2020 From: yasufum.o at gmail.com (Yasufumi Ogawa) Date: Fri, 7 Aug 2020 17:37:16 +0900 Subject: [tacker] PTL on vacation Message-ID: <9f29f573-69f5-d6d2-20d9-5c5ef7d775f4@gmail.com> I will be on vacation from 10th to 17th Aug. I would like to skip the next IRC meeting because many of tacker members are also on vacation next week. Thanks, Yasufumi From berndbausch at gmail.com Fri Aug 7 09:09:31 2020 From: berndbausch at gmail.com (Bernd Bausch) Date: Fri, 7 Aug 2020 18:09:31 +0900 Subject: [openstack-community] [infra] Problem with ask.openstack.org In-Reply-To: References: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> Message-ID: <21ed671d-63d2-53e1-1ce0-31b977515be6@gmail.com> Sending people to Stackoverflow directly is a good option IMO. This suggestion was made before. Of course, I would lose my 7700 karma points, but I can stomach it :) On 8/7/2020 9:36 AM, Laurent Dumont wrote: > It's definitely a tough sell. I'm not sure if it's worse to not have a > community driven "a-la-Stackoverflow" style or one that is not super > in shape. > > I would rather see go into archive mode only if it's too much of an > Operational burden to keep running :( > > On Thu, Aug 6, 2020 at 8:33 PM Bernd Bausch > wrote: > > While ask.openstack.org is not > necessarily loved by many, it continues > to be used, and there are still people who answer questions. > > Recently, one of its features ceased working. I am talking about the > "responses" page that lists all responses to questions that I have > answered or commented on. This makes it very hard to follow up on > such > questions; I don't have a tool to see if somebody anwered my > question or > is the person who asked a question has provided updates. > > Is there anybody who can fix this? > > I know that some people would like to do away with > ask.openstack.org > entirely, since the software is bug-ridden and nobody manages the > site. > My personal opinion is that the current situation is worse than no > "ask" > site at all, since people might ask questions, get partial answers > and > no follow-up. This can create a negative view of the OpenStack > community. > > In short, either fix it or remove it. Unfortunately I don't have the > means to do either. > > Bernd. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Fri Aug 7 09:18:48 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Fri, 7 Aug 2020 11:18:48 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: Thanks a lot for starting this discussion, I am also quite concerned about this. At StackHPC we started looking into CloudKitty a year ago, when the community was still fairly active. There was an IRC meeting every month or so throughout 2019. Patches were getting merged. Unfortunately in 2020 activity stopped abruptly. There hasn't been any IRC meeting since early December and no patch has been merged since the end of March. I have submitted straightforward stable backports of bug fixes which have not received any answer. I am well aware of the difficulty of keeping up with open-source project maintenance when work deadlines are always taking priority. If the existing core team would be willing to grant +2 votes to more people, I would be happy to participate in the maintenance of the project. We've now deployed CloudKitty for several of our customers and have to maintain a stable fork anyway. We would rather maintain upstream directly! Pierre Riteau (priteau) On Tue, 4 Aug 2020 at 23:22, Kendall Nelson wrote: > > I think the majority of 'maintenance' activities at the moment for Cloudkitty are the reviewing of open patches in gerrit [1] and triaging bugs that are reported in Launchpad[2] as they come in. When things come up on this mailing list that have the cloudkitty tag in the subject line (like this email), weighing in on them would also be helpful. > > If you need help getting setup with gerrit, I am happy to assist anyway I can :) > > -Kendall Nelson (diablo_rojo) > > [1] https://review.opendev.org/#/q/project:openstack/cloudkitty+OR+project:openstack/python-cloudkittyclient+OR+project:openstack/cloudkitty-dashboard > [2] https://launchpad.net/cloudkitty > > > On Tue, Aug 4, 2020 at 6:21 AM Rafael Weingärtner wrote: >> >> I am not sure how the projects/communities here in OpenStack are maintained and conducted, but I could for sure help. >> I am a committer and PMC for some Apache projects; therefore, I am a bit familiar with some processes in OpenSource communities. >> >> On Tue, Aug 4, 2020 at 5:11 AM Mark Goddard wrote: >>> >>> On Thu, 30 Jul 2020 at 14:43, Rafael Weingärtner >>> wrote: >>> > >>> > We are working on it. So far we have 3 open proposals there, but we do not have enough karma to move things along. >>> > Besides these 3 open proposals, we do have more ongoing extensions that have not yet been proposed to the community. >>> >>> It's good to hear you want to help improve cloudkitty, however it >>> sounds like what is required is help with maintaining the project. Is >>> that something you could be involved with? >>> Mark >>> >>> > >>> > On Thu, Jul 30, 2020 at 10:22 AM Sean McGinnis wrote: >>> >> >>> >> Posting here to raise awareness, and start discussion about next steps. >>> >> >>> >> It appears there is no one working on Cloudkitty anymore. No patches >>> >> have been merged for several months now, including simple bot proposed >>> >> patches. It would appear no one is maintaining this project anymore. >>> >> >>> >> I know there is a need out there for this type of functionality, so >>> >> maybe this will raise awareness and get some attention to it. But >>> >> barring that, I am wondering if we should start the process to retire >>> >> this project. >>> >> >>> >> From a Victoria release perspective, it is milestone-2 week, so we >>> >> should make a decision if any of the Cloudkitty deliverables should be >>> >> included in this release or not. We can certainly force releases of >>> >> whatever is the latest, but I think that is a bit risky since these >>> >> repos have never merged the job template change for victoria and >>> >> therefore are not even testing with Python 3.8. That is an official >>> >> runtime for Victoria, so we run the risk of having issues with the code >>> >> if someone runs under 3.8 but we have not tested to make sure there are >>> >> no problems doing so. >>> >> >>> >> I am hoping this at least starts the discussion. I will not propose any >>> >> release patches to remove anything until we have had a chance to discuss >>> >> the situation. >>> >> >>> >> Sean >>> >> >>> >> >>> > >>> > >>> > -- >>> > Rafael Weingärtner >> >> >> >> -- >> Rafael Weingärtner From skaplons at redhat.com Fri Aug 7 10:19:38 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 7 Aug 2020 12:19:38 +0200 Subject: [neutron][OVS firewall] Multicast non-IGMP traffic is allowed by default, not in iptables FW (LP#1889631) In-Reply-To: References: Message-ID: <38E3A820-FD9D-4A2B-B989-4735092D304F@redhat.com> Hi, > On 4 Aug 2020, at 19:05, Rodolfo Alonso Hernandez wrote: > > Hello all: > > First of all, the link: https://bugs.launchpad.net/neutron/+bug/1889631 > > To sum up the bug: in iptables FW, the non-IGMP multicast traffic from 224.0.0.x was blocked; this is not happening in OVS FW. > > That was discussed today in the Neutron meeting today [1]. We face two possible situations here: > - If we block this traffic now, some deployments using the OVS FW will experience an unexpected network blockage. I would be for this option but left stable branches not touched. Additionally we should of course add release note with info that this behaviour changed now and also we can add upgrade check which will write warning about that if any of the agents in the DB is using “openvswitch” firewall driver. I don’t think we can do anything more to warn users about such change. > - Deployments migrating from iptables to OVS FW, now won't be able to explicitly allow this traffic (or block it by default). This also breaks the current API, because some rules won't have any effect (those ones allowing this traffic). This is current issue, right? If we would fix it as You proposed above, then behaviour between both drivers would be the same. Am I understanding correct? > > A possible solution is to add a new knob in the FW configuration; this config option will allow to block or not this traffic by default. Remember that the FW can only create permissive rules, not blocking ones. I don’t like to add yet another config knob for that. And also as I think Akihiro mentioned it’s not good practice to change API behaviour depending on config options. This wouldn’t be discoverable in API. > > Any feedback is welcome! > > Regards. > > [1]http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-08-04-14.00.log.html#l-136 > > — Slawek Kaplonski Principal software engineer Red Hat From ralonsoh at redhat.com Fri Aug 7 10:30:53 2020 From: ralonsoh at redhat.com (Rodolfo Alonso Hernandez) Date: Fri, 7 Aug 2020 11:30:53 +0100 Subject: [neutron][OVS firewall] Multicast non-IGMP traffic is allowed by default, not in iptables FW (LP#1889631) In-Reply-To: <38E3A820-FD9D-4A2B-B989-4735092D304F@redhat.com> References: <38E3A820-FD9D-4A2B-B989-4735092D304F@redhat.com> Message-ID: Hi Slawek: I agree with Akihiro and you: - This should be fixed to match both FW behaviour, but only in master. - Of course, a "big" release note to make this public. - Not to add a knob that changes the API behaviour. I'll wait for more feedback. Although I'll be on PTO, I'll check and reply to the mail. Thank you and regards. On Fri, Aug 7, 2020 at 11:19 AM Slawek Kaplonski wrote: > Hi, > > > On 4 Aug 2020, at 19:05, Rodolfo Alonso Hernandez > wrote: > > > > Hello all: > > > > First of all, the link: https://bugs.launchpad.net/neutron/+bug/1889631 > > > > To sum up the bug: in iptables FW, the non-IGMP multicast traffic from > 224.0.0.x was blocked; this is not happening in OVS FW. > > > > That was discussed today in the Neutron meeting today [1]. We face two > possible situations here: > > - If we block this traffic now, some deployments using the OVS FW will > experience an unexpected network blockage. > > I would be for this option but left stable branches not touched. > Additionally we should of course add release note with info that this > behaviour changed now and also we can add upgrade check which will write > warning about that if any of the agents in the DB is using “openvswitch” > firewall driver. > I don’t think we can do anything more to warn users about such change. > > > - Deployments migrating from iptables to OVS FW, now won't be able to > explicitly allow this traffic (or block it by default). This also breaks > the current API, because some rules won't have any effect (those ones > allowing this traffic). > > This is current issue, right? If we would fix it as You proposed above, > then behaviour between both drivers would be the same. Am I understanding > correct? > > > > > A possible solution is to add a new knob in the FW configuration; this > config option will allow to block or not this traffic by default. Remember > that the FW can only create permissive rules, not blocking ones. > > I don’t like to add yet another config knob for that. And also as I think > Akihiro mentioned it’s not good practice to change API behaviour depending > on config options. This wouldn’t be discoverable in API. > > > > > Any feedback is welcome! > > > > Regards. > > > > [1] > http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-08-04-14.00.log.html#l-136 > > > > > > — > Slawek Kaplonski > Principal software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Fri Aug 7 12:45:15 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 7 Aug 2020 12:45:15 +0000 Subject: [openstack-community] [infra] Problem with ask.openstack.org In-Reply-To: <21ed671d-63d2-53e1-1ce0-31b977515be6@gmail.com> References: <6d128727-27b5-ff0d-6798-fbcf72998012@gmail.com> <21ed671d-63d2-53e1-1ce0-31b977515be6@gmail.com> Message-ID: <20200807124515.7k5xhijkj6mi4lec@yuggoth.org> On 2020-08-07 18:09:31 +0900 (+0900), Bernd Bausch wrote: > Sending people to Stackoverflow directly is a good option IMO. [...] Yes, ask.openstack.org was originally created for two reasons: 1. We could not keep up with the constant spam load on forums.openstack.org, but when we wanted to shut it down we kept hearing that many OpenStack users needed us to provide a Web forum because they wouldn't/couldn't use E-mail. 2. When we approached Stackexchange/Stackoverflow about getting a site like Ubuntu had, they said OpenStack was not popular enough software to warrant that. OSF originally contracted the author of Askbot to assist in maintaining the ask.openstack.org site, but his interests eventually moved on to other endeavors and the site has sat unmaintained (except for an occasional reboot by community infrastructure team sysadmins) for a number of years now. At this point it's a liability, and unless folks are interested in getting it back into a well-managed state I think we probably have no choice but to phase it out. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From gmann at ghanshyammann.com Fri Aug 7 13:33:18 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 07 Aug 2020 08:33:18 -0500 Subject: [TC] New Office Hour Plans In-Reply-To: References: Message-ID: <173c920361f.e63c2eb4109191.7004381406663804589@ghanshyammann.com> ---- On Thu, 06 Aug 2020 19:00:13 -0500 Kendall Nelson wrote ---- > Hello! > After taking a look at the poll results, Mohammed and I have two proposed plans for office hours: > Plan A: Two office hours instead of three. This gives us slightly more coverage than one office hour without overextending ourselves to cover three office hours. Mohammed and I were thinking that one of the reasons why three office hours wasn't working was that it was kind of a big time commitment and TC members could easily rationalize not going to ones later in the week if they had already attended one earlier in the week. The two times that enable most TC members to attend at least one, if not both, would be Monday @14:00 UTC (TC members available: Belmiro, Rico, Kristi, Jay, Mohammed, myself, and Nate + non members) and Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). > Plan B: Move to a single office hour on Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). Having only one office hour gives it more weight and importance and that should hopefully encourage more attendance from both community members and TC members alike. Thanks Kendal for following up on office hours plan. The idea of having multiple office hours was to cover TC availability across different TZ. Even Asia TZ office hours might not have many TC members available but still, someone to address/ack the issues and bring it to TC when most of the members are available. There was no expectation for all TC members to be present in all three office hours so all office hours being inactive might be due to some other reason not due to *many office hours*. Thursday office hour was most TC available one which is not the case anymore. I MO, we should consider the 'covering most of TZ (as much we can) for TC-availability' so in first option we can move either of the office hour in different TZ. I still in favor of moving to weekly TC meeting (in alternate TZ or so) than office hours but I am ok to give office hours a another try with new time. -gmann > I guess Plan C is to go ahead with Plan A and then if we don't see activity during the Monday time slot, to reduce down to one office hour and go with Plan B. > Please check out the patches Mohammed posted [1][2] and vote on what you'd prefer! > -Kendall (diablo_rojo) > [1] Dual Office Hour: https://review.opendev.org/#/c/745201/[2] Single Office Hour: https://review.opendev.org/#/c/745200/ From gmann at ghanshyammann.com Fri Aug 7 14:10:53 2020 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 07 Aug 2020 09:10:53 -0500 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Thanks, Pierre for helping with this. ttx has reached out to PTL (Justin Ferrieu (jferrieu) ) but I am not sure if he got any response back. Can you also send email to PTL as well as the current core team to add you in the core list for project maintenance? Please note that, migration of CI/CD to ubuntu work might break the cloudkitty gate if patches are not merged on time. I am still working on few repos though. -gmann ---- On Fri, 07 Aug 2020 04:18:48 -0500 Pierre Riteau wrote ---- > Thanks a lot for starting this discussion, I am also quite concerned about this. > > At StackHPC we started looking into CloudKitty a year ago, when the > community was still fairly active. There was an IRC meeting every > month or so throughout 2019. Patches were getting merged. > > Unfortunately in 2020 activity stopped abruptly. There hasn't been any > IRC meeting since early December and no patch has been merged since > the end of March. I have submitted straightforward stable backports of > bug fixes which have not received any answer. > > I am well aware of the difficulty of keeping up with open-source > project maintenance when work deadlines are always taking priority. If > the existing core team would be willing to grant +2 votes to more > people, I would be happy to participate in the maintenance of the > project. We've now deployed CloudKitty for several of our customers > and have to maintain a stable fork anyway. We would rather maintain > upstream directly! > > Pierre Riteau (priteau) > > > On Tue, 4 Aug 2020 at 23:22, Kendall Nelson wrote: > > > > I think the majority of 'maintenance' activities at the moment for Cloudkitty are the reviewing of open patches in gerrit [1] and triaging bugs that are reported in Launchpad[2] as they come in. When things come up on this mailing list that have the cloudkitty tag in the subject line (like this email), weighing in on them would also be helpful. > > > > If you need help getting setup with gerrit, I am happy to assist anyway I can :) > > > > -Kendall Nelson (diablo_rojo) > > > > [1] https://review.opendev.org/#/q/project:openstack/cloudkitty+OR+project:openstack/python-cloudkittyclient+OR+project:openstack/cloudkitty-dashboard > > [2] https://launchpad.net/cloudkitty > > > > > > On Tue, Aug 4, 2020 at 6:21 AM Rafael Weingärtner wrote: > >> > >> I am not sure how the projects/communities here in OpenStack are maintained and conducted, but I could for sure help. > >> I am a committer and PMC for some Apache projects; therefore, I am a bit familiar with some processes in OpenSource communities. > >> > >> On Tue, Aug 4, 2020 at 5:11 AM Mark Goddard wrote: > >>> > >>> On Thu, 30 Jul 2020 at 14:43, Rafael Weingärtner > >>> wrote: > >>> > > >>> > We are working on it. So far we have 3 open proposals there, but we do not have enough karma to move things along. > >>> > Besides these 3 open proposals, we do have more ongoing extensions that have not yet been proposed to the community. > >>> > >>> It's good to hear you want to help improve cloudkitty, however it > >>> sounds like what is required is help with maintaining the project. Is > >>> that something you could be involved with? > >>> Mark > >>> > >>> > > >>> > On Thu, Jul 30, 2020 at 10:22 AM Sean McGinnis wrote: > >>> >> > >>> >> Posting here to raise awareness, and start discussion about next steps. > >>> >> > >>> >> It appears there is no one working on Cloudkitty anymore. No patches > >>> >> have been merged for several months now, including simple bot proposed > >>> >> patches. It would appear no one is maintaining this project anymore. > >>> >> > >>> >> I know there is a need out there for this type of functionality, so > >>> >> maybe this will raise awareness and get some attention to it. But > >>> >> barring that, I am wondering if we should start the process to retire > >>> >> this project. > >>> >> > >>> >> From a Victoria release perspective, it is milestone-2 week, so we > >>> >> should make a decision if any of the Cloudkitty deliverables should be > >>> >> included in this release or not. We can certainly force releases of > >>> >> whatever is the latest, but I think that is a bit risky since these > >>> >> repos have never merged the job template change for victoria and > >>> >> therefore are not even testing with Python 3.8. That is an official > >>> >> runtime for Victoria, so we run the risk of having issues with the code > >>> >> if someone runs under 3.8 but we have not tested to make sure there are > >>> >> no problems doing so. > >>> >> > >>> >> I am hoping this at least starts the discussion. I will not propose any > >>> >> release patches to remove anything until we have had a chance to discuss > >>> >> the situation. > >>> >> > >>> >> Sean > >>> >> > >>> >> > >>> > > >>> > > >>> > -- > >>> > Rafael Weingärtner > >> > >> > >> > >> -- > >> Rafael Weingärtner > > From mark at stackhpc.com Fri Aug 7 14:11:12 2020 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 7 Aug 2020 15:11:12 +0100 Subject: [kolla] Kolla klub break Message-ID: Hi, We agreed in Wednesday's IRC meeting to take a short summer break from the klub. Let's meet again on 10th September. Thanks to everyone who has taken part in these meetings so far, we've had some really great discussions. As always, if anyone has ideas for topics, please add them to the Google doc. Looking forward to some more great sessions in September. https://docs.google.com/document/d/1EwQs2GXF-EvJZamEx9vQAOSDB5tCjsDCJyHQN5_4_Sw/edit# Thanks, Mark From balazs.gibizer at est.tech Fri Aug 7 15:26:53 2020 From: balazs.gibizer at est.tech (=?iso-8859-1?q?Bal=E1zs?= Gibizer) Date: Fri, 07 Aug 2020 17:26:53 +0200 Subject: [nova] Nova PTL is on PTO until 24th of Aug Message-ID: Hi, I will be on vacation during the next two weeks. Cheers, gibi From pierre at stackhpc.com Fri Aug 7 16:10:45 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Fri, 7 Aug 2020 18:10:45 +0200 Subject: Helping out with CloudKitty maintenance Message-ID: Hello, Following the discussion about the state of CloudKitty [1], I would like to volunteer my help with maintaining the project, as no one of the core team appears to be active at the moment. I have been working with CloudKitty for about a year and have used both the Gnocchi and Monasca collectors. Being a core reviewer on two other OpenStack projects, I am familiar with the process of maintaining OpenStack code. Would it be possible to get core reviewer privileges to help? I would initially focus on keeping CI green and making sure bug fixes are merged and backported. Thanks in advance, Pierre Riteau (priteau) [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016384.html From rafaelweingartner at gmail.com Fri Aug 7 16:21:53 2020 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Fri, 7 Aug 2020 13:21:53 -0300 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: Message-ID: I see. Thanks for the heads up. I will try to dedicate some time every week for these tasks. On Tue, Aug 4, 2020 at 6:22 PM Kendall Nelson wrote: > I think the majority of 'maintenance' activities at the moment for > Cloudkitty are the reviewing of open patches in gerrit [1] and triaging > bugs that are reported in Launchpad[2] as they come in. When things come up > on this mailing list that have the cloudkitty tag in the subject line (like > this email), weighing in on them would also be helpful. > > If you need help getting setup with gerrit, I am happy to assist anyway I > can :) > > -Kendall Nelson (diablo_rojo) > > [1] > https://review.opendev.org/#/q/project:openstack/cloudkitty+OR+project:openstack/python-cloudkittyclient+OR+project:openstack/cloudkitty-dashboard > [2] https://launchpad.net/cloudkitty > > > On Tue, Aug 4, 2020 at 6:21 AM Rafael Weingärtner < > rafaelweingartner at gmail.com> wrote: > >> I am not sure how the projects/communities here in OpenStack are >> maintained and conducted, but I could for sure help. >> I am a committer and PMC for some Apache projects; therefore, I am a bit >> familiar with some processes in OpenSource communities. >> >> On Tue, Aug 4, 2020 at 5:11 AM Mark Goddard wrote: >> >>> On Thu, 30 Jul 2020 at 14:43, Rafael Weingärtner >>> wrote: >>> > >>> > We are working on it. So far we have 3 open proposals there, but we do >>> not have enough karma to move things along. >>> > Besides these 3 open proposals, we do have more ongoing extensions >>> that have not yet been proposed to the community. >>> >>> It's good to hear you want to help improve cloudkitty, however it >>> sounds like what is required is help with maintaining the project. Is >>> that something you could be involved with? >>> Mark >>> >>> > >>> > On Thu, Jul 30, 2020 at 10:22 AM Sean McGinnis >>> wrote: >>> >> >>> >> Posting here to raise awareness, and start discussion about next >>> steps. >>> >> >>> >> It appears there is no one working on Cloudkitty anymore. No patches >>> >> have been merged for several months now, including simple bot proposed >>> >> patches. It would appear no one is maintaining this project anymore. >>> >> >>> >> I know there is a need out there for this type of functionality, so >>> >> maybe this will raise awareness and get some attention to it. But >>> >> barring that, I am wondering if we should start the process to retire >>> >> this project. >>> >> >>> >> From a Victoria release perspective, it is milestone-2 week, so we >>> >> should make a decision if any of the Cloudkitty deliverables should be >>> >> included in this release or not. We can certainly force releases of >>> >> whatever is the latest, but I think that is a bit risky since these >>> >> repos have never merged the job template change for victoria and >>> >> therefore are not even testing with Python 3.8. That is an official >>> >> runtime for Victoria, so we run the risk of having issues with the >>> code >>> >> if someone runs under 3.8 but we have not tested to make sure there >>> are >>> >> no problems doing so. >>> >> >>> >> I am hoping this at least starts the discussion. I will not propose >>> any >>> >> release patches to remove anything until we have had a chance to >>> discuss >>> >> the situation. >>> >> >>> >> Sean >>> >> >>> >> >>> > >>> > >>> > -- >>> > Rafael Weingärtner >>> >> >> >> -- >> Rafael Weingärtner >> > -- Rafael Weingärtner -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.ramirez at opencloud.es Fri Aug 7 16:30:30 2020 From: luis.ramirez at opencloud.es (Luis Ramirez) Date: Fri, 7 Aug 2020 18:30:30 +0200 Subject: Helping out with CloudKitty maintenance In-Reply-To: References: Message-ID: Hi, +1. We need to move fwd to keep it active. I’m also working on a charm for CloudKitty. Br Luis Rmz El El vie, 7 ago 2020 a las 18:16, Pierre Riteau escribió: > Hello, > > Following the discussion about the state of CloudKitty [1], I would > like to volunteer my help with maintaining the project, as no one of > the core team appears to be active at the moment. I have been working > with CloudKitty for about a year and have used both the Gnocchi and > Monasca collectors. Being a core reviewer on two other OpenStack > projects, I am familiar with the process of maintaining OpenStack > code. > > Would it be possible to get core reviewer privileges to help? I would > initially focus on keeping CI green and making sure bug fixes are > merged and backported. > > Thanks in advance, > Pierre Riteau (priteau) > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016384.html > > -- Br, Luis Rmz Blockchain, DevOps & Open Source Cloud Solutions Architect ---------------------------------------- Founder & CEO OpenCloud.es luis.ramirez at opencloud.es Skype ID: d.overload Hangouts: luis.ramirez at opencloud.es +34 911 950 123 / +39 392 1289553 / +49 152 26917722 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungleboyj at gmail.com Fri Aug 7 17:12:25 2020 From: jungleboyj at gmail.com (Jay Bryant) Date: Fri, 7 Aug 2020 12:12:25 -0500 Subject: [TC] New Office Hour Plans In-Reply-To: <173c920361f.e63c2eb4109191.7004381406663804589@ghanshyammann.com> References: <173c920361f.e63c2eb4109191.7004381406663804589@ghanshyammann.com> Message-ID: <976bf811-536b-faff-cb30-dbab1ac6d83a@gmail.com> On 8/7/2020 8:33 AM, Ghanshyam Mann wrote: > ---- On Thu, 06 Aug 2020 19:00:13 -0500 Kendall Nelson wrote ---- > > Hello! > > After taking a look at the poll results, Mohammed and I have two proposed plans for office hours: > > Plan A: Two office hours instead of three. This gives us slightly more coverage than one office hour without overextending ourselves to cover three office hours. Mohammed and I were thinking that one of the reasons why three office hours wasn't working was that it was kind of a big time commitment and TC members could easily rationalize not going to ones later in the week if they had already attended one earlier in the week. The two times that enable most TC members to attend at least one, if not both, would be Monday @14:00 UTC (TC members available: Belmiro, Rico, Kristi, Jay, Mohammed, myself, and Nate + non members) and Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). > > Plan B: Move to a single office hour on Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). Having only one office hour gives it more weight and importance and that should hopefully encourage more attendance from both community members and TC members alike. > > Thanks Kendal for following up on office hours plan. > > The idea of having multiple office hours was to cover TC availability across different TZ. Even Asia TZ office > hours might not have many TC members available but still, someone to address/ack the issues and bring it to > TC when most of the members are available. There was no expectation for all TC members to be present in all > three office hours so all office hours being inactive might be due to some other reason not due to *many office hours*. > > Thursday office hour was most TC available one which is not the case anymore. > > I MO, we should consider the 'covering most of TZ (as much we can) for TC-availability' so in first option we can move > either of the office hour in different TZ. I think that Gmann makes a good point here.  If we are going to have multiple office hours one should be in an AP timezone.  Was there a second time where the most people were in the AP timeframe were available? So, reduce to two office hours and try to cover both sides of the world? Jay > > I still in favor of moving to weekly TC meeting (in alternate TZ or so) than office hours but I am ok to give office hours a > another try with new time. > > -gmann > > > I guess Plan C is to go ahead with Plan A and then if we don't see activity during the Monday time slot, to reduce down to one office hour and go with Plan B. > > Please check out the patches Mohammed posted [1][2] and vote on what you'd prefer! > > -Kendall (diablo_rojo) > > [1] Dual Office Hour: https://review.opendev.org/#/c/745201/[2] Single Office Hour: https://review.opendev.org/#/c/745200/ > From sean.mcginnis at gmx.com Fri Aug 7 19:56:38 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Fri, 7 Aug 2020 14:56:38 -0500 Subject: [all] Proposed Wallaby cycle schedule Message-ID: <2e56de68-c416-e3ea-f3da-caaf9399287d@gmx.com> Hey everyone, The Victoria cycle is going by fast, and it's already time to start planning some of the early things for the Wallaby release. One of the first steps for that is actually deciding on the release schedule. Typically we have done this based on when the next Summit event was planned to take place. Due to several reasons, we don't have a date yet for the first 2021 event. The current thinking is it will likely take place in May (nothing is set, just an educated guess, so please don't use that for any other planning). So for the sake of figuring out the release schedule, we are targeting a release date in early May. Hopefully this will then align well with event plans. I have a proposed release schedule up for review here: https://review.opendev.org/#/c/744729/ For ease of viewing (until the job logs are garbage collected), you can see the rendered schedule here: https://0e6b8aeca433e85b429b-46fd243db6dc394538bd0555f339eba5.ssl.cf1.rackcdn.com/744729/3/check/openstack-tox-docs/4f76901/docs/wallaby/schedule.html There are always outside conflicts, but I think this has aligned mostly well with major holidays. But please feel free to comment on the patch if you see any major issues that we may have not considered. Thanks! Sean From mnaser at vexxhost.com Fri Aug 7 20:22:08 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Fri, 7 Aug 2020 16:22:08 -0400 Subject: [TC] New Office Hour Plans In-Reply-To: <173c920361f.e63c2eb4109191.7004381406663804589@ghanshyammann.com> References: <173c920361f.e63c2eb4109191.7004381406663804589@ghanshyammann.com> Message-ID: On Fri, Aug 7, 2020 at 9:37 AM Ghanshyam Mann wrote: > > ---- On Thu, 06 Aug 2020 19:00:13 -0500 Kendall Nelson wrote ---- > > Hello! > > After taking a look at the poll results, Mohammed and I have two proposed plans for office hours: > > Plan A: Two office hours instead of three. This gives us slightly more coverage than one office hour without overextending ourselves to cover three office hours. Mohammed and I were thinking that one of the reasons why three office hours wasn't working was that it was kind of a big time commitment and TC members could easily rationalize not going to ones later in the week if they had already attended one earlier in the week. The two times that enable most TC members to attend at least one, if not both, would be Monday @14:00 UTC (TC members available: Belmiro, Rico, Kristi, Jay, Mohammed, myself, and Nate + non members) and Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). > > Plan B: Move to a single office hour on Wednesday @ 15:00 UTC (TC members available: Rico, Kristi, Jay, Mohammed, myself, Ghanshyam, and Nate + non members). Having only one office hour gives it more weight and importance and that should hopefully encourage more attendance from both community members and TC members alike. > > Thanks Kendal for following up on office hours plan. > > The idea of having multiple office hours was to cover TC availability across different TZ. Even Asia TZ office > hours might not have many TC members available but still, someone to address/ack the issues and bring it to > TC when most of the members are available. There was no expectation for all TC members to be present in all > three office hours so all office hours being inactive might be due to some other reason not due to *many office hours*. I think if we limit the number of times, then more people can likely show up because it's a smaller commitment. The 2 office hours except Thursday are pretty much non-existant at that point > Thursday office hour was most TC available one which is not the case anymore. The Wednesday was actually the time where we had 10 people mention they'd be available, 9 of them being TC members. I'm hoping that is the most successful time line > I MO, we should consider the 'covering most of TZ (as much we can) for TC-availability' so in first option we can move > either of the office hour in different TZ. > > I still in favor of moving to weekly TC meeting (in alternate TZ or so) than office hours but I am ok to give office hours a > another try with new time. I think given the commitment I see here, I am confident Wednesday should be successful: https://doodle.com/poll/q27t8pucq7b8xbme > -gmann > > > I guess Plan C is to go ahead with Plan A and then if we don't see activity during the Monday time slot, to reduce down to one office hour and go with Plan B. > > Please check out the patches Mohammed posted [1][2] and vote on what you'd prefer! > > -Kendall (diablo_rojo) > > [1] Dual Office Hour: https://review.opendev.org/#/c/745201/[2] Single Office Hour: https://review.opendev.org/#/c/745200/ > -- Mohammed Naser VEXXHOST, Inc. From its-openstack at zohocorp.com Fri Aug 7 07:21:32 2020 From: its-openstack at zohocorp.com (its-openstack at zohocorp.com) Date: Fri, 07 Aug 2020 12:51:32 +0530 Subject: Openstack-Train VCPU issue in Hyper-V Message-ID: <173c7cbda12.e31c17678315.6864041805036536996@zohocorp.com> Dear Team,    We are using Openstack-Train in our organization.We have created windows server 2016 Std R2 instances with this flavor m5.xlarge ( RAM - 65536 , Disk - 500 , VCPUs - 16 ).Once Hyper-V future enabled in this instances VCPU count is automatically reduced to 1 core after restart.Even we have enabled nested virtualisation in openstack compute server.Herewith attached screenshot for your references.Please help us to short out this issue. #cat /sys/module/kvm_intel/parameters/nested Y Regards, Sysadmin. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Before Hyper-V.bmp Type: application/octet-stream Size: 2306502 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: After Hyper-V .png Type: image/png Size: 47337 bytes Desc: not available URL: From cohuck at redhat.com Fri Aug 7 11:59:42 2020 From: cohuck at redhat.com (Cornelia Huck) Date: Fri, 7 Aug 2020 13:59:42 +0200 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <4cf2824c803c96496e846c5b06767db305e9fb5a.camel@redhat.com> References: <20200727072440.GA28676@joy-OptiPlex-7040> <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> <20200805093338.GC30485@joy-OptiPlex-7040> <20200805105319.GF2177@nanopsycho> <4cf2824c803c96496e846c5b06767db305e9fb5a.camel@redhat.com> Message-ID: <20200807135942.5d56a202.cohuck@redhat.com> On Wed, 05 Aug 2020 12:35:01 +0100 Sean Mooney wrote: > On Wed, 2020-08-05 at 12:53 +0200, Jiri Pirko wrote: > > Wed, Aug 05, 2020 at 11:33:38AM CEST, yan.y.zhao at intel.com wrote: (...) > > > software_version: device driver's version. > > > in .[.bugfix] scheme, where there is no > > > compatibility across major versions, minor versions have > > > forward compatibility (ex. 1-> 2 is ok, 2 -> 1 is not) and > > > bugfix version number indicates some degree of internal > > > improvement that is not visible to the user in terms of > > > features or compatibility, > > > > > > vendor specific attributes: each vendor may define different attributes > > > device id : device id of a physical devices or mdev's parent pci device. > > > it could be equal to pci id for pci devices > > > aggregator: used together with mdev_type. e.g. aggregator=2 together > > > with i915-GVTg_V5_4 means 2*1/4=1/2 of a gen9 Intel > > > graphics device. > > > remote_url: for a local NVMe VF, it may be configured with a remote > > > url of a remote storage and all data is stored in the > > > remote side specified by the remote url. > > > ... > just a minor not that i find ^ much more simmple to understand then > the current proposal with self and compatiable. > if i have well defiend attibute that i can parse and understand that allow > me to calulate the what is and is not compatible that is likely going to > more useful as you wont have to keep maintianing a list of other compatible > devices every time a new sku is released. > > in anycase thank for actully shareing ^ as it make it simpler to reson about what > you have previously proposed. So, what would be the most helpful format? A 'software_version' field that follows the conventions outlined above, and other (possibly optional) fields that have to match? (...) > > Thanks for the explanation, I'm still fuzzy about the details. > > Anyway, I suggest you to check "devlink dev info" command we have > > implemented for multiple drivers. > > is devlink exposed as a filesytem we can read with just open? > openstack will likely try to leverage libvirt to get this info but when we > cant its much simpler to read sysfs then it is to take a a depenency on a commandline > too and have to fork shell to execute it and parse the cli output. > pyroute2 which we use in some openstack poject has basic python binding for devlink but im not > sure how complete it is as i think its relitivly new addtion. if we need to take a dependcy > we will but that would be a drawback fo devlink not that that is a large one just something > to keep in mind. A devlinkfs, maybe? At least for reading information (IIUC, "devlink dev info" is only about information retrieval, right?) From sean.mcginnis at gmx.com Fri Aug 7 21:09:00 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Fri, 7 Aug 2020 16:09:00 -0500 Subject: [sigs][vendors] Proposal to create Hardware Vendor SIG Message-ID: <5d4928c2-8e14-82a7-c06b-dd8df4de44fb@gmx.com> Hey everyone, OpenStack is a community for creating open source software, but by the nature of infrastructure management, there is a strong intersection with hardware, and therefore hardware vendors. Nova, Cinder, Neutron, Ironic, and many others need to interact with hardware and support vendor specific drivers for this interaction. There is currently a spectrum where some of this hardware interaction is done as openly as possible, while others develop their integration "glue" behind closed doors. Part of this is the disconnect between the fully open development of OpenStack, and the traditionally proprietary development of many products. For those that do want to do their vendor-specific development as openly as possible - and hopefully attract the involvement of customers, partners, and others in the community - there hasn't been a great venue for this so far. Some vendors that have done open development have even had difficulty finding a place in the community and ended up deciding to just develop behind closed doors. I would like to try to change this, so I am proposing the creation of the Hardware Vendor SIG as a place where we can collaborate on vendor specific things, and encourage development to happen in the open. This would be a place for any vendors and interested parties to work together to fix bugs, implement features, and overall improve the quality of anything that helps provide that glue to bridge the gap between our open source services and vendor hardware. This would include servers, storage, networking, and really anything else that plays a role in being able to set up an OpenStack cloud. This is a call out to any others interested in participating. If you are interested in this effort, and if you have any existing code (whether hosted on OpenDev, hosted on GitHub, or hosted on your own platform) that you think would be a good fit for this, please add your contact information and any relevant details here: https://etherpad.opendev.org/p/HardwareVendorSIG Also, please feel free to show your support by voting on the proposal to create this SIG here: https://review.opendev.org/#/c/745185/ Thanks! Sean From zigo at debian.org Fri Aug 7 21:19:45 2020 From: zigo at debian.org (Thomas Goirand) Date: Fri, 7 Aug 2020 23:19:45 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: On 8/7/20 4:10 PM, Ghanshyam Mann wrote: > Thanks, Pierre for helping with this. > > ttx has reached out to PTL (Justin Ferrieu (jferrieu) ) > but I am not sure if he got any response back. The end of the very good maintenance of Cloudkitty matched the date when objectif libre was sold to Linkbynet. Maybe the new owner don't care enough? This is very disappointing as I've been using it for some time already, and that I was satisfied by it (ie: it does the job...), and especially that latest releases are able to scale correctly. I very much would love if Pierre Riteau was successful in taking over. Good luck Pierre! I'll try to help whenever I can and if I'm not too busy. Cheers, Thomas Goirand (zigo) From Arkady.Kanevsky at dell.com Sat Aug 8 02:48:01 2020 From: Arkady.Kanevsky at dell.com (Kanevsky, Arkady) Date: Sat, 8 Aug 2020 02:48:01 +0000 Subject: [sigs][vendors] Proposal to create Hardware Vendor SIG In-Reply-To: <5d4928c2-8e14-82a7-c06b-dd8df4de44fb@gmx.com> References: <5d4928c2-8e14-82a7-c06b-dd8df4de44fb@gmx.com> Message-ID: Great idea. Long time overdue. Great place for many out-of-tree repos. Thanks Arkady -----Original Message----- From: Sean McGinnis Sent: Friday, August 7, 2020 4:09 PM To: openstack-discuss Subject: [sigs][vendors] Proposal to create Hardware Vendor SIG [EXTERNAL EMAIL] Hey everyone, OpenStack is a community for creating open source software, but by the nature of infrastructure management, there is a strong intersection with hardware, and therefore hardware vendors. Nova, Cinder, Neutron, Ironic, and many others need to interact with hardware and support vendor specific drivers for this interaction. There is currently a spectrum where some of this hardware interaction is done as openly as possible, while others develop their integration "glue" behind closed doors. Part of this is the disconnect between the fully open development of OpenStack, and the traditionally proprietary development of many products. For those that do want to do their vendor-specific development as openly as possible - and hopefully attract the involvement of customers, partners, and others in the community - there hasn't been a great venue for this so far. Some vendors that have done open development have even had difficulty finding a place in the community and ended up deciding to just develop behind closed doors. I would like to try to change this, so I am proposing the creation of the Hardware Vendor SIG as a place where we can collaborate on vendor specific things, and encourage development to happen in the open. This would be a place for any vendors and interested parties to work together to fix bugs, implement features, and overall improve the quality of anything that helps provide that glue to bridge the gap between our open source services and vendor hardware. This would include servers, storage, networking, and really anything else that plays a role in being able to set up an OpenStack cloud. This is a call out to any others interested in participating. If you are interested in this effort, and if you have any existing code (whether hosted on OpenDev, hosted on GitHub, or hosted on your own platform) that you think would be a good fit for this, please add your contact information and any relevant details here: https://etherpad.opendev.org/p/HardwareVendorSIG Also, please feel free to show your support by voting on the proposal to create this SIG here: https://review.opendev.org/#/c/745185/ Thanks! Sean From dev.faz at gmail.com Sat Aug 8 04:30:09 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Sat, 8 Aug 2020 06:30:09 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: <20200806144016.GP31915@sync> References: <20200806144016.GP31915@sync> Message-ID: Hi, we also have this issue. Our solution was (up to now) to delete the queues with a script or even reset the complete cluster. We just upgraded rabbitmq to the latest version - without luck. Anyone else seeing this issue? Fabian Arnaud Morin schrieb am Do., 6. Aug. 2020, 16:47: > Hey all, > > I would like to ask the community about a rabbit issue we have from time > to time. > > In our current architecture, we have a cluster of rabbits (3 nodes) for > all our OpenStack services (mostly nova and neutron). > > When one node of this cluster is down, the cluster continue working (we > use pause_minority strategy). > But, sometimes, the third server is not able to recover automatically > and need a manual intervention. > After this intervention, we restart the rabbitmq-server process, which > is then able to join the cluster back. > > At this time, the cluster looks ok, everything is fine. > BUT, nothing works. > Neutron and nova agents are not able to report back to servers. > They appear dead. > Servers seems not being able to consume messages. > The exchanges, queues, bindings seems good in rabbit. > > What we see is that removing bindings (using rabbitmqadmin delete > binding or the web interface) and recreate them again (using the same > routing key) brings the service back up and running. > > Doing this for all queues is really painful. Our next plan is to > automate it, but is there anyone in the community already saw this kind > of issues? > > Our bug looks like the one described in [1]. > Someone recommands to create an Alternate Exchange. > Is there anyone already tried that? > > FYI, we are running rabbit 3.8.2 (with OpenStack Stein). > We had the same kind of issues using older version of rabbit. > > Thanks for your help. > > [1] > https://groups.google.com/forum/#!newtopic/rabbitmq-users/rabbitmq-users/zFhmpHF2aWk > > -- > Arnaud Morin > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bhartendu at gmail.com Sat Aug 8 07:00:51 2020 From: bhartendu at gmail.com (Bhartendu) Date: Sat, 8 Aug 2020 12:30:51 +0530 Subject: Openstack VMmachine internet connectivity issue Message-ID: Hi All, I need help or any pointer to solve Openstack VM machine internet connectivity issue. I am successfully able to create VM machines on Openstack cloud. Facing following Openstack VM machine internet connectivity issues: 1) There is no internet connectivity from Openstack VM machine. 2) ping is successful till Openstack machine (192.168.0.166), but Gateway ip (192.168.0.1) not reachable from Openstack VM machine. 3) No website reachable from Openstack VM machine. Openstack is installed on Virtualbox VM machine (CentOS 8.2). Connectivity information: Internet<=====>Router(192.168.0.1)<=====>Oracle VM machine (CentOS 8.2; 192.168.0.166)<=====>Openstack VM Machine (192.168.0.174) Any help or trigger is much appreciated. ----------------------------------------------- Openstack VM Machine (192.168.0.174) Logs ----------------------------------------------- $ ip route default via 192.168.0.1 dev eth0 169.254.169.254 via 192.168.0.171 dev eth0 192.168.0.0/24 dev eth0 scope link src 192.168.0.174 $ ping 192.168.0.166 PING 192.168.0.166 (192.168.0.166): 56 data bytes 64 bytes from 192.168.0.166: seq=0 ttl=64 time=2.657 ms 64 bytes from 192.168.0.166: seq=1 ttl=64 time=1.196 ms 64 bytes from 192.168.0.166: seq=2 ttl=64 time=1.312 ms 64 bytes from 192.168.0.166: seq=3 ttl=64 time=0.875 ms 64 bytes from 192.168.0.166: seq=4 ttl=64 time=0.782 ms ^C --- 192.168.0.166 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 0.782/1.364/2.657 ms $ ping 192.168.0.1 PING 192.168.0.1 (192.168.0.1): 56 data bytes ^C --- 192.168.0.1 ping statistics --- 3 packets transmitted, 0 packets received, 100% packet loss $ ping google.com ping: bad address 'google.com' $ $ sudo cat /etc/resolv.conf nameserver 192.168.0.1 nameserver 192.168.0.166 nameserver 8.8.8.8 $ $ ip route default via 192.168.0.1 dev eth0 169.254.169.254 via 192.168.0.171 dev eth0 192.168.0.0/24 dev eth0 scope link src 192.168.0.174 $ $ ifconfig eth0 Link encap:Ethernet HWaddr FA:16:3E:17:F2:F9 inet addr:192.168.0.174 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe17:f2f9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:572 errors:0 dropped:0 overruns:0 frame:0 TX packets:571 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:51697 (50.4 KiB) TX bytes:46506 (45.4 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:35 errors:0 dropped:0 overruns:0 frame:0 TX packets:35 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3764 (3.6 KiB) TX bytes:3764 (3.6 KiB) $ ----------------------------------------------------- Oracle VM machine (CentOS 8.2; 192.168.0.166) Logs ----------------------------------------------------- [root at openstack ~]# ovs-vsctl show 6718c4ce-6a58-463c-95e4-20a34edbe041 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-int fail_mode: secure datapath_type: system Port br-int Interface br-int type: internal Port "tap50a66e3f-00" Interface "tap50a66e3f-00" Port "patch-br-int-to-provnet-27804412-47c0-497d-8f2e-8c0bf8b04df1" Interface "patch-br-int-to-provnet-27804412-47c0-497d-8f2e-8c0bf8b04df1" type: patch options: {peer="patch-provnet-27804412-47c0-497d-8f2e-8c0bf8b04df1-to-br-int"} Port "tap6ea42aaf-80" Interface "tap6ea42aaf-80" Port "tap743cdf36-c8" Interface "tap743cdf36-c8" Port "tap2a93518d-90" Interface "tap2a93518d-90" Bridge br-ex fail_mode: standalone Port "patch-provnet-27804412-47c0-497d-8f2e-8c0bf8b04df1-to-br-int" Interface "patch-provnet-27804412-47c0-497d-8f2e-8c0bf8b04df1-to-br-int" type: patch options: {peer="patch-br-int-to-provnet-27804412-47c0-497d-8f2e-8c0bf8b04df1"} Port br-ex Interface br-ex type: internal Port "enp0s3" Interface "enp0s3" ovs_version: "2.12.0" [root at openstack ~]# ip a s enp0s3 2: enp0s3: mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000 link/ether 08:00:27:cd:fc:4f brd ff:ff:ff:ff:ff:ff [root at openstack ~]# ip a s br-ex 13: br-ex: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 08:00:27:cd:fc:4f brd ff:ff:ff:ff:ff:ff inet 192.168.0.166/24 brd 192.168.0.255 scope global br-ex valid_lft forever preferred_lft forever inet6 fe80::f057:6bff:fe69:1f47/64 scope link valid_lft forever preferred_lft forever [root at openstack ~]# [root at openstack ~]# cat /etc/resolv.conf # Generated by NetworkManager search example.com nameserver 192.168.0.1 nameserver 8.8.8.8 [root at openstack ~]# [root at openstack ~]# ip route default via 192.168.0.1 dev br-ex 169.254.0.0/16 dev br-ex scope link metric 1013 192.168.0.0/24 dev br-ex proto kernel scope link src 192.168.0.166 [root at openstack ~]# [root at openstack ~]# ifconfig br-ex: flags=4163 mtu 1500 inet 192.168.0.166 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::f057:6bff:fe69:1f47 prefixlen 64 scopeid 0x20 ether 08:00:27:cd:fc:4f txqueuelen 1000 (Ethernet) RX packets 2781 bytes 505203 (493.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1441 bytes 159474 (155.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s3: flags=4163 mtu 1500 ether 08:00:27:cd:fc:4f txqueuelen 1000 (Ethernet) RX packets 94856 bytes 111711956 (106.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 33859 bytes 15114156 (14.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 1741807 bytes 441610166 (421.1 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1741807 bytes 441610166 (421.1 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 tap2a93518d-90: flags=4163 mtu 1500 inet6 fe80::681d:e6ff:fe1d:963f prefixlen 64 scopeid 0x20 ether 6a:1d:e6:1d:96:3f txqueuelen 1000 (Ethernet) RX packets 55 bytes 6540 (6.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 2950 bytes 916693 (895.2 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 tap50a66e3f-00: flags=4163 mtu 1442 ether fe:16:3e:06:df:bf txqueuelen 1000 (Ethernet) RX packets 689 bytes 59074 (57.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 703 bytes 58933 (57.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 tap6ea42aaf-80: flags=4163 mtu 1500 inet6 fe80::745b:6aff:fe1c:d270 prefixlen 64 scopeid 0x20 ether 76:5b:6a:1c:d2:70 txqueuelen 1000 (Ethernet) RX packets 63 bytes 7332 (7.1 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 126 bytes 10692 (10.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 tap743cdf36-c8: flags=4163 mtu 1500 ether fe:16:3e:17:f2:f9 txqueuelen 1000 (Ethernet) RX packets 1005 bytes 91770 (89.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3555 bytes 902987 (881.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root at openstack ~]# Thanks & Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From massimo.sgaravatto at gmail.com Sat Aug 8 07:36:40 2020 From: massimo.sgaravatto at gmail.com (Massimo Sgaravatto) Date: Sat, 8 Aug 2020 09:36:40 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: References: <20200806144016.GP31915@sync> Message-ID: We also see the issue. When it happens stopping and restarting the rabbit cluster usually helps. I thought the problem was because of a wrong setting in the openstack services conf files: I missed these settings (that I am now going to add): [oslo_messaging_rabbit] rabbit_ha_queues = true amqp_durable_queues = true Cheers, Massimo On Sat, Aug 8, 2020 at 6:34 AM Fabian Zimmermann wrote: > Hi, > > we also have this issue. > > Our solution was (up to now) to delete the queues with a script or even > reset the complete cluster. > > We just upgraded rabbitmq to the latest version - without luck. > > Anyone else seeing this issue? > > Fabian > > > > Arnaud Morin schrieb am Do., 6. Aug. 2020, 16:47: > >> Hey all, >> >> I would like to ask the community about a rabbit issue we have from time >> to time. >> >> In our current architecture, we have a cluster of rabbits (3 nodes) for >> all our OpenStack services (mostly nova and neutron). >> >> When one node of this cluster is down, the cluster continue working (we >> use pause_minority strategy). >> But, sometimes, the third server is not able to recover automatically >> and need a manual intervention. >> After this intervention, we restart the rabbitmq-server process, which >> is then able to join the cluster back. >> >> At this time, the cluster looks ok, everything is fine. >> BUT, nothing works. >> Neutron and nova agents are not able to report back to servers. >> They appear dead. >> Servers seems not being able to consume messages. >> The exchanges, queues, bindings seems good in rabbit. >> >> What we see is that removing bindings (using rabbitmqadmin delete >> binding or the web interface) and recreate them again (using the same >> routing key) brings the service back up and running. >> >> Doing this for all queues is really painful. Our next plan is to >> automate it, but is there anyone in the community already saw this kind >> of issues? >> >> Our bug looks like the one described in [1]. >> Someone recommands to create an Alternate Exchange. >> Is there anyone already tried that? >> >> FYI, we are running rabbit 3.8.2 (with OpenStack Stein). >> We had the same kind of issues using older version of rabbit. >> >> Thanks for your help. >> >> [1] >> https://groups.google.com/forum/#!newtopic/rabbitmq-users/rabbitmq-users/zFhmpHF2aWk >> >> -- >> Arnaud Morin >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dev.faz at gmail.com Sat Aug 8 13:06:36 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Sat, 8 Aug 2020 15:06:36 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: References: <20200806144016.GP31915@sync> Message-ID: Hi, dont know if durable queues help, but should be enabled by rabbitmq policy which (alone) doesnt seem to fix this (we have this active) Fabian Massimo Sgaravatto schrieb am Sa., 8. Aug. 2020, 09:36: > We also see the issue. When it happens stopping and restarting the rabbit > cluster usually helps. > > I thought the problem was because of a wrong setting in the openstack > services conf files: I missed these settings (that I am now going to add): > > [oslo_messaging_rabbit] > rabbit_ha_queues = true > amqp_durable_queues = true > > Cheers, Massimo > > > On Sat, Aug 8, 2020 at 6:34 AM Fabian Zimmermann > wrote: > >> Hi, >> >> we also have this issue. >> >> Our solution was (up to now) to delete the queues with a script or even >> reset the complete cluster. >> >> We just upgraded rabbitmq to the latest version - without luck. >> >> Anyone else seeing this issue? >> >> Fabian >> >> >> >> Arnaud Morin schrieb am Do., 6. Aug. 2020, >> 16:47: >> >>> Hey all, >>> >>> I would like to ask the community about a rabbit issue we have from time >>> to time. >>> >>> In our current architecture, we have a cluster of rabbits (3 nodes) for >>> all our OpenStack services (mostly nova and neutron). >>> >>> When one node of this cluster is down, the cluster continue working (we >>> use pause_minority strategy). >>> But, sometimes, the third server is not able to recover automatically >>> and need a manual intervention. >>> After this intervention, we restart the rabbitmq-server process, which >>> is then able to join the cluster back. >>> >>> At this time, the cluster looks ok, everything is fine. >>> BUT, nothing works. >>> Neutron and nova agents are not able to report back to servers. >>> They appear dead. >>> Servers seems not being able to consume messages. >>> The exchanges, queues, bindings seems good in rabbit. >>> >>> What we see is that removing bindings (using rabbitmqadmin delete >>> binding or the web interface) and recreate them again (using the same >>> routing key) brings the service back up and running. >>> >>> Doing this for all queues is really painful. Our next plan is to >>> automate it, but is there anyone in the community already saw this kind >>> of issues? >>> >>> Our bug looks like the one described in [1]. >>> Someone recommands to create an Alternate Exchange. >>> Is there anyone already tried that? >>> >>> FYI, we are running rabbit 3.8.2 (with OpenStack Stein). >>> We had the same kind of issues using older version of rabbit. >>> >>> Thanks for your help. >>> >>> [1] >>> https://groups.google.com/forum/#!newtopic/rabbitmq-users/rabbitmq-users/zFhmpHF2aWk >>> >>> -- >>> Arnaud Morin >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwm2012 at gmail.com Sun Aug 9 15:54:47 2020 From: pwm2012 at gmail.com (pwm) Date: Sun, 9 Aug 2020 23:54:47 +0800 Subject: DNS server instead of /etc/hosts file on Infra Server Message-ID: Hi, Anyone interested in replacing the /etc/hosts file entry with a DNS server on the openstack-ansible deployment? Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From monika.samal at outlook.com Sun Aug 9 11:59:20 2020 From: monika.samal at outlook.com (Monika Samal) Date: Sun, 9 Aug 2020 11:59:20 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , , Message-ID: ________________________________ From: Monika Samal Sent: Friday, August 7, 2020 4:41:52 AM To: Mark Goddard ; Michael Johnson Cc: Fabian Zimmermann ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer I tried following above document still facing same Octavia connection error with amphora image. Regards, Monika ________________________________ From: Mark Goddard Sent: Thursday, August 6, 2020 1:16:01 PM To: Michael Johnson Cc: Monika Samal ; Fabian Zimmermann ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer On Wed, 5 Aug 2020 at 16:16, Michael Johnson > wrote: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard > wrote: On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson > Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann > Cc: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From monika.samal at outlook.com Sun Aug 9 12:02:01 2020 From: monika.samal at outlook.com (Monika Samal) Date: Sun, 9 Aug 2020 12:02:01 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , , , Message-ID: Hi All, Below is the error am getting, i tried configuring network issue as well still finding it difficult to resolve. Below is my log...if somebody can help me resolving it..it would be great help since its very urgent... http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ Regards, Monika ________________________________ From: Monika Samal Sent: Sunday, 9 August, 2020, 5:29 pm To: Mark Goddard; Michael Johnson; openstack-discuss Cc: Fabian Zimmermann Subject: Re: [openstack-community] Octavia :; Unable to create load balancer ________________________________ From: Monika Samal Sent: Friday, August 7, 2020 4:41:52 AM To: Mark Goddard ; Michael Johnson Cc: Fabian Zimmermann ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer I tried following above document still facing same Octavia connection error with amphora image. Regards, Monika ________________________________ From: Mark Goddard Sent: Thursday, August 6, 2020 1:16:01 PM To: Michael Johnson Cc: Monika Samal ; Fabian Zimmermann ; openstack-discuss Subject: Re: [openstack-community] Octavia :; Unable to create load balancer On Wed, 5 Aug 2020 at 16:16, Michael Johnson > wrote: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard > wrote: On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson > Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann > Cc: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnsomor at gmail.com Mon Aug 10 05:44:29 2020 From: johnsomor at gmail.com (Michael Johnson) Date: Sun, 9 Aug 2020 22:44:29 -0700 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: That looks like there is still a kolla networking issue where the amphora are not able to reach the controller processes. Please fix the lb-mgmt-net such that it can reach the amphora and the controller containers. This should be setup via the deployment tool, kolla in this case. Michael On Sun, Aug 9, 2020 at 5:02 AM Monika Samal wrote: > Hi All, > > Below is the error am getting, i tried configuring network issue as well > still finding it difficult to resolve. > > Below is my log...if somebody can help me resolving it..it would be great > help since its very urgent... > > http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ > > Regards, > Monika > ------------------------------ > *From:* Monika Samal > *Sent:* Sunday, 9 August, 2020, 5:29 pm > *To:* Mark Goddard; Michael Johnson; openstack-discuss > *Cc:* Fabian Zimmermann > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > ------------------------------ > *From:* Monika Samal > *Sent:* Friday, August 7, 2020 4:41:52 AM > *To:* Mark Goddard ; Michael Johnson < > johnsomor at gmail.com> > *Cc:* Fabian Zimmermann ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > I tried following above document still facing same Octavia connection > error with amphora image. > > Regards, > Monika > ------------------------------ > *From:* Mark Goddard > *Sent:* Thursday, August 6, 2020 1:16:01 PM > *To:* Michael Johnson > *Cc:* Monika Samal ; Fabian Zimmermann < > dev.faz at gmail.com>; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > On Wed, 5 Aug 2020 at 16:16, Michael Johnson wrote: > > Looking at that error, it appears that the lb-mgmt-net is not setup > correctly. The Octavia controller containers are not able to reach the > amphora instances on the lb-mgmt-net subnet. > > I don't know how kolla is setup to connect the containers to the neutron > lb-mgmt-net network. Maybe the above documents will help with that. > > > Right now it's up to the operator to configure that. The kolla > documentation doesn't prescribe any particular setup. We're working on > automating it in Victoria. > > > Michael > > On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard wrote: > > > > On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: > > Hello Guys, > > With Michaels help I was able to solve the problem but now there is > another error I was able to create my network on vlan but still error > persist. PFB the logs: > > http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ > > Kindly help > > regards, > Monika > ------------------------------ > *From:* Michael Johnson > *Sent:* Monday, August 3, 2020 9:10 PM > *To:* Fabian Zimmermann > *Cc:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Yeah, it looks like nova is failing to boot the instance. > > Check this setting in your octavia.conf files: > https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id > > Also, if kolla-ansible didn't set both of these values correctly, please > open bug reports for kolla-ansible. These all should have been configured > by the deployment tool. > > > I wasn't following this thread due to no [kolla] tag, but here are the > recently added docs for Octavia in kolla [1]. Note > the octavia_service_auth_project variable which was added to migrate from > the admin project to the service project for octavia resources. We're > lacking proper automation for the flavor, image etc, but it is being worked > on in Victoria [2]. > > [1] > https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html > [2] https://review.opendev.org/740180 > > Michael > > On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: > > Seems like the flavor is missing or empty '' - check for typos and enable > debug. > > Check if the nova req contains valid information/flavor. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 15:46: > > It's registered > > Get Outlook for Android > ------------------------------ > *From:* Fabian Zimmermann > *Sent:* Monday, August 3, 2020 7:08:21 PM > *To:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Did you check the (nova) flavor you use in octavia. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 10:53: > > After Michael suggestion I was able to create load balancer but there is > error in status. > > > > PFB the error link: > > http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ > ------------------------------ > *From:* Monika Samal > *Sent:* Monday, August 3, 2020 2:08 PM > *To:* Michael Johnson > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Thanks a ton Michael for helping me out > ------------------------------ > *From:* Michael Johnson > *Sent:* Friday, July 31, 2020 3:57 AM > *To:* Monika Samal > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Just to close the loop on this, the octavia.conf file had > "project_name = admin" instead of "project_name = service" in the > [service_auth] section. This was causing the keystone errors when > Octavia was communicating with neutron. > > I don't know if that is a bug in kolla-ansible or was just a local > configuration issue. > > Michael > > On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > > > Hello Fabian,, > > > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > > > Regards, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > Hi, > > > > just to debug, could you replace the auth_type password with v3password? > > > > And do a curl against your :5000 and :35357 urls and paste the output. > > > > Fabian > > > > Monika Samal schrieb am Do., 30. Juli 2020, > 22:15: > > > > Hello Fabian, > > > > http://paste.openstack.org/show/796477/ > > > > Thanks, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > The sections should be > > > > service_auth > > keystone_authtoken > > > > if i read the docs correctly. Maybe you can just paste your config > (remove/change passwords) to paste.openstack.org and post the link? > > > > Fabian > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dev.faz at gmail.com Mon Aug 10 05:49:36 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Mon, 10 Aug 2020 07:49:36 +0200 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: Hi, to test your connection you can create an instance im the octavia network and try to ping/ssh from your controller (dont forget a suitable security group) Fabian Michael Johnson schrieb am Mo., 10. Aug. 2020, 07:44: > > That looks like there is still a kolla networking issue where the amphora > are not able to reach the controller processes. Please fix the lb-mgmt-net > such that it can reach the amphora and the controller containers. This > should be setup via the deployment tool, kolla in this case. > > Michael > > On Sun, Aug 9, 2020 at 5:02 AM Monika Samal > wrote: > >> Hi All, >> >> Below is the error am getting, i tried configuring network issue as well >> still finding it difficult to resolve. >> >> Below is my log...if somebody can help me resolving it..it would be great >> help since its very urgent... >> >> http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ >> >> Regards, >> Monika >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Sunday, 9 August, 2020, 5:29 pm >> *To:* Mark Goddard; Michael Johnson; openstack-discuss >> *Cc:* Fabian Zimmermann >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Friday, August 7, 2020 4:41:52 AM >> *To:* Mark Goddard ; Michael Johnson < >> johnsomor at gmail.com> >> *Cc:* Fabian Zimmermann ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> I tried following above document still facing same Octavia connection >> error with amphora image. >> >> Regards, >> Monika >> ------------------------------ >> *From:* Mark Goddard >> *Sent:* Thursday, August 6, 2020 1:16:01 PM >> *To:* Michael Johnson >> *Cc:* Monika Samal ; Fabian Zimmermann < >> dev.faz at gmail.com>; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> >> >> On Wed, 5 Aug 2020 at 16:16, Michael Johnson wrote: >> >> Looking at that error, it appears that the lb-mgmt-net is not setup >> correctly. The Octavia controller containers are not able to reach the >> amphora instances on the lb-mgmt-net subnet. >> >> I don't know how kolla is setup to connect the containers to the neutron >> lb-mgmt-net network. Maybe the above documents will help with that. >> >> >> Right now it's up to the operator to configure that. The kolla >> documentation doesn't prescribe any particular setup. We're working on >> automating it in Victoria. >> >> >> Michael >> >> On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard wrote: >> >> >> >> On Tue, 4 Aug 2020 at 16:58, Monika Samal >> wrote: >> >> Hello Guys, >> >> With Michaels help I was able to solve the problem but now there is >> another error I was able to create my network on vlan but still error >> persist. PFB the logs: >> >> http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ >> >> Kindly help >> >> regards, >> Monika >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Monday, August 3, 2020 9:10 PM >> *To:* Fabian Zimmermann >> *Cc:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Yeah, it looks like nova is failing to boot the instance. >> >> Check this setting in your octavia.conf files: >> https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id >> >> Also, if kolla-ansible didn't set both of these values correctly, please >> open bug reports for kolla-ansible. These all should have been configured >> by the deployment tool. >> >> >> I wasn't following this thread due to no [kolla] tag, but here are the >> recently added docs for Octavia in kolla [1]. Note >> the octavia_service_auth_project variable which was added to migrate from >> the admin project to the service project for octavia resources. We're >> lacking proper automation for the flavor, image etc, but it is being worked >> on in Victoria [2]. >> >> [1] >> https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html >> [2] https://review.opendev.org/740180 >> >> Michael >> >> On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann >> wrote: >> >> Seems like the flavor is missing or empty '' - check for typos and enable >> debug. >> >> Check if the nova req contains valid information/flavor. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 15:46: >> >> It's registered >> >> Get Outlook for Android >> ------------------------------ >> *From:* Fabian Zimmermann >> *Sent:* Monday, August 3, 2020 7:08:21 PM >> *To:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Did you check the (nova) flavor you use in octavia. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 10:53: >> >> After Michael suggestion I was able to create load balancer but there is >> error in status. >> >> >> >> PFB the error link: >> >> http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Monday, August 3, 2020 2:08 PM >> *To:* Michael Johnson >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Thanks a ton Michael for helping me out >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Friday, July 31, 2020 3:57 AM >> *To:* Monika Samal >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Just to close the loop on this, the octavia.conf file had >> "project_name = admin" instead of "project_name = service" in the >> [service_auth] section. This was causing the keystone errors when >> Octavia was communicating with neutron. >> >> I don't know if that is a bug in kolla-ansible or was just a local >> configuration issue. >> >> Michael >> >> On Thu, Jul 30, 2020 at 1:39 PM Monika Samal >> wrote: >> > >> > Hello Fabian,, >> > >> > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ >> > >> > Regards, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:57 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > Hi, >> > >> > just to debug, could you replace the auth_type password with v3password? >> > >> > And do a curl against your :5000 and :35357 urls and paste the output. >> > >> > Fabian >> > >> > Monika Samal schrieb am Do., 30. Juli 2020, >> 22:15: >> > >> > Hello Fabian, >> > >> > http://paste.openstack.org/show/796477/ >> > >> > Thanks, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:38 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > The sections should be >> > >> > service_auth >> > keystone_authtoken >> > >> > if i read the docs correctly. Maybe you can just paste your config >> (remove/change passwords) to paste.openstack.org and post the link? >> > >> > Fabian >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From marino.mrc at gmail.com Mon Aug 10 07:49:28 2020 From: marino.mrc at gmail.com (Marco Marino) Date: Mon, 10 Aug 2020 09:49:28 +0200 Subject: [ironic][tripleo][ussuri] Problem with bare metal provisioning and old RAID controllers In-Reply-To: References: Message-ID: Hi, I'm sorry if I reopen this thread, but I cannot find a solution at the moment. Please, can someone give me some hint on how to detect megaraid controllers with IPA? I think this could be useful for many users. PS: I can do any test, I have a 6 servers test environment (5 nodes + undercloud) with megaraid controllers (Poweredge R620) Thank you Il giorno mar 4 ago 2020 alle ore 12:57 Marco Marino ha scritto: > Here is what I did: > # /usr/lib/dracut/skipcpio > /home/stack/images/ironic-python-agent.initramfs | zcat | cpio -ivd | pax -r > # mount dd-megaraid_sas-07.710.50.00-1.el8_2.elrepo.iso /mnt/ > # rpm2cpio > /mnt/rpms/x86_64/kmod-megaraid_sas-07.710.50.00-1.el8_2.elrepo.x86_64.rpm | > pax -r > # find . 2>/dev/null | cpio --quiet -c -o | gzip -8 > > /home/stack/images/ironic-python-agent.initramfs > # chown stack: /home/stack/images/ironic-python-agent.initramfs > (undercloud) [stack at undercloud ~]$ openstack overcloud image upload > --update-existing --image-path /home/stack/images/ > > At this point I checked that agent.ramdisk in /var/lib/ironic/httpboot has > an update timestamp > > Then > (undercloud) [stack at undercloud ~]$ openstack overcloud node introspect > --provide controller2 > /usr/lib64/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't > resolve package from __spec__ or __package__, falling back on __name__ and > __path__ > return f(*args, **kwds) > > PLAY [Baremetal Introspection for multiple Ironic Nodes] > *********************** > 2020-08-04 12:32:26.684368 | ecf4bbd2-e605-20dd-3da9-000000000008 | > TASK | Check for required inputs > 2020-08-04 12:32:26.739797 | ecf4bbd2-e605-20dd-3da9-000000000008 | > SKIPPED | Check for required inputs | localhost | item=node_uuids > 2020-08-04 12:32:26.746684 | ecf4bbd2-e605-20dd-3da9-00000000000a | > TASK | Set node_uuids_intro fact > [WARNING]: Failure using method (v2_playbook_on_task_start) in callback > plugin > ( 0x7f1b0f9bce80>): maximum recursion depth exceeded while calling a Python > object > 2020-08-04 12:32:26.828985 | ecf4bbd2-e605-20dd-3da9-00000000000a | > OK | Set node_uuids_intro fact | localhost > 2020-08-04 12:32:26.834281 | ecf4bbd2-e605-20dd-3da9-00000000000c | > TASK | Notice > 2020-08-04 12:32:26.911106 | ecf4bbd2-e605-20dd-3da9-00000000000c | > SKIPPED | Notice | localhost > 2020-08-04 12:32:26.916344 | ecf4bbd2-e605-20dd-3da9-00000000000e | > TASK | Set concurrency fact > 2020-08-04 12:32:26.994087 | ecf4bbd2-e605-20dd-3da9-00000000000e | > OK | Set concurrency fact | localhost > 2020-08-04 12:32:27.005932 | ecf4bbd2-e605-20dd-3da9-000000000010 | > TASK | Check if validation enabled > 2020-08-04 12:32:27.116425 | ecf4bbd2-e605-20dd-3da9-000000000010 | > SKIPPED | Check if validation enabled | localhost > 2020-08-04 12:32:27.129120 | ecf4bbd2-e605-20dd-3da9-000000000011 | > TASK | Run Validations > 2020-08-04 12:32:27.239850 | ecf4bbd2-e605-20dd-3da9-000000000011 | > SKIPPED | Run Validations | localhost > 2020-08-04 12:32:27.251796 | ecf4bbd2-e605-20dd-3da9-000000000012 | > TASK | Fail if validations are disabled > 2020-08-04 12:32:27.362050 | ecf4bbd2-e605-20dd-3da9-000000000012 | > SKIPPED | Fail if validations are disabled | localhost > 2020-08-04 12:32:27.373947 | ecf4bbd2-e605-20dd-3da9-000000000014 | > TASK | Start baremetal introspection > > > 2020-08-04 12:48:19.944028 | ecf4bbd2-e605-20dd-3da9-000000000014 | > CHANGED | Start baremetal introspection | localhost > 2020-08-04 12:48:19.966517 | ecf4bbd2-e605-20dd-3da9-000000000015 | > TASK | Nodes that passed introspection > 2020-08-04 12:48:20.130913 | ecf4bbd2-e605-20dd-3da9-000000000015 | > OK | Nodes that passed introspection | localhost | result={ > "changed": false, > "msg": " 00c5e81b-1e5d-442b-b64f-597a604051f7" > } > 2020-08-04 12:48:20.142919 | ecf4bbd2-e605-20dd-3da9-000000000016 | > TASK | Nodes that failed introspection > 2020-08-04 12:48:20.305004 | ecf4bbd2-e605-20dd-3da9-000000000016 | > OK | Nodes that failed introspection | localhost | result={ > "changed": false, > "failed_when_result": false, > "msg": " All nodes completed introspection successfully!" > } > 2020-08-04 12:48:20.316860 | ecf4bbd2-e605-20dd-3da9-000000000017 | > TASK | Node introspection failed and no results are provided > 2020-08-04 12:48:20.427675 | ecf4bbd2-e605-20dd-3da9-000000000017 | > SKIPPED | Node introspection failed and no results are provided | localhost > > PLAY RECAP > ********************************************************************* > localhost : ok=5 changed=1 unreachable=0 > failed=0 skipped=6 rescued=0 ignored=0 > [WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin > ( 0x7f1b0f9bce80>): _output() missing 1 required positional argument: 'color' > Successfully introspected nodes: ['controller2'] > Exception occured while running the command > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", > line 340, in prepare_command > cmdline_args = self.loader.load_file('args', string_types, > encoding=None) > File "/usr/lib/python3.6/site-packages/ansible_runner/loader.py", line > 164, in load_file > contents = parsed_data = self.get_contents(path) > File "/usr/lib/python3.6/site-packages/ansible_runner/loader.py", line > 98, in get_contents > raise ConfigurationError('specified path does not exist %s' % path) > ansible_runner.exceptions.ConfigurationError: specified path does not > exist /tmp/tripleop89yr8i8/args > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line > 34, in run > super(Command, self).run(parsed_args) > File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line > 41, in run > return super(Command, self).run(parsed_args) > File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in > run > return_code = self.take_action(parsed_args) or 0 > File > "/usr/lib/python3.6/site-packages/tripleoclient/v2/overcloud_node.py", line > 210, in take_action > node_uuids=parsed_args.node_uuids, > File > "/usr/lib/python3.6/site-packages/tripleoclient/workflows/baremetal.py", > line 134, in provide > 'node_uuids': node_uuids > File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line > 659, in run_ansible_playbook > runner_config.prepare() > File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", > line 174, in prepare > self.prepare_command() > File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", > line 346, in prepare_command > self.command = self.generate_ansible_command() > File "/usr/lib/python3.6/site-packages/ansible_runner/runner_config.py", > line 415, in generate_ansible_command > v = 'v' * self.verbosity > TypeError: can't multiply sequence by non-int of type 'ClientManager' > can't multiply sequence by non-int of type 'ClientManager' > (undercloud) [stack at undercloud ~]$ > > > and > (undercloud) [stack at undercloud ~]$ openstack baremetal node show > controller2 > .... > | properties | {'local_gb': '0', 'cpus': '24', 'cpu_arch': > 'x86_64', 'memory_mb': '32768', 'capabilities': > 'cpu_vt:true,cpu_aes:true,cpu_hugepages:true,cpu_hugepages_1g:true,cpu_txt:true'} > > > It seems that megaraid driver is correctly inserted in ramdisk: > # lsinitrd /var/lib/ironic/httpboot/agent.ramdisk | grep megaraid > /bin/lsinitrd: line 276: warning: command substitution: ignored null byte > in input > -rw-r--r-- 1 root root 50 Apr 28 21:55 > etc/depmod.d/kmod-megaraid_sas.conf > drwxr-xr-x 2 root root 0 Aug 4 12:13 > usr/lib/modules/4.18.0-193.6.3.el8_2.x86_64/kernel/drivers/scsi/megaraid > -rw-r--r-- 1 root root 68240 Aug 4 12:13 > usr/lib/modules/4.18.0-193.6.3.el8_2.x86_64/kernel/drivers/scsi/megaraid/megaraid_sas.ko.xz > drwxr-xr-x 2 root root 0 Apr 28 21:55 > usr/lib/modules/4.18.0-193.el8.x86_64/extra/megaraid_sas > -rw-r--r-- 1 root root 309505 Apr 28 21:55 > usr/lib/modules/4.18.0-193.el8.x86_64/extra/megaraid_sas/megaraid_sas.ko > drwxr-xr-x 2 root root 0 Apr 28 21:55 > usr/share/doc/kmod-megaraid_sas-07.710.50.00 > -rw-r--r-- 1 root root 18092 Apr 28 21:55 > usr/share/doc/kmod-megaraid_sas-07.710.50.00/GPL-v2.0.txt > -rw-r--r-- 1 root root 1152 Apr 28 21:55 > usr/share/doc/kmod-megaraid_sas-07.710.50.00/greylist.txt > > If the solution is to use a Centos7 ramdisk, please can you give me some > hint? I have no idea on how to build a new ramdisk from scratch > Thank you > > > > > > > > > Il giorno mar 4 ago 2020 alle ore 12:33 Dmitry Tantsur < > dtantsur at redhat.com> ha scritto: > >> Hi, >> >> On Tue, Aug 4, 2020 at 11:58 AM Marco Marino >> wrote: >> >>> Hi, I'm trying to install openstack Ussuri on Centos 8 hardware using >>> tripleo. I'm using a relatively old hardware (dell PowerEdge R620) with old >>> RAID controllers, deprecated in RHEL8/Centos8. Here is some basic >>> information: >>> # lspci | grep -i raid >>> 00:1f.2 RAID bus controller: Intel Corporation C600/X79 series chipset >>> SATA RAID Controller (rev 05) >>> 02:00.0 RAID bus controller: Broadcom / LSI MegaRAID SAS 2008 [Falcon] >>> (rev 03) >>> >>> I'm able to manually install centos 8 using DUD driver from here -> >>> https://elrepo.org/linux/dud/el8/x86_64/dd-megaraid_sas-07.710.50.00-1.el8_2.elrepo.iso >>> (basically I add inst.dd and I use an usb pendrive with iso). >>> Is there a way to do bare metal provisioning using openstack on this >>> kind of server? At the moment, when I launch "openstack overcloud node >>> introspect --provide controller1" it doesn't recognize disks (local_gb = 0 >>> in properties) and in inspector logs I see: >>> Jun 22 11:12:42 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:42.261 1543 DEBUG root [-] Still waiting for the root >>> device to appear, attempt 1 of 10 wait_for_disks >>> /usr/lib/python3.6/site-packages/ironic_python_agent/hardware.py:652 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.299 1543 DEBUG oslo_concurrency.processutils [-] >>> Running cmd (subprocess): udevadm settle execute >>> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.357 1543 DEBUG oslo_concurrency.processutils [-] CMD >>> "udevadm settle" returned: 0 in 0.058s execute >>> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.392 1543 DEBUG ironic_lib.utils [-] Execution >>> completed, command line is "udevadm settle" execute >>> /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.426 1543 DEBUG ironic_lib.utils [-] Command stdout is: >>> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.460 1543 DEBUG ironic_lib.utils [-] Command stderr is: >>> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.496 1543 WARNING root [-] Path /dev/disk/by-path is >>> inaccessible, /dev/disk/by-path/* version of block device name is >>> unavailable Cause: [Errno 2] No such file or directory: >>> '/dev/disk/by-path': FileNotFoundError: [Errno 2] No such file or >>> directory: '/dev/disk/by-path' >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.549 1543 DEBUG oslo_concurrency.processutils [-] >>> Running cmd (subprocess): lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE execute >>> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:372 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.647 1543 DEBUG oslo_concurrency.processutils [-] CMD >>> "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" returned: 0 in 0.097s execute >>> /usr/lib/python3.6/site-packages/oslo_concurrency/processutils.py:409 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.683 1543 DEBUG ironic_lib.utils [-] Execution >>> completed, command line is "lsblk -Pbia -oKNAME,MODEL,SIZE,ROTA,TYPE" >>> execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:101 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.719 1543 DEBUG ironic_lib.utils [-] Command stdout is: >>> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:103 >>> Jun 22 11:12:45 localhost.localdomain ironic-python-agent[1543]: >>> 2018-06-22 11:12:45.755 1543 DEBUG ironic_lib.utils [-] Command stderr is: >>> "" execute /usr/lib/python3.6/site-packages/ironic_lib/utils.py:104 >>> >>> Is there a way to solve the issue? For example, can I modify ramdisk and >>> include DUD driver? I tried this guide: >>> https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/partner_integration/overcloud_images#initrd_modifying_the_initial_ramdisks >>> >>> but I don't know how to include an ISO instead of an rpm packet as >>> described in the example. >>> >> >> Indeed, I don't think you can use ISO as it is, you'll need to figure out >> what is inside. If it's an RPM (as I assume), you'll need to extract it and >> install into the ramdisk. >> >> If nothing helps, you can try building a ramdisk with CentOS 7, the >> (very) recent versions of ironic-python-agent-builder allow using Python 3 >> on CentOS 7. >> >> Dmitry >> >> >>> Thank you, >>> Marco >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Mon Aug 10 08:07:04 2020 From: mark at stackhpc.com (Mark Goddard) Date: Mon, 10 Aug 2020 09:07:04 +0100 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: On Mon, 10 Aug 2020 at 06:44, Michael Johnson wrote: > > That looks like there is still a kolla networking issue where the amphora > are not able to reach the controller processes. Please fix the lb-mgmt-net > such that it can reach the amphora and the controller containers. This > should be setup via the deployment tool, kolla in this case. > As mentioned before, Kolla doesn't currently do this - it is up to the user. We're improving the integration in the Victoria cycle. > Michael > > On Sun, Aug 9, 2020 at 5:02 AM Monika Samal > wrote: > >> Hi All, >> >> Below is the error am getting, i tried configuring network issue as well >> still finding it difficult to resolve. >> >> Below is my log...if somebody can help me resolving it..it would be great >> help since its very urgent... >> >> http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ >> >> Regards, >> Monika >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Sunday, 9 August, 2020, 5:29 pm >> *To:* Mark Goddard; Michael Johnson; openstack-discuss >> *Cc:* Fabian Zimmermann >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Friday, August 7, 2020 4:41:52 AM >> *To:* Mark Goddard ; Michael Johnson < >> johnsomor at gmail.com> >> *Cc:* Fabian Zimmermann ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> I tried following above document still facing same Octavia connection >> error with amphora image. >> >> Regards, >> Monika >> ------------------------------ >> *From:* Mark Goddard >> *Sent:* Thursday, August 6, 2020 1:16:01 PM >> *To:* Michael Johnson >> *Cc:* Monika Samal ; Fabian Zimmermann < >> dev.faz at gmail.com>; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> >> >> On Wed, 5 Aug 2020 at 16:16, Michael Johnson wrote: >> >> Looking at that error, it appears that the lb-mgmt-net is not setup >> correctly. The Octavia controller containers are not able to reach the >> amphora instances on the lb-mgmt-net subnet. >> >> I don't know how kolla is setup to connect the containers to the neutron >> lb-mgmt-net network. Maybe the above documents will help with that. >> >> >> Right now it's up to the operator to configure that. The kolla >> documentation doesn't prescribe any particular setup. We're working on >> automating it in Victoria. >> >> >> Michael >> >> On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard wrote: >> >> >> >> On Tue, 4 Aug 2020 at 16:58, Monika Samal >> wrote: >> >> Hello Guys, >> >> With Michaels help I was able to solve the problem but now there is >> another error I was able to create my network on vlan but still error >> persist. PFB the logs: >> >> http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ >> >> Kindly help >> >> regards, >> Monika >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Monday, August 3, 2020 9:10 PM >> *To:* Fabian Zimmermann >> *Cc:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Yeah, it looks like nova is failing to boot the instance. >> >> Check this setting in your octavia.conf files: >> https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id >> >> Also, if kolla-ansible didn't set both of these values correctly, please >> open bug reports for kolla-ansible. These all should have been configured >> by the deployment tool. >> >> >> I wasn't following this thread due to no [kolla] tag, but here are the >> recently added docs for Octavia in kolla [1]. Note >> the octavia_service_auth_project variable which was added to migrate from >> the admin project to the service project for octavia resources. We're >> lacking proper automation for the flavor, image etc, but it is being worked >> on in Victoria [2]. >> >> [1] >> https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html >> [2] https://review.opendev.org/740180 >> >> Michael >> >> On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann >> wrote: >> >> Seems like the flavor is missing or empty '' - check for typos and enable >> debug. >> >> Check if the nova req contains valid information/flavor. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 15:46: >> >> It's registered >> >> Get Outlook for Android >> ------------------------------ >> *From:* Fabian Zimmermann >> *Sent:* Monday, August 3, 2020 7:08:21 PM >> *To:* Monika Samal ; openstack-discuss < >> openstack-discuss at lists.openstack.org> >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Did you check the (nova) flavor you use in octavia. >> >> Fabian >> >> Monika Samal schrieb am Mo., 3. Aug. 2020, >> 10:53: >> >> After Michael suggestion I was able to create load balancer but there is >> error in status. >> >> >> >> PFB the error link: >> >> http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ >> ------------------------------ >> *From:* Monika Samal >> *Sent:* Monday, August 3, 2020 2:08 PM >> *To:* Michael Johnson >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Thanks a ton Michael for helping me out >> ------------------------------ >> *From:* Michael Johnson >> *Sent:* Friday, July 31, 2020 3:57 AM >> *To:* Monika Samal >> *Cc:* Fabian Zimmermann ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> *Subject:* Re: [openstack-community] Octavia :; Unable to create load >> balancer >> >> Just to close the loop on this, the octavia.conf file had >> "project_name = admin" instead of "project_name = service" in the >> [service_auth] section. This was causing the keystone errors when >> Octavia was communicating with neutron. >> >> I don't know if that is a bug in kolla-ansible or was just a local >> configuration issue. >> >> Michael >> >> On Thu, Jul 30, 2020 at 1:39 PM Monika Samal >> wrote: >> > >> > Hello Fabian,, >> > >> > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ >> > >> > Regards, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:57 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > Hi, >> > >> > just to debug, could you replace the auth_type password with v3password? >> > >> > And do a curl against your :5000 and :35357 urls and paste the output. >> > >> > Fabian >> > >> > Monika Samal schrieb am Do., 30. Juli 2020, >> 22:15: >> > >> > Hello Fabian, >> > >> > http://paste.openstack.org/show/796477/ >> > >> > Thanks, >> > Monika >> > ________________________________ >> > From: Fabian Zimmermann >> > Sent: Friday, July 31, 2020 1:38 AM >> > To: Monika Samal >> > Cc: Michael Johnson ; Amy Marrich ; >> openstack-discuss ; >> community at lists.openstack.org >> > Subject: Re: [openstack-community] Octavia :; Unable to create load >> balancer >> > >> > The sections should be >> > >> > service_auth >> > keystone_authtoken >> > >> > if i read the docs correctly. Maybe you can just paste your config >> (remove/change passwords) to paste.openstack.org and post the link? >> > >> > Fabian >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From moreira.belmiro.email.lists at gmail.com Mon Aug 10 08:13:24 2020 From: moreira.belmiro.email.lists at gmail.com (Belmiro Moreira) Date: Mon, 10 Aug 2020 10:13:24 +0200 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients Message-ID: Hi, during the last PTG the TC discussed the problem of supporting different clients (OpenStack Client - OSC vs python-*clients) [1]. Currently, we don't have feature parity between the OSC and the python-*clients. Different OpenStack projects invest in different clients. This can be a huge problem for users/ops. Depending on the projects deployed in their infrastructures, they need to use different clients for different tasks. It's confusing because of the partial implementation in the OSC. There was also the proposal to enforce new functionality only in the SDK (and optionally the OSC) and not the project’s specific clients to stop increasing the disparity between the two. We would like to understand first the problems and missing pieces that projects are facing to move into OSC and help to overcome them. Let us know. Belmiro, on behalf of the TC [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015418.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From dikonoor at in.ibm.com Mon Aug 10 08:16:57 2020 From: dikonoor at in.ibm.com (Divya K Konoor) Date: Mon, 10 Aug 2020 13:46:57 +0530 Subject: [openstack-community] Keystone and DBNonExistent Errors In-Reply-To: References: Message-ID: Hi, I am using OpenStack Keystone Stein and run into the below error often where Keystone public process(listening to 5000) is running inside Apache httpd runs into the below. This problem is resolved with a restart of httpd service. Has anyone run into a similar issue ? This is seen soon after httpd is restarted and does not happen all the time. My environment has MariaDB backend. This problem is not limited to the assignment table and is seen across all other tables in Keystone. MariaDB service is functional and all the tables are in place. [Fri Aug 07 08:20:59.936087 2020] [:info] [pid 1420287] mod_wsgi (pid=1420287, process='keystone-public', application=''): Loading WSGI script '/usr/bin/keystone-wsgi-public'. [Fri Aug 07 08:20:59.936089 2020] [:info] [pid 1420288] mod_wsgi (pid=1420288, process='keystone-admin', application=''): Loading WSGI script '/usr/bin/keystone-wsgi-admin'. [Fri Aug 07 08:20:59.943431 2020] [ssl:info] [pid 1420290] [client 1.2.3.95:35762] AH01964: Connection to child 1 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:00.009317 2020] [ssl:info] [pid 1420291] [client 1.2.3.113:60132] AH01964: Connection to child 2 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:01.243594 2020] [ssl:info] [pid 1420289] [client 1.2.3.50:53996] AH01964: Connection to child 0 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:01.386329 2020] [ssl:info] [pid 1420293] [client x.x.x.x:38645] AH01964: Connection to child 4 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:01.824041 2020] [ssl:info] [pid 1420349] [client 1.2.3.101:42974] AH01964: Connection to child 5 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:02.949166 2020] [ssl:info] [pid 1420378] [client 1.2.3.50:54014] AH01964: Connection to child 9 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:02.949172 2020] [ssl:info] [pid 1420379] [client 1.2.3.80:46924] AH01964: Connection to child 10 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:03.286057 2020] [:info] [pid 1420287] mod_wsgi (pid=1420287): Create interpreter '1.2.3.50:5000|'. [Fri Aug 07 08:21:03.287286 2020] [:info] [pid 1420287] [remote 1.2.3.95:156] mod_wsgi (pid=1420287, process='keystone-public', application='1.2.3.50:5000|'): Loading WSGI script '/usr/bin/keystone-wsgi-public'. [Fri Aug 07 08:21:04.675059 2020] [ssl:info] [pid 1420436] [client 1.2.3.50:54032] AH01964: Connection to child 12 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:04.705975 2020] [ssl:info] [pid 1420437] [client 1.2.3.107:59554] AH01964: Connection to child 13 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:06.960940 2020] [ssl:info] [pid 1420438] [client 1.2.3.80:46970] AH01964: Connection to child 14 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:07.661670 2020] [ssl:info] [pid 1420349] [client 1.2.3.50:54124] AH01964: Connection to child 5 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:07.683383 2020] [ssl:info] [pid 1420292] [client x.x.x.x:30065] AH01964: Connection to child 3 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:08.442956 2020] [:error] [pid 1420287] [remote x.x.x.x:144] mod_wsgi (pid=1420287): Exception occurred processing WSGI script '/usr/bin/keystone-wsgi-public'. [Fri Aug 07 08:21:08.443002 2020] [:error] [pid 1420287] [remote x.x.x.x:144] Traceback (most recent call last): [Fri Aug 07 08:21:08.443017 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__ [Fri Aug 07 08:21:08.443509 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.wsgi_app(environ, start_response) [Fri Aug 07 08:21:08.443525 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/werkzeug/contrib/fixers.py", line 152, in __call__ [Fri Aug 07 08:21:08.443630 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.app(environ, start_response) [Fri Aug 07 08:21:08.443644 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.443746 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.443756 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.443773 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.443781 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__ [Fri Aug 07 08:21:08.443844 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = req.get_response(self.application) .. ... .... .. [Fri Aug 07 08:21:08.450055 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1216, in creator [Fri Aug 07 08:21:08.450071 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return fn(*arg, **kw) [Fri Aug 07 08:21:08.450080 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 128, in get_roles_for_user_and_project [Fri Aug 07 08:21:08.450975 2020] [:error] [pid 1420287] [remote x.x.x.x:144] user_id=user_id, project_id=project_id, effective=True) [Fri Aug 07 08:21:08.450985 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 116, in wrapped [Fri Aug 07 08:21:08.451001 2020] [:error] [pid 1420287] [remote x.x.x.x:144] __ret_val = __f(*args, **kwargs) [Fri Aug 07 08:21:08.451009 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 999, in list_role_assignments [Fri Aug 07 08:21:08.451025 2020] [:error] [pid 1420287] [remote x.x.x.x:144] strip_domain_roles) [Fri Aug 07 08:21:08.451033 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 845, in _list_effective_role_assignments [Fri Aug 07 08:21:08.451049 2020] [:error] [pid 1420287] [remote x.x.x.x:144] domain_id=domain_id, inherited=inherited) [Fri Aug 07 08:21:08.451057 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 780, in list_role_assignments_for_actor [Fri Aug 07 08:21:08.451072 2020] [:error] [pid 1420287] [remote x.x.x.x:144] group_ids=group_ids, inherited_to_projects=False) [Fri Aug 07 08:21:08.451081 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/backends/sql.py", line 248, in list_role_assignments [Fri Aug 07 08:21:08.451599 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return [denormalize_role(ref) for ref in query.all()] [Fri Aug 07 08:21:08.451609 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2925, in all [Fri Aug 07 08:21:08.451632 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return list(self) [Fri Aug 07 08:21:08.451641 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3081, in __iter__ [Fri Aug 07 08:21:08.451656 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self._execute_and_instances(context) [Fri Aug 07 08:21:08.451665 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3106, in _execute_and_instances [Fri Aug 07 08:21:08.451683 2020] [:error] [pid 1420287] [remote x.x.x.x:144] result = conn.execute(querycontext.statement, self._params) [Fri Aug 07 08:21:08.451691 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute [Fri Aug 07 08:21:08.451711 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return meth(self, multiparams, params) [Fri Aug 07 08:21:08.451720 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection [Fri Aug 07 08:21:08.451736 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return connection._execute_clauseelement(self, multiparams, params) [Fri Aug 07 08:21:08.451745 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement [Fri Aug 07 08:21:08.451762 2020] [:error] [pid 1420287] [remote x.x.x.x:144] distilled_params, [Fri Aug 07 08:21:08.451771 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context [Fri Aug 07 08:21:08.451786 2020] [:error] [pid 1420287] [remote x.x.x.x:144] e, statement, parameters, cursor, context [Fri Aug 07 08:21:08.451795 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception [Fri Aug 07 08:21:08.451810 2020] [:error] [pid 1420287] [remote x.x.x.x:144] util.raise_from_cause(newraise, exc_info) [Fri Aug 07 08:21:08.451818 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause [Fri Aug 07 08:21:08.451834 2020] [:error] [pid 1420287] [remote x.x.x.x:144] reraise(type(exception), exception, tb=exc_tb, cause=cause) [Fri Aug 07 08:21:08.451843 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context [Fri Aug 07 08:21:08.451858 2020] [:error] [pid 1420287] [remote x.x.x.x:144] cursor, statement, parameters, context [Fri Aug 07 08:21:08.451866 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute [Fri Aug 07 08:21:08.451882 2020] [:error] [pid 1420287] [remote x.x.x.x:144] cursor.execute(statement, parameters) [Fri Aug 07 08:21:08.451923 2020] [:error] [pid 1420287] [remote x.x.x.x:144] DBNonExistentTable: (sqlite3.OperationalError) no such table: assignment [SQL: u'SELECT assignment.type AS assignment_type, assignment.actor_id AS assignment_actor_id, assignment.target_id AS assignment_target_id, assignment.role_id AS assignment_role_id, assignment.inherited AS assignment_inherited \\nFROM assignment \\nWHERE assignment.actor_id IN (?) AND assignment.target_id IN (?) AND assignment.type IN (?) AND assignment.inherited = 0'] [parameters: ('15c2fe91e053af57a997c568c117c908d59c138f996bdc19ae97e9f16df12345', '12345978536e45ab8a279e2b0fa4f947', 'UserProject')] (Background on this error at: http://sqlalche.me/e/e3q8) Regards, Divya -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Mon Aug 10 08:26:24 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 10 Aug 2020 10:26:24 +0200 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: References: Message-ID: On Mon, Aug 10, 2020 at 10:19 AM Belmiro Moreira < moreira.belmiro.email.lists at gmail.com> wrote: > Hi, > during the last PTG the TC discussed the problem of supporting different > clients (OpenStack Client - OSC vs python-*clients) [1]. > Currently, we don't have feature parity between the OSC and the > python-*clients. > Is it true of any client? I guess some are just OSC plugins 100%. Do we know which clients have this disparity? Personally, I encountered this with Glance the most and Cinder to some extent (but I believe over the course of action Cinder got all features I wanted from it in the OSC). -yoctozepto -------------- next part -------------- An HTML attachment was scrubbed... URL: From ltoscano at redhat.com Mon Aug 10 08:37:22 2020 From: ltoscano at redhat.com (Luigi Toscano) Date: Mon, 10 Aug 2020 10:37:22 +0200 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: References: Message-ID: <1668118.VLH7GnMWUR@whitebase.usersys.redhat.com> On Monday, 10 August 2020 10:26:24 CEST Radosław Piliszek wrote: > On Mon, Aug 10, 2020 at 10:19 AM Belmiro Moreira < > > moreira.belmiro.email.lists at gmail.com> wrote: > > Hi, > > during the last PTG the TC discussed the problem of supporting different > > clients (OpenStack Client - OSC vs python-*clients) [1]. > > Currently, we don't have feature parity between the OSC and the > > python-*clients. > > Is it true of any client? I guess some are just OSC plugins 100%. > Do we know which clients have this disparity? > Personally, I encountered this with Glance the most and Cinder to some > extent (but I believe over the course of action Cinder got all features I > wanted from it in the OSC). As far as I know there is still a huge problem with microversion handling which impacts some cinder features. It has been discussed in the past and still present. -- Luigi From thierry at openstack.org Mon Aug 10 10:01:37 2020 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 10 Aug 2020 12:01:37 +0200 Subject: [largescale-sig] Next meeting: August 12, 16utc Message-ID: <6e7a4e43-08f4-3030-2eb0-9311f27d9647@openstack.org> Hi everyone, In order to accommodate US members, the Large Scale SIG recently decided to rotate between an EU-APAC-friendly time and an US-EU-friendly time. Our next meeting will be the first US-EU meeting, on Wednesday, August 12 at 16 UTC[1] in the #openstack-meeting-3 channel on IRC: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200812T16 Feel free to add topics to our agenda at: https://etherpad.openstack.org/p/large-scale-sig-meeting A reminder of the TODOs we had from last meeting, in case you have time to make progress on them: - amorin to add some meat to the wiki page before we push the Nova doc patch further - all to describe briefly how you solved metrics/billing in your deployment in https://etherpad.openstack.org/p/large-scale-sig-documentation Talk to you all on Wednesday, -- Thierry Carrez From emilien at redhat.com Mon Aug 10 12:29:22 2020 From: emilien at redhat.com (Emilien Macchi) Date: Mon, 10 Aug 2020 08:29:22 -0400 Subject: [puppet][congress] Retiring puppet-congress In-Reply-To: References: Message-ID: On Sat, Jun 20, 2020 at 12:44 PM Takashi Kajinami wrote: > Hello, > > > As you know, Congress project has been retired already[1], > so we will retire its puppet module, puppet-congress in > openstack puppet project as well. > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014292.html > > Because congress was directly retired instead of getting migrated > to x namespace, we'll follow the same way about puppet-congress retirement > and won't create x/puppet-congress. > > Thank you for the contribution made for the project ! > Please let us know if you have any concerns about this retirement. > +2 -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From ildiko.vancsa at gmail.com Mon Aug 10 12:31:38 2020 From: ildiko.vancsa at gmail.com (Ildiko Vancsa) Date: Mon, 10 Aug 2020 14:31:38 +0200 Subject: [upstream-institute] Virtual training sign-up and planning Message-ID: Hi mentors, I’m reaching out to you as the next Open Infrastructure Summit is approaching quickly so it is time to start planning for the next OpenStack Upstream Institute. As the next event will be virtual we will need to re-think the training format and experience to make sure our audience gets the most out of it. I created a new entry on our training occasions wiki page here: https://wiki.openstack.org/wiki/OpenStack_Upstream_Institute_Occasions#Virtual_Training.2C_2020 Please __sign up on the wiki__ if you would like to participate in the preparations and running the virtual training. As it is still vacation season I think we can target the last week of August or first week of September to have the first prep meeting and can collect ideas here or discuss them on the #openstack-upstream-institute IRC channel on Freenode in the meantime. Please let me know if you have any questions or need any help with signing up on the wiki. Thanks and Best Regards, Ildikó From monika.samal at outlook.com Mon Aug 10 07:32:06 2020 From: monika.samal at outlook.com (Monika Samal) Date: Mon, 10 Aug 2020 07:32:06 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , Message-ID: Sure, I am trying and will confirm shortly Get Outlook for Android ________________________________ From: Fabian Zimmermann Sent: Monday, August 10, 2020 11:19:36 AM To: Michael Johnson Cc: Monika Samal ; openstack-discuss ; Mark Goddard Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Hi, to test your connection you can create an instance im the octavia network and try to ping/ssh from your controller (dont forget a suitable security group) Fabian Michael Johnson > schrieb am Mo., 10. Aug. 2020, 07:44: That looks like there is still a kolla networking issue where the amphora are not able to reach the controller processes. Please fix the lb-mgmt-net such that it can reach the amphora and the controller containers. This should be setup via the deployment tool, kolla in this case. Michael On Sun, Aug 9, 2020 at 5:02 AM Monika Samal > wrote: Hi All, Below is the error am getting, i tried configuring network issue as well still finding it difficult to resolve. Below is my log...if somebody can help me resolving it..it would be great help since its very urgent... http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ Regards, Monika ________________________________ From: Monika Samal > Sent: Sunday, 9 August, 2020, 5:29 pm To: Mark Goddard; Michael Johnson; openstack-discuss Cc: Fabian Zimmermann Subject: Re: [openstack-community] Octavia :; Unable to create load balancer ________________________________ From: Monika Samal > Sent: Friday, August 7, 2020 4:41:52 AM To: Mark Goddard >; Michael Johnson > Cc: Fabian Zimmermann >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer I tried following above document still facing same Octavia connection error with amphora image. Regards, Monika ________________________________ From: Mark Goddard > Sent: Thursday, August 6, 2020 1:16:01 PM To: Michael Johnson > Cc: Monika Samal >; Fabian Zimmermann >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer On Wed, 5 Aug 2020 at 16:16, Michael Johnson > wrote: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard > wrote: On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson > Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann > Cc: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From dikonoor at in.ibm.com Mon Aug 10 07:54:21 2020 From: dikonoor at in.ibm.com (Divya K Konoor) Date: Mon, 10 Aug 2020 13:24:21 +0530 Subject: [openstack-community] Keystone and DBNonExistent Errors In-Reply-To: References: Message-ID: Hi, I am using OpenStack Keystone Stein and run into the below error often where Keystone public process(listening to 5000) is running inside Apache httpd runs into the below. This problem is resolved with a restart of httpd service. Has anyone run into a similar issue ? This is seen soon after httpd is restarted and does not happen all the time. My environment has MariaDB backend. This problem is not limited to the assignment table and is seen across all other tables in Keystone. MariaDB service is functional and all the tables are in place. [Fri Aug 07 08:20:59.936087 2020] [:info] [pid 1420287] mod_wsgi (pid=1420287, process='keystone-public', application=''): Loading WSGI script '/usr/bin/keystone-wsgi-public'. [Fri Aug 07 08:20:59.936089 2020] [:info] [pid 1420288] mod_wsgi (pid=1420288, process='keystone-admin', application=''): Loading WSGI script '/usr/bin/keystone-wsgi-admin'. [Fri Aug 07 08:20:59.943431 2020] [ssl:info] [pid 1420290] [client 1.2.3.95:35762] AH01964: Connection to child 1 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:00.009317 2020] [ssl:info] [pid 1420291] [client 1.2.3.113:60132] AH01964: Connection to child 2 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:01.243594 2020] [ssl:info] [pid 1420289] [client 1.2.3.50:53996] AH01964: Connection to child 0 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:01.386329 2020] [ssl:info] [pid 1420293] [client x.x.x.x:38645] AH01964: Connection to child 4 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:01.824041 2020] [ssl:info] [pid 1420349] [client 1.2.3.101:42974] AH01964: Connection to child 5 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:02.949166 2020] [ssl:info] [pid 1420378] [client 1.2.3.50:54014] AH01964: Connection to child 9 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:02.949172 2020] [ssl:info] [pid 1420379] [client 1.2.3.80:46924] AH01964: Connection to child 10 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:03.286057 2020] [:info] [pid 1420287] mod_wsgi (pid=1420287): Create interpreter '1.2.3.50:5000|'. [Fri Aug 07 08:21:03.287286 2020] [:info] [pid 1420287] [remote 1.2.3.95:156] mod_wsgi (pid=1420287, process='keystone-public', application='1.2.3.50:5000|'): Loading WSGI script '/usr/bin/keystone-wsgi-public'. [Fri Aug 07 08:21:04.675059 2020] [ssl:info] [pid 1420436] [client 1.2.3.50:54032] AH01964: Connection to child 12 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:04.705975 2020] [ssl:info] [pid 1420437] [client 1.2.3.107:59554] AH01964: Connection to child 13 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:06.960940 2020] [ssl:info] [pid 1420438] [client 1.2.3.80:46970] AH01964: Connection to child 14 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:07.661670 2020] [ssl:info] [pid 1420349] [client 1.2.3.50:54124] AH01964: Connection to child 5 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:07.683383 2020] [ssl:info] [pid 1420292] [client x.x.x.x:30065] AH01964: Connection to child 3 established (server 1.2.3.50:5000) [Fri Aug 07 08:21:08.442956 2020] [:error] [pid 1420287] [remote x.x.x.x:144] mod_wsgi (pid=1420287): Exception occurred processing WSGI script '/usr/bin/keystone-wsgi-public'. [Fri Aug 07 08:21:08.443002 2020] [:error] [pid 1420287] [remote x.x.x.x:144] Traceback (most recent call last): [Fri Aug 07 08:21:08.443017 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__ [Fri Aug 07 08:21:08.443509 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.wsgi_app(environ, start_response) [Fri Aug 07 08:21:08.443525 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/werkzeug/contrib/fixers.py", line 152, in __call__ [Fri Aug 07 08:21:08.443630 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.app(environ, start_response) [Fri Aug 07 08:21:08.443644 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.443746 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.443756 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.443773 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.443781 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__ [Fri Aug 07 08:21:08.443844 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = req.get_response(self.application) [Fri Aug 07 08:21:08.443859 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send [Fri Aug 07 08:21:08.444194 2020] [:error] [pid 1420287] [remote x.x.x.x:144] application, catch_exc_info=False) [Fri Aug 07 08:21:08.444203 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application [Fri Aug 07 08:21:08.444220 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_iter = application(self.environ, start_response) [Fri Aug 07 08:21:08.444229 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 143, in __call__ [Fri Aug 07 08:21:08.444245 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return resp(environ, start_response) [Fri Aug 07 08:21:08.444253 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.444268 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.444276 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.444292 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.444300 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__ [Fri Aug 07 08:21:08.444315 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = req.get_response(self.application) [Fri Aug 07 08:21:08.444323 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send [Fri Aug 07 08:21:08.444338 2020] [:error] [pid 1420287] [remote x.x.x.x:144] application, catch_exc_info=False) [Fri Aug 07 08:21:08.444346 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application [Fri Aug 07 08:21:08.444361 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_iter = application(self.environ, start_response) [Fri Aug 07 08:21:08.444370 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.444385 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.444393 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.444408 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.444416 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 112, in __call__ [Fri Aug 07 08:21:08.444476 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return request.get_response(self.application) [Fri Aug 07 08:21:08.444485 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send [Fri Aug 07 08:21:08.444501 2020] [:error] [pid 1420287] [remote x.x.x.x:144] application, catch_exc_info=False) [Fri Aug 07 08:21:08.444509 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application [Fri Aug 07 08:21:08.444524 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_iter = application(self.environ, start_response) [Fri Aug 07 08:21:08.444533 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.444547 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.444556 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.444571 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.444587 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/oslo_middleware/request_id.py", line 58, in __call__ [Fri Aug 07 08:21:08.444636 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = req.get_response(self.application) [Fri Aug 07 08:21:08.444645 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send [Fri Aug 07 08:21:08.444660 2020] [:error] [pid 1420287] [remote x.x.x.x:144] application, catch_exc_info=False) [Fri Aug 07 08:21:08.444669 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application [Fri Aug 07 08:21:08.444684 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_iter = application(self.environ, start_response) [Fri Aug 07 08:21:08.444698 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/server/flask/request_processing/middleware/url_normalize.py", line 38, in __call__ [Fri Aug 07 08:21:08.444750 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.app(environ, start_response) [Fri Aug 07 08:21:08.444759 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.444774 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.444783 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.444797 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.444810 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 333, in __call__ [Fri Aug 07 08:21:08.444828 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = req.get_response(self._app) [Fri Aug 07 08:21:08.444836 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send [Fri Aug 07 08:21:08.444851 2020] [:error] [pid 1420287] [remote x.x.x.x:144] application, catch_exc_info=False) [Fri Aug 07 08:21:08.444859 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application [Fri Aug 07 08:21:08.444874 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_iter = application(self.environ, start_response) [Fri Aug 07 08:21:08.444883 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 129, in __call__ [Fri Aug 07 08:21:08.444897 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = self.call_func(req, *args, **kw) [Fri Aug 07 08:21:08.444906 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/dec.py", line 193, in call_func [Fri Aug 07 08:21:08.444920 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.func(req, *args, **kwargs) [Fri Aug 07 08:21:08.444929 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/oslo_middleware/base.py", line 131, in __call__ [Fri Aug 07 08:21:08.444944 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = req.get_response(self.application) [Fri Aug 07 08:21:08.444952 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1314, in send [Fri Aug 07 08:21:08.444967 2020] [:error] [pid 1420287] [remote x.x.x.x:144] application, catch_exc_info=False) [Fri Aug 07 08:21:08.444975 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/webob/request.py", line 1278, in call_application [Fri Aug 07 08:21:08.444990 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_iter = application(self.environ, start_response) [Fri Aug 07 08:21:08.444998 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/werkzeug/wsgi.py", line 826, in __call__ [Fri Aug 07 08:21:08.445279 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return app(environ, start_response) [Fri Aug 07 08:21:08.445288 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app [Fri Aug 07 08:21:08.445304 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = self.handle_exception(e) [Fri Aug 07 08:21:08.445316 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445490 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445500 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445516 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445524 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445539 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445547 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445562 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445570 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445585 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445593 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445608 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445616 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445630 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445639 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445654 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445662 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445676 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445685 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445699 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445708 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445722 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445731 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445745 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445758 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445772 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445780 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445795 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445803 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445818 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445826 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445841 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445849 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445863 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445871 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445886 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445894 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445908 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445917 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445931 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445939 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445954 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445962 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445976 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.445984 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.445999 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446007 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446021 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446030 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446044 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446052 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446067 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446075 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446090 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446098 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446113 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446121 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446135 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446143 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446158 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446166 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception [Fri Aug 07 08:21:08.446181 2020] [:error] [pid 1420287] [remote x.x.x.x:144] reraise(exc_type, exc_value, tb) [Fri Aug 07 08:21:08.446190 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 266, in error_router [Fri Aug 07 08:21:08.446204 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.handle_error(e) [Fri Aug 07 08:21:08.446212 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app [Fri Aug 07 08:21:08.446227 2020] [:error] [pid 1420287] [remote x.x.x.x:144] response = self.full_dispatch_request() [Fri Aug 07 08:21:08.446236 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request [Fri Aug 07 08:21:08.446250 2020] [:error] [pid 1420287] [remote x.x.x.x:144] rv = self.handle_user_exception(e) [Fri Aug 07 08:21:08.446259 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446273 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446282 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446296 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446304 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446318 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446327 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446341 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446349 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446363 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446372 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446386 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446395 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446409 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446417 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446432 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446440 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446454 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446463 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446477 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446485 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446500 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446508 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446522 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446531 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446545 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446553 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446568 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446576 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446590 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446599 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446613 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446621 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446636 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446644 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446658 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446667 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446681 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446689 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446704 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446712 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446727 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446735 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446749 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446757 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446772 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446780 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446795 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446803 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446817 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446826 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446840 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446848 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446863 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446871 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446885 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446893 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446908 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446916 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 269, in error_router [Fri Aug 07 08:21:08.446930 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return original_handler(e) [Fri Aug 07 08:21:08.446938 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception [Fri Aug 07 08:21:08.446953 2020] [:error] [pid 1420287] [remote x.x.x.x:144] reraise(exc_type, exc_value, tb) [Fri Aug 07 08:21:08.446962 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 266, in error_router [Fri Aug 07 08:21:08.446976 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.handle_error(e) [Fri Aug 07 08:21:08.446984 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request [Fri Aug 07 08:21:08.446999 2020] [:error] [pid 1420287] [remote x.x.x.x:144] rv = self.dispatch_request() [Fri Aug 07 08:21:08.447007 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request [Fri Aug 07 08:21:08.447022 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.view_functions[rule.endpoint](**req.view_args) [Fri Aug 07 08:21:08.447031 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 458, in wrapper [Fri Aug 07 08:21:08.447046 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = resource(*args, **kwargs) [Fri Aug 07 08:21:08.447055 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask/views.py", line 88, in view [Fri Aug 07 08:21:08.447119 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self.dispatch_request(*args, **kwargs) [Fri Aug 07 08:21:08.447128 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/flask_restful/__init__.py", line 573, in dispatch_request [Fri Aug 07 08:21:08.447144 2020] [:error] [pid 1420287] [remote x.x.x.x:144] resp = meth(*args, **kwargs) [Fri Aug 07 08:21:08.447152 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/server/flask/common.py", line 1060, in wrapper [Fri Aug 07 08:21:08.447392 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return f(*args, **kwargs) [Fri Aug 07 08:21:08.447406 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/api/auth.py", line 312, in post [Fri Aug 07 08:21:08.447551 2020] [:error] [pid 1420287] [remote x.x.x.x:144] token = authentication.authenticate_for_token(auth_data) [Fri Aug 07 08:21:08.447561 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/api/_shared/authentication.py", line 229, in authenticate_for_token [Fri Aug 07 08:21:08.447652 2020] [:error] [pid 1420287] [remote x.x.x.x:144] app_cred_id=app_cred_id, parent_audit_id=token_audit_id) [Fri Aug 07 08:21:08.447662 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 116, in wrapped [Fri Aug 07 08:21:08.447679 2020] [:error] [pid 1420287] [remote x.x.x.x:144] __ret_val = __f(*args, **kwargs) [Fri Aug 07 08:21:08.447687 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/token/provider.py", line 252, in issue_token [Fri Aug 07 08:21:08.447706 2020] [:error] [pid 1420287] [remote x.x.x.x:144] token.mint(token_id, issued_at) [Fri Aug 07 08:21:08.447714 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/models/token_model.py", line 563, in mint [Fri Aug 07 08:21:08.448498 2020] [:error] [pid 1420287] [remote x.x.x.x:144] self._validate_project_scope() [Fri Aug 07 08:21:08.448508 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/models/token_model.py", line 512, in _validate_project_scope [Fri Aug 07 08:21:08.448525 2020] [:error] [pid 1420287] [remote x.x.x.x:144] if self.project_scoped and not self.roles: [Fri Aug 07 08:21:08.448533 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/models/token_model.py", line 438, in roles [Fri Aug 07 08:21:08.448549 2020] [:error] [pid 1420287] [remote x.x.x.x:144] roles = self._get_project_roles() [Fri Aug 07 08:21:08.448557 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/models/token_model.py", line 400, in _get_project_roles [Fri Aug 07 08:21:08.448573 2020] [:error] [pid 1420287] [remote x.x.x.x:144] self.user_id, self.project_id [Fri Aug 07 08:21:08.448581 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 116, in wrapped [Fri Aug 07 08:21:08.448597 2020] [:error] [pid 1420287] [remote x.x.x.x:144] __ret_val = __f(*args, **kwargs) [Fri Aug 07 08:21:08.448605 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1220, in decorate [Fri Aug 07 08:21:08.449478 2020] [:error] [pid 1420287] [remote x.x.x.x:144] should_cache_fn) [Fri Aug 07 08:21:08.449488 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 825, in get_or_create [Fri Aug 07 08:21:08.449504 2020] [:error] [pid 1420287] [remote x.x.x.x:144] async_creator) as value: [Fri Aug 07 08:21:08.449512 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/lock.py", line 154, in __enter__ [Fri Aug 07 08:21:08.449967 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self._enter() [Fri Aug 07 08:21:08.449977 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/lock.py", line 94, in _enter [Fri Aug 07 08:21:08.449995 2020] [:error] [pid 1420287] [remote x.x.x.x:144] generated = self._enter_create(createdtime) [Fri Aug 07 08:21:08.450004 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/lock.py", line 145, in _enter_create [Fri Aug 07 08:21:08.450020 2020] [:error] [pid 1420287] [remote x.x.x.x:144] created = self.creator() [Fri Aug 07 08:21:08.450029 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 792, in gen_value [Fri Aug 07 08:21:08.450046 2020] [:error] [pid 1420287] [remote x.x.x.x:144] created_value = creator() [Fri Aug 07 08:21:08.450055 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/dogpile/cache/region.py", line 1216, in creator [Fri Aug 07 08:21:08.450071 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return fn(*arg, **kw) [Fri Aug 07 08:21:08.450080 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 128, in get_roles_for_user_and_project [Fri Aug 07 08:21:08.450975 2020] [:error] [pid 1420287] [remote x.x.x.x:144] user_id=user_id, project_id=project_id, effective=True) [Fri Aug 07 08:21:08.450985 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 116, in wrapped [Fri Aug 07 08:21:08.451001 2020] [:error] [pid 1420287] [remote x.x.x.x:144] __ret_val = __f(*args, **kwargs) [Fri Aug 07 08:21:08.451009 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 999, in list_role_assignments [Fri Aug 07 08:21:08.451025 2020] [:error] [pid 1420287] [remote x.x.x.x:144] strip_domain_roles) [Fri Aug 07 08:21:08.451033 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 845, in _list_effective_role_assignments [Fri Aug 07 08:21:08.451049 2020] [:error] [pid 1420287] [remote x.x.x.x:144] domain_id=domain_id, inherited=inherited) [Fri Aug 07 08:21:08.451057 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/core.py", line 780, in list_role_assignments_for_actor [Fri Aug 07 08:21:08.451072 2020] [:error] [pid 1420287] [remote x.x.x.x:144] group_ids=group_ids, inherited_to_projects=False) [Fri Aug 07 08:21:08.451081 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib/python2.7/site-packages/keystone/assignment/backends/sql.py", line 248, in list_role_assignments [Fri Aug 07 08:21:08.451599 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return [denormalize_role(ref) for ref in query.all()] [Fri Aug 07 08:21:08.451609 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2925, in all [Fri Aug 07 08:21:08.451632 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return list(self) [Fri Aug 07 08:21:08.451641 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3081, in __iter__ [Fri Aug 07 08:21:08.451656 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return self._execute_and_instances(context) [Fri Aug 07 08:21:08.451665 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 3106, in _execute_and_instances [Fri Aug 07 08:21:08.451683 2020] [:error] [pid 1420287] [remote x.x.x.x:144] result = conn.execute(querycontext.statement, self._params) [Fri Aug 07 08:21:08.451691 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute [Fri Aug 07 08:21:08.451711 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return meth(self, multiparams, params) [Fri Aug 07 08:21:08.451720 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection [Fri Aug 07 08:21:08.451736 2020] [:error] [pid 1420287] [remote x.x.x.x:144] return connection._execute_clauseelement(self, multiparams, params) [Fri Aug 07 08:21:08.451745 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement [Fri Aug 07 08:21:08.451762 2020] [:error] [pid 1420287] [remote x.x.x.x:144] distilled_params, [Fri Aug 07 08:21:08.451771 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context [Fri Aug 07 08:21:08.451786 2020] [:error] [pid 1420287] [remote x.x.x.x:144] e, statement, parameters, cursor, context [Fri Aug 07 08:21:08.451795 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception [Fri Aug 07 08:21:08.451810 2020] [:error] [pid 1420287] [remote x.x.x.x:144] util.raise_from_cause(newraise, exc_info) [Fri Aug 07 08:21:08.451818 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 296, in raise_from_cause [Fri Aug 07 08:21:08.451834 2020] [:error] [pid 1420287] [remote x.x.x.x:144] reraise(type(exception), exception, tb=exc_tb, cause=cause) [Fri Aug 07 08:21:08.451843 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context [Fri Aug 07 08:21:08.451858 2020] [:error] [pid 1420287] [remote x.x.x.x:144] cursor, statement, parameters, context [Fri Aug 07 08:21:08.451866 2020] [:error] [pid 1420287] [remote x.x.x.x:144] File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute [Fri Aug 07 08:21:08.451882 2020] [:error] [pid 1420287] [remote x.x.x.x:144] cursor.execute(statement, parameters) [Fri Aug 07 08:21:08.451923 2020] [:error] [pid 1420287] [remote x.x.x.x:144] DBNonExistentTable: (sqlite3.OperationalError) no such table: assignment [SQL: u'SELECT assignment.type AS assignment_type, assignment.actor_id AS assignment_actor_id, assignment.target_id AS assignment_target_id, assignment.role_id AS assignment_role_id, assignment.inherited AS assignment_inherited \\nFROM assignment \\nWHERE assignment.actor_id IN (?) AND assignment.target_id IN (?) AND assignment.type IN (?) AND assignment.inherited = 0'] [parameters: ('15c2fe91e053af57a997c568c117c908d59c138f996bdc19ae97e9f16df12345', '12345978536e45ab8a279e2b0fa4f947', 'UserProject')] (Background on this error at: http://sqlalche.me/e/e3q8) Regards, Divya -------------- next part -------------- An HTML attachment was scrubbed... URL: From yan.y.zhao at intel.com Mon Aug 10 07:46:31 2020 From: yan.y.zhao at intel.com (Yan Zhao) Date: Mon, 10 Aug 2020 15:46:31 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200805105319.GF2177@nanopsycho> References: <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> <20200805093338.GC30485@joy-OptiPlex-7040> <20200805105319.GF2177@nanopsycho> Message-ID: <20200810074631.GA29059@joy-OptiPlex-7040> On Wed, Aug 05, 2020 at 12:53:19PM +0200, Jiri Pirko wrote: > Wed, Aug 05, 2020 at 11:33:38AM CEST, yan.y.zhao at intel.com wrote: > >On Wed, Aug 05, 2020 at 04:02:48PM +0800, Jason Wang wrote: > >> > >> On 2020/8/5 下午3:56, Jiri Pirko wrote: > >> > Wed, Aug 05, 2020 at 04:41:54AM CEST, jasowang at redhat.com wrote: > >> > > On 2020/8/5 上午10:16, Yan Zhao wrote: > >> > > > On Wed, Aug 05, 2020 at 10:22:15AM +0800, Jason Wang wrote: > >> > > > > On 2020/8/5 上午12:35, Cornelia Huck wrote: > >> > > > > > [sorry about not chiming in earlier] > >> > > > > > > >> > > > > > On Wed, 29 Jul 2020 16:05:03 +0800 > >> > > > > > Yan Zhao wrote: > >> > > > > > > >> > > > > > > On Mon, Jul 27, 2020 at 04:23:21PM -0600, Alex Williamson wrote: > >> > > > > > (...) > >> > > > > > > >> > > > > > > > Based on the feedback we've received, the previously proposed interface > >> > > > > > > > is not viable. I think there's agreement that the user needs to be > >> > > > > > > > able to parse and interpret the version information. Using json seems > >> > > > > > > > viable, but I don't know if it's the best option. Is there any > >> > > > > > > > precedent of markup strings returned via sysfs we could follow? > >> > > > > > I don't think encoding complex information in a sysfs file is a viable > >> > > > > > approach. Quoting Documentation/filesystems/sysfs.rst: > >> > > > > > > >> > > > > > "Attributes should be ASCII text files, preferably with only one value > >> > > > > > per file. It is noted that it may not be efficient to contain only one > >> > > > > > value per file, so it is socially acceptable to express an array of > >> > > > > > values of the same type. > >> > > > > > Mixing types, expressing multiple lines of data, and doing fancy > >> > > > > > formatting of data is heavily frowned upon." > >> > > > > > > >> > > > > > Even though this is an older file, I think these restrictions still > >> > > > > > apply. > >> > > > > +1, that's another reason why devlink(netlink) is better. > >> > > > > > >> > > > hi Jason, > >> > > > do you have any materials or sample code about devlink, so we can have a good > >> > > > study of it? > >> > > > I found some kernel docs about it but my preliminary study didn't show me the > >> > > > advantage of devlink. > >> > > > >> > > CC Jiri and Parav for a better answer for this. > >> > > > >> > > My understanding is that the following advantages are obvious (as I replied > >> > > in another thread): > >> > > > >> > > - existing users (NIC, crypto, SCSI, ib), mature and stable > >> > > - much better error reporting (ext_ack other than string or errno) > >> > > - namespace aware > >> > > - do not couple with kobject > >> > Jason, what is your use case? > >> > >> > >> I think the use case is to report device compatibility for live migration. > >> Yan proposed a simple sysfs based migration version first, but it looks not > >> sufficient and something based on JSON is discussed. > >> > >> Yan, can you help to summarize the discussion so far for Jiri as a > >> reference? > >> > >yes. > >we are currently defining an device live migration compatibility > >interface in order to let user space like openstack and libvirt knows > >which two devices are live migration compatible. > >currently the devices include mdev (a kernel emulated virtual device) > >and physical devices (e.g. a VF of a PCI SRIOV device). > > > >the attributes we want user space to compare including > >common attribues: > > device_api: vfio-pci, vfio-ccw... > > mdev_type: mdev type of mdev or similar signature for physical device > > It specifies a device's hardware capability. e.g. > > i915-GVTg_V5_4 means it's of 1/4 of a gen9 Intel graphics > > device. > > software_version: device driver's version. > > in .[.bugfix] scheme, where there is no > > compatibility across major versions, minor versions have > > forward compatibility (ex. 1-> 2 is ok, 2 -> 1 is not) and > > bugfix version number indicates some degree of internal > > improvement that is not visible to the user in terms of > > features or compatibility, > > > >vendor specific attributes: each vendor may define different attributes > > device id : device id of a physical devices or mdev's parent pci device. > > it could be equal to pci id for pci devices > > aggregator: used together with mdev_type. e.g. aggregator=2 together > > with i915-GVTg_V5_4 means 2*1/4=1/2 of a gen9 Intel > > graphics device. > > remote_url: for a local NVMe VF, it may be configured with a remote > > url of a remote storage and all data is stored in the > > remote side specified by the remote url. > > ... > > > >Comparing those attributes by user space alone is not an easy job, as it > >can't simply assume an equal relationship between source attributes and > >target attributes. e.g. > >for a source device of mdev_type=i915-GVTg_V5_4,aggregator=2, (1/2 of > >gen9), it actually could find a compatible device of > >mdev_type=i915-GVTg_V5_8,aggregator=4 (also 1/2 of gen9), > >if mdev_type of i915-GVTg_V5_4 is not available in the target machine. > > > >So, in our current proposal, we want to create two sysfs attributes > >under a device sysfs node. > >/sys//migration/self > >/sys//migration/compatible > > > >#cat /sys//migration/self > >device_type=vfio_pci > >mdev_type=i915-GVTg_V5_4 > >device_id=8086591d > >aggregator=2 > >software_version=1.0.0 > > > >#cat /sys//migration/compatible > >device_type=vfio_pci > >mdev_type=i915-GVTg_V5_{val1:int:2,4,8} > >device_id=8086591d > >aggregator={val1}/2 > >software_version=1.0.0 > > > >The /sys//migration/self specifies self attributes of > >a device. > >The /sys//migration/compatible specifies the list of > >compatible devices of a device. as in the example, compatible devices > >could have > > device_type == vfio_pci && > > device_id == 8086591d && > > software_version == 1.0.0 && > > ( > > (mdev_type of i915-GVTg_V5_2 && aggregator==1) || > > (mdev_type of i915-GVTg_V5_4 && aggregator==2) || > > (mdev_type of i915-GVTg_V5_8 && aggregator=4) > > ) > > > >by comparing whether a target device is in compatible list of source > >device, the user space can know whether a two devices are live migration > >compatible. > > > >Additional notes: > >1)software_version in the compatible list may not be necessary as it > >already has a major.minor.bugfix scheme. > >2)for vendor attribute like remote_url, it may not be statically > >assigned and could be changed with a device interface. > > > >So, as Cornelia pointed that it's not good to use complex format in > >a sysfs attribute, we'd like to know whether there're other good ways to > >our use case, e.g. splitting a single attribute to multiple simple sysfs > >attributes as what Cornelia suggested or devlink that Jason has strongly > >recommended. > > Hi Yan. > Hi Jiri, > Thanks for the explanation, I'm still fuzzy about the details. > Anyway, I suggest you to check "devlink dev info" command we have > implemented for multiple drivers. You can try netdevsim to test this. > I think that the info you need to expose might be put there. do you mean drivers/net/netdevsim/ ? > > Devlink creates instance per-device. Specific device driver calls into > devlink core to create the instance. What device do you have? What the devlink core is net/core/devlink.c ? > driver is it handled by? It looks that the devlink is for network device specific, and in devlink.h, it says include/uapi/linux/devlink.h - Network physical device Netlink interface, I feel like it's not very appropriate for a GPU driver to use this interface. Is that right? Thanks Yan From monika.samal at outlook.com Mon Aug 10 11:12:31 2020 From: monika.samal at outlook.com (Monika Samal) Date: Mon, 10 Aug 2020 11:12:31 +0000 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: , Message-ID: Hey Fabian, I tried creating and testing instance with my available subnet created for loadbalancer , I am not able to ping it. Please find below ip a output for controller and deployment node: Controller Node: 30.0.0.14 [cid:18b1d2b8-1adf-45f1-9dd6-b185223e060e] Deployment Node: 30.0.0.11 [cid:20b35bae-677f-462c-8c38-7fafdb058219] [cid:66fb7ccf-ba53-4662-a0b8-b129da885f1a] ________________________________ From: Fabian Zimmermann Sent: Monday, August 10, 2020 11:19 AM To: Michael Johnson Cc: Monika Samal ; openstack-discuss ; Mark Goddard Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Hi, to test your connection you can create an instance im the octavia network and try to ping/ssh from your controller (dont forget a suitable security group) Fabian Michael Johnson > schrieb am Mo., 10. Aug. 2020, 07:44: That looks like there is still a kolla networking issue where the amphora are not able to reach the controller processes. Please fix the lb-mgmt-net such that it can reach the amphora and the controller containers. This should be setup via the deployment tool, kolla in this case. Michael On Sun, Aug 9, 2020 at 5:02 AM Monika Samal > wrote: Hi All, Below is the error am getting, i tried configuring network issue as well still finding it difficult to resolve. Below is my log...if somebody can help me resolving it..it would be great help since its very urgent... http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ Regards, Monika ________________________________ From: Monika Samal > Sent: Sunday, 9 August, 2020, 5:29 pm To: Mark Goddard; Michael Johnson; openstack-discuss Cc: Fabian Zimmermann Subject: Re: [openstack-community] Octavia :; Unable to create load balancer ________________________________ From: Monika Samal > Sent: Friday, August 7, 2020 4:41:52 AM To: Mark Goddard >; Michael Johnson > Cc: Fabian Zimmermann >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer I tried following above document still facing same Octavia connection error with amphora image. Regards, Monika ________________________________ From: Mark Goddard > Sent: Thursday, August 6, 2020 1:16:01 PM To: Michael Johnson > Cc: Monika Samal >; Fabian Zimmermann >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer On Wed, 5 Aug 2020 at 16:16, Michael Johnson > wrote: Looking at that error, it appears that the lb-mgmt-net is not setup correctly. The Octavia controller containers are not able to reach the amphora instances on the lb-mgmt-net subnet. I don't know how kolla is setup to connect the containers to the neutron lb-mgmt-net network. Maybe the above documents will help with that. Right now it's up to the operator to configure that. The kolla documentation doesn't prescribe any particular setup. We're working on automating it in Victoria. Michael On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard > wrote: On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: Hello Guys, With Michaels help I was able to solve the problem but now there is another error I was able to create my network on vlan but still error persist. PFB the logs: http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ Kindly help regards, Monika ________________________________ From: Michael Johnson > Sent: Monday, August 3, 2020 9:10 PM To: Fabian Zimmermann > Cc: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Yeah, it looks like nova is failing to boot the instance. Check this setting in your octavia.conf files: https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id Also, if kolla-ansible didn't set both of these values correctly, please open bug reports for kolla-ansible. These all should have been configured by the deployment tool. I wasn't following this thread due to no [kolla] tag, but here are the recently added docs for Octavia in kolla [1]. Note the octavia_service_auth_project variable which was added to migrate from the admin project to the service project for octavia resources. We're lacking proper automation for the flavor, image etc, but it is being worked on in Victoria [2]. [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html [2] https://review.opendev.org/740180 Michael On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: Seems like the flavor is missing or empty '' - check for typos and enable debug. Check if the nova req contains valid information/flavor. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 15:46: It's registered Get Outlook for Android ________________________________ From: Fabian Zimmermann > Sent: Monday, August 3, 2020 7:08:21 PM To: Monika Samal >; openstack-discuss > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Did you check the (nova) flavor you use in octavia. Fabian Monika Samal > schrieb am Mo., 3. Aug. 2020, 10:53: After Michael suggestion I was able to create load balancer but there is error in status. [X] PFB the error link: http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ ________________________________ From: Monika Samal > Sent: Monday, August 3, 2020 2:08 PM To: Michael Johnson > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Thanks a ton Michael for helping me out ________________________________ From: Michael Johnson > Sent: Friday, July 31, 2020 3:57 AM To: Monika Samal > Cc: Fabian Zimmermann >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer Just to close the loop on this, the octavia.conf file had "project_name = admin" instead of "project_name = service" in the [service_auth] section. This was causing the keystone errors when Octavia was communicating with neutron. I don't know if that is a bug in kolla-ansible or was just a local configuration issue. Michael On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > Hello Fabian,, > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > Regards, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > Hi, > > just to debug, could you replace the auth_type password with v3password? > > And do a curl against your :5000 and :35357 urls and paste the output. > > Fabian > > Monika Samal > schrieb am Do., 30. Juli 2020, 22:15: > > Hello Fabian, > > http://paste.openstack.org/show/796477/ > > Thanks, > Monika > ________________________________ > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > To: Monika Samal > > Cc: Michael Johnson >; Amy Marrich >; openstack-discuss >; community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load balancer > > The sections should be > > service_auth > keystone_authtoken > > if i read the docs correctly. Maybe you can just paste your config (remove/change passwords) to paste.openstack.org and post the link? > > Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 97899 bytes Desc: image.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 22490 bytes Desc: image.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24398 bytes Desc: image.png URL: From juliaashleykreger at gmail.com Mon Aug 10 16:17:41 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 10 Aug 2020 09:17:41 -0700 Subject: [Ironic] User Survey question Message-ID: Greetings awesome Artificial Intelligences and fellow humanoid carbon units! This week I need to submit the question for the 2021 user survey. We discussed this some during our weekly IRC meeting today.[0] Presently, the question is: "Ironic: What would you find most useful if it was part of ironic?" I'd like to propose we collect more data in order to enable us to make informed decisions for features and maintenance work moving forward. While this is long term thinking, I'm wondering if operators would be interested in collecting and submitting some basic data or using a tool, to submit anonymous usage data so we can gain insight into hardware types in use, numbers of machines, which interfaces are used, etc. So I'm thinking something along the lines of: "Ironic: Would you be willing to submit anonymous usage statistics (Number of nodes, conductors, which drivers are in use, etc) if such a tool existed? Yes/No/Not Applicable" Thoughts? Feelings? Concerns? Other ideas? -Julia [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html From dev.faz at gmail.com Mon Aug 10 16:57:57 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Mon, 10 Aug 2020 18:57:57 +0200 Subject: [openstack-community] Octavia :; Unable to create load balancer In-Reply-To: References: Message-ID: Hi, Check if the vlan of eth1 is reachable by the compute nodes on the nic connected to br-ex. Is the lb-net created with the correct vlan-id so the traffic is able to flow from the nic to br-ex to the instance? Proof this with tcpdump (-e) Fabian connected to the Monika Samal schrieb am Mo., 10. Aug. 2020, 13:12: > Hey Fabian, > > I tried creating and testing instance with my available subnet created for > loadbalancer , I am not able to ping it. > > Please find below ip a output for controller and deployment node: > > Controller Node: 30.0.0.14 > > Deployment Node: 30.0.0.11 > > > > ------------------------------ > *From:* Fabian Zimmermann > *Sent:* Monday, August 10, 2020 11:19 AM > *To:* Michael Johnson > *Cc:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org>; Mark Goddard > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Hi, > > to test your connection you can create an instance im the octavia network > and try to ping/ssh from your controller (dont forget a suitable security > group) > > Fabian > > Michael Johnson schrieb am Mo., 10. Aug. 2020, > 07:44: > > > That looks like there is still a kolla networking issue where the amphora > are not able to reach the controller processes. Please fix the lb-mgmt-net > such that it can reach the amphora and the controller containers. This > should be setup via the deployment tool, kolla in this case. > > Michael > > On Sun, Aug 9, 2020 at 5:02 AM Monika Samal > wrote: > > Hi All, > > Below is the error am getting, i tried configuring network issue as well > still finding it difficult to resolve. > > Below is my log...if somebody can help me resolving it..it would be great > help since its very urgent... > > http://paste.openstack.org/show/TsagcQX2ZKd6rhhsYcYd/ > > Regards, > Monika > ------------------------------ > *From:* Monika Samal > *Sent:* Sunday, 9 August, 2020, 5:29 pm > *To:* Mark Goddard; Michael Johnson; openstack-discuss > *Cc:* Fabian Zimmermann > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > ------------------------------ > *From:* Monika Samal > *Sent:* Friday, August 7, 2020 4:41:52 AM > *To:* Mark Goddard ; Michael Johnson < > johnsomor at gmail.com> > *Cc:* Fabian Zimmermann ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > I tried following above document still facing same Octavia connection > error with amphora image. > > Regards, > Monika > ------------------------------ > *From:* Mark Goddard > *Sent:* Thursday, August 6, 2020 1:16:01 PM > *To:* Michael Johnson > *Cc:* Monika Samal ; Fabian Zimmermann < > dev.faz at gmail.com>; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > On Wed, 5 Aug 2020 at 16:16, Michael Johnson wrote: > > Looking at that error, it appears that the lb-mgmt-net is not setup > correctly. The Octavia controller containers are not able to reach the > amphora instances on the lb-mgmt-net subnet. > > I don't know how kolla is setup to connect the containers to the neutron > lb-mgmt-net network. Maybe the above documents will help with that. > > > Right now it's up to the operator to configure that. The kolla > documentation doesn't prescribe any particular setup. We're working on > automating it in Victoria. > > > Michael > > On Wed, Aug 5, 2020 at 12:53 AM Mark Goddard wrote: > > > > On Tue, 4 Aug 2020 at 16:58, Monika Samal > wrote: > > Hello Guys, > > With Michaels help I was able to solve the problem but now there is > another error I was able to create my network on vlan but still error > persist. PFB the logs: > > http://paste.openstack.org/show/fEixSudZ6lzscxYxsG1z/ > > Kindly help > > regards, > Monika > ------------------------------ > *From:* Michael Johnson > *Sent:* Monday, August 3, 2020 9:10 PM > *To:* Fabian Zimmermann > *Cc:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Yeah, it looks like nova is failing to boot the instance. > > Check this setting in your octavia.conf files: > https://docs.openstack.org/octavia/latest/configuration/configref.html#controller_worker.amp_flavor_id > > Also, if kolla-ansible didn't set both of these values correctly, please > open bug reports for kolla-ansible. These all should have been configured > by the deployment tool. > > > I wasn't following this thread due to no [kolla] tag, but here are the > recently added docs for Octavia in kolla [1]. Note > the octavia_service_auth_project variable which was added to migrate from > the admin project to the service project for octavia resources. We're > lacking proper automation for the flavor, image etc, but it is being worked > on in Victoria [2]. > > [1] > https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavia.html > [2] https://review.opendev.org/740180 > > Michael > > On Mon, Aug 3, 2020 at 7:53 AM Fabian Zimmermann > wrote: > > Seems like the flavor is missing or empty '' - check for typos and enable > debug. > > Check if the nova req contains valid information/flavor. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 15:46: > > It's registered > > Get Outlook for Android > ------------------------------ > *From:* Fabian Zimmermann > *Sent:* Monday, August 3, 2020 7:08:21 PM > *To:* Monika Samal ; openstack-discuss < > openstack-discuss at lists.openstack.org> > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Did you check the (nova) flavor you use in octavia. > > Fabian > > Monika Samal schrieb am Mo., 3. Aug. 2020, > 10:53: > > After Michael suggestion I was able to create load balancer but there is > error in status. > > > > PFB the error link: > > http://paste.openstack.org/show/meNZCeuOlFkfjj189noN/ > ------------------------------ > *From:* Monika Samal > *Sent:* Monday, August 3, 2020 2:08 PM > *To:* Michael Johnson > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Thanks a ton Michael for helping me out > ------------------------------ > *From:* Michael Johnson > *Sent:* Friday, July 31, 2020 3:57 AM > *To:* Monika Samal > *Cc:* Fabian Zimmermann ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > *Subject:* Re: [openstack-community] Octavia :; Unable to create load > balancer > > Just to close the loop on this, the octavia.conf file had > "project_name = admin" instead of "project_name = service" in the > [service_auth] section. This was causing the keystone errors when > Octavia was communicating with neutron. > > I don't know if that is a bug in kolla-ansible or was just a local > configuration issue. > > Michael > > On Thu, Jul 30, 2020 at 1:39 PM Monika Samal > wrote: > > > > Hello Fabian,, > > > > http://paste.openstack.org/show/QxKv2Ai697qulp9UWTjY/ > > > > Regards, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:57 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > Hi, > > > > just to debug, could you replace the auth_type password with v3password? > > > > And do a curl against your :5000 and :35357 urls and paste the output. > > > > Fabian > > > > Monika Samal schrieb am Do., 30. Juli 2020, > 22:15: > > > > Hello Fabian, > > > > http://paste.openstack.org/show/796477/ > > > > Thanks, > > Monika > > ________________________________ > > From: Fabian Zimmermann > > Sent: Friday, July 31, 2020 1:38 AM > > To: Monika Samal > > Cc: Michael Johnson ; Amy Marrich ; > openstack-discuss ; > community at lists.openstack.org > > Subject: Re: [openstack-community] Octavia :; Unable to create load > balancer > > > > The sections should be > > > > service_auth > > keystone_authtoken > > > > if i read the docs correctly. Maybe you can just paste your config > (remove/change passwords) to paste.openstack.org and post the link? > > > > Fabian > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 97899 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 22490 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24398 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 24398 bytes Desc: not available URL: From opensrloo at gmail.com Mon Aug 10 17:28:46 2020 From: opensrloo at gmail.com (Ruby Loo) Date: Mon, 10 Aug 2020 13:28:46 -0400 Subject: [Ironic] User Survey question In-Reply-To: References: Message-ID: Hi Julia, Please remind me, are we allowed one question? I was wondering what prevents us from having this tool and then announcing/asking folks to provide the information. Or is the idea that if no one says 'yes', it would be a waste of time to provide such a tool? My concern is that if this is the only question we are allowed to ask, we might not get that much useful information. What about pain-points wrt ironic? Could we ask that? --ruby On Mon, Aug 10, 2020 at 12:23 PM Julia Kreger wrote: > Greetings awesome Artificial Intelligences and fellow humanoid carbon > units! > > This week I need to submit the question for the 2021 user survey. We > discussed this some during our weekly IRC meeting today.[0] > > Presently, the question is: > > "Ironic: What would you find most useful if it was part of ironic?" > > I'd like to propose we collect more data in order to enable us to make > informed decisions for features and maintenance work moving forward. > While this is long term thinking, I'm wondering if operators would be > interested in collecting and submitting some basic data or using a > tool, to submit anonymous usage data so we can gain insight into > hardware types in use, numbers of machines, which interfaces are used, > etc. > > So I'm thinking something along the lines of: > > "Ironic: Would you be willing to submit anonymous usage statistics > (Number of nodes, conductors, which drivers are in use, etc) if such a > tool existed? Yes/No/Not Applicable" > > Thoughts? Feelings? Concerns? Other ideas? > > -Julia > > > [0]: > http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From allison at openstack.org Mon Aug 10 17:30:48 2020 From: allison at openstack.org (Allison Price) Date: Mon, 10 Aug 2020 12:30:48 -0500 Subject: [Ironic] User Survey question In-Reply-To: References: Message-ID: <6E7AE88E-3E6A-4791-AFDB-600DB8E2AC40@openstack.org> Coming solely from the User Survey POV, each project and SIG is allowed up to 2 questions. We create that limit to ensure that the survey does not get too terribly long. If the Ironic team would like to add one question, we can. Thanks! Allison > On Aug 10, 2020, at 12:28 PM, Ruby Loo wrote: > > Hi Julia, > > Please remind me, are we allowed one question? > > I was wondering what prevents us from having this tool and then announcing/asking folks to provide the information. Or is the idea that if no one says 'yes', it would be a waste of time to provide such a tool? My concern is that if this is the only question we are allowed to ask, we might not get that much useful information. > > What about pain-points wrt ironic? Could we ask that? > > --ruby > > On Mon, Aug 10, 2020 at 12:23 PM Julia Kreger > wrote: > Greetings awesome Artificial Intelligences and fellow humanoid carbon units! > > This week I need to submit the question for the 2021 user survey. We > discussed this some during our weekly IRC meeting today.[0] > > Presently, the question is: > > "Ironic: What would you find most useful if it was part of ironic?" > > I'd like to propose we collect more data in order to enable us to make > informed decisions for features and maintenance work moving forward. > While this is long term thinking, I'm wondering if operators would be > interested in collecting and submitting some basic data or using a > tool, to submit anonymous usage data so we can gain insight into > hardware types in use, numbers of machines, which interfaces are used, > etc. > > So I'm thinking something along the lines of: > > "Ironic: Would you be willing to submit anonymous usage statistics > (Number of nodes, conductors, which drivers are in use, etc) if such a > tool existed? Yes/No/Not Applicable" > > Thoughts? Feelings? Concerns? Other ideas? > > -Julia > > > [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Mon Aug 10 19:13:21 2020 From: kennelson11 at gmail.com (Kendall Nelson) Date: Mon, 10 Aug 2020 12:13:21 -0700 Subject: [all] Virtual PTG October 2020 Dates & Registration Message-ID: Hello Everyone! I'm sure you all have been anxiously awaiting the announcement of the dates for the next virtual PTG! After polling the community[1] and balancing the pros and cons, we have decided the PTG will take place the week after the Open Infrastructure Summit[2][3] from October 26th to October 30th, 2020. PTG registration is now open[4]. Like last time, it is free, but we will again be using it to communicate details about the event (schedules, passwords, etc), so please register! Later this week we will send out info about signing up teams. Also, the same as last time, we will have an ethercalc signup and a survey to gather some other data about your team. -the Kendalls (diablo_rojo & wendallkaters) [1] ML Poll for PTG Dates: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016098.html [2] Summit Site: https://www.openstack.org/summit/2020/ [3] Summit Registration: https://openinfrasummit2020.eventbrite.com [4] PTG Registration: https://october2020ptg.eventbrite.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.bell at cern.ch Mon Aug 10 20:37:27 2020 From: tim.bell at cern.ch (timbell) Date: Mon, 10 Aug 2020 22:37:27 +0200 Subject: [Ironic] User Survey question In-Reply-To: References: Message-ID: Ceph has done something like this with good results - https://docs.ceph.com/docs/master/mgr/telemetry/ I think the things that have helped this to be successful are - easy way to see what you would send - option (not the default) to provide more details such as company and contact I think many OpenStack projects could benefit from this sort of approach for - capacity growth - rate of upgrade - support some of the user survey activities by automatically collecting data rather than asking for responses in a manual survey Would it be possible to consider an Oslo module so the infrastructure could be common and then we make it anonymous opt-in ? Tim > On 10 Aug 2020, at 18:17, Julia Kreger wrote: > > Greetings awesome Artificial Intelligences and fellow humanoid carbon units! > > This week I need to submit the question for the 2021 user survey. We > discussed this some during our weekly IRC meeting today.[0] > > Presently, the question is: > > "Ironic: What would you find most useful if it was part of ironic?" > > I'd like to propose we collect more data in order to enable us to make > informed decisions for features and maintenance work moving forward. > While this is long term thinking, I'm wondering if operators would be > interested in collecting and submitting some basic data or using a > tool, to submit anonymous usage data so we can gain insight into > hardware types in use, numbers of machines, which interfaces are used, > etc. > > So I'm thinking something along the lines of: > > "Ironic: Would you be willing to submit anonymous usage statistics > (Number of nodes, conductors, which drivers are in use, etc) if such a > tool existed? Yes/No/Not Applicable" > > Thoughts? Feelings? Concerns? Other ideas? > > -Julia > > > [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emilien at redhat.com Mon Aug 10 20:48:09 2020 From: emilien at redhat.com (Emilien Macchi) Date: Mon, 10 Aug 2020 16:48:09 -0400 Subject: [tripleo][ansible] the current action plugins use patterns are suboptimal? In-Reply-To: <385dc8d7-198f-64ce-908f-49ab823ed229@redhat.com> References: <6feb1d83-5cc8-1916-90a7-1a6b54593310@redhat.com> <385dc8d7-198f-64ce-908f-49ab823ed229@redhat.com> Message-ID: On Tue, Aug 4, 2020 at 5:42 AM Bogdan Dobrelya wrote: (...) > I can understand that ansible should not be fixed for some composition > tasks what require iterations and have complex logic for its "unit of > work". And such ones also should be unit tested indeed. What I do not > fully understand though is then what abandoning paunch for its action > plugin had bought for us in the end? > > Paunch was self-contained and had no external dependencies on > fast-changing ansible frameworks. There was also no need for paunch to > handle the ansible-specific execution strategies and nuances, like "what > if that action plugin is called in async or in the check-mode?" Unit > tests exited in paunch as well. It was easy to backport changes within a > single code base. > > So, looking back retrospectively, was rewriting paunch as an action > plugin a simplification of the deployment framework? Please reply to > yourself honestly. It does pretty same things but differently and added > external framework. It is now also self-contained action plugin, since > traditional tasks cannot be used in loops for this goal because of > performance reasons. > I asked myself the same questions several times and to me the major driver around removing paunch was to move the container logic into Ansible modules which would be community supported vs paunch-runner code. The Ansible role version has also brought more plugability with other components of the framework (Ansible strategies, debugging, etc) but I don't think it was the major reason why we wrote it. The simplification aspect was mostly about removing the CLI which I don't think was needed at the end; and also the unique name thing which was a mistake and caused us many troubles as you know. To summarize, action plugins may be a good solution indeed, but perhaps > we should go back and use paunch instead of ansible? Same applies for > *some* other tasks? That would also provide a balance, for action > plugins, tasks and common sense. > Sagi is currently working on replacing the podman_containers action plugin by a module that will be able to process multiple containers at the same time, so we'll have less tasks and potentially faster operations at scale. -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Mon Aug 10 20:53:26 2020 From: fungi at yuggoth.org (Jeremy Stanley) Date: Mon, 10 Aug 2020 20:53:26 +0000 Subject: [Ironic] User Survey question In-Reply-To: References: Message-ID: <20200810205325.42rr7i4qr4jhdow3@yuggoth.org> On 2020-08-10 22:37:27 +0200 (+0200), timbell wrote: > Ceph has done something like this with good results - > https://docs.ceph.com/docs/master/mgr/telemetry/ > > I think the things that have helped this to be successful are > > - easy way to see what you would send > - option (not the default) to provide more details such as company > and contact [...] Other prior art which springs to mind: Debian has provided a popcon tool for ages, as an opt-in means of periodically providing feedback on what packages are seeing use in their distro. It's current incarnation can submit reports via SMTP or HTTP protocols for added flexibility. https://popcon.debian.org/ OpenBSD takes a low-effort approach and suggests a command in their install guide which the admin can run to send a copy of dmesg output to the project so they can keep track of what sorts of hardware is running their operating system out in the wild. https://www.openbsd.org/faq/faq4.html#SendDmesg -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From Arkady.Kanevsky at dell.com Mon Aug 10 21:06:19 2020 From: Arkady.Kanevsky at dell.com (Kanevsky, Arkady) Date: Mon, 10 Aug 2020 21:06:19 +0000 Subject: [Ironic] User Survey question In-Reply-To: <6E7AE88E-3E6A-4791-AFDB-600DB8E2AC40@openstack.org> References: <6E7AE88E-3E6A-4791-AFDB-600DB8E2AC40@openstack.org> Message-ID: Do we know what % of current deployments use Ironic? I recall several years back it was 25%. But do not recall seeing latest info. Then Julia question on size and which components of Ironic are being used. Maybe if we treat "N/A" answers as they do not use Ironic it first into a single question. I do love open ended question where users can ask for improvements/extensions. Thanks, Arkady From: Allison Price Sent: Monday, August 10, 2020 12:31 PM To: Ruby Loo Cc: Julia Kreger; openstack-discuss Subject: Re: [Ironic] User Survey question [EXTERNAL EMAIL] Coming solely from the User Survey POV, each project and SIG is allowed up to 2 questions. We create that limit to ensure that the survey does not get too terribly long. If the Ironic team would like to add one question, we can. Thanks! Allison On Aug 10, 2020, at 12:28 PM, Ruby Loo > wrote: Hi Julia, Please remind me, are we allowed one question? I was wondering what prevents us from having this tool and then announcing/asking folks to provide the information. Or is the idea that if no one says 'yes', it would be a waste of time to provide such a tool? My concern is that if this is the only question we are allowed to ask, we might not get that much useful information. What about pain-points wrt ironic? Could we ask that? --ruby On Mon, Aug 10, 2020 at 12:23 PM Julia Kreger > wrote: Greetings awesome Artificial Intelligences and fellow humanoid carbon units! This week I need to submit the question for the 2021 user survey. We discussed this some during our weekly IRC meeting today.[0] Presently, the question is: "Ironic: What would you find most useful if it was part of ironic?" I'd like to propose we collect more data in order to enable us to make informed decisions for features and maintenance work moving forward. While this is long term thinking, I'm wondering if operators would be interested in collecting and submitting some basic data or using a tool, to submit anonymous usage data so we can gain insight into hardware types in use, numbers of machines, which interfaces are used, etc. So I'm thinking something along the lines of: "Ironic: Would you be willing to submit anonymous usage statistics (Number of nodes, conductors, which drivers are in use, etc) if such a tool existed? Yes/No/Not Applicable" Thoughts? Feelings? Concerns? Other ideas? -Julia [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From allison at openstack.org Mon Aug 10 21:13:31 2020 From: allison at openstack.org (Allison Price) Date: Mon, 10 Aug 2020 16:13:31 -0500 Subject: [Ironic] User Survey question In-Reply-To: References: <6E7AE88E-3E6A-4791-AFDB-600DB8E2AC40@openstack.org> Message-ID: <1DFB0CD9-008D-4937-AD70-F3150BEC823F@openstack.org> As of right now for the 2020 version, we are sitting at 19% and for 2019, it was the same. > On Aug 10, 2020, at 4:06 PM, Kanevsky, Arkady wrote: > > Do we know what % of current deployments use Ironic? > I recall several years back it was 25%. But do not recall seeing latest info. > Then Julia question on size and which components of Ironic are being used. > Maybe if we treat “N/A” answers as they do not use Ironic it first into a single question. > I do love open ended question where users can ask for improvements/extensions. > Thanks, > Arkady > > From: Allison Price > > Sent: Monday, August 10, 2020 12:31 PM > To: Ruby Loo > Cc: Julia Kreger; openstack-discuss > Subject: Re: [Ironic] User Survey question > > [EXTERNAL EMAIL] > > Coming solely from the User Survey POV, each project and SIG is allowed up to 2 questions. We create that limit to ensure that the survey does not get too terribly long. > > If the Ironic team would like to add one question, we can. > > Thanks! > Allison > > > > > On Aug 10, 2020, at 12:28 PM, Ruby Loo > wrote: > > Hi Julia, > > Please remind me, are we allowed one question? > > I was wondering what prevents us from having this tool and then announcing/asking folks to provide the information. Or is the idea that if no one says 'yes', it would be a waste of time to provide such a tool? My concern is that if this is the only question we are allowed to ask, we might not get that much useful information. > > What about pain-points wrt ironic? Could we ask that? > > --ruby > > On Mon, Aug 10, 2020 at 12:23 PM Julia Kreger > wrote: > Greetings awesome Artificial Intelligences and fellow humanoid carbon units! > > This week I need to submit the question for the 2021 user survey. We > discussed this some during our weekly IRC meeting today.[0] > > Presently, the question is: > > "Ironic: What would you find most useful if it was part of ironic?" > > I'd like to propose we collect more data in order to enable us to make > informed decisions for features and maintenance work moving forward. > While this is long term thinking, I'm wondering if operators would be > interested in collecting and submitting some basic data or using a > tool, to submit anonymous usage data so we can gain insight into > hardware types in use, numbers of machines, which interfaces are used, > etc. > > So I'm thinking something along the lines of: > > "Ironic: Would you be willing to submit anonymous usage statistics > (Number of nodes, conductors, which drivers are in use, etc) if such a > tool existed? Yes/No/Not Applicable" > > Thoughts? Feelings? Concerns? Other ideas? > > -Julia > > > [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From miguel at mlavalle.com Mon Aug 10 22:07:29 2020 From: miguel at mlavalle.com (Miguel Lavalle) Date: Mon, 10 Aug 2020 17:07:29 -0500 Subject: [neutron] bug deputy report August 3 - 9 Message-ID: Critical ====== https://bugs.launchpad.net/neutron/+bug/1890445 [ovn] Tempest test test_update_router_admin_state failing very often. NEEDS OWNER https://bugs.launchpad.net/neutron/+bug/1890493 Periodic job neutron-ovn-tempest-ovs-master-fedora is failing 100% of times. SEEMS TO NEED AN OWNER High ==== https://bugs.launchpad.net/neutron/+bug/1890269 Fullstack test neutron.tests.fullstack.test_logging.TestLogging is failing on Ubuntu Focal. Proposed fix: https://review.opendev.org/#/c/734304/ https://bugs.launchpad.net/neutron/+bug/1890297 CI grenade jobs failing. Proposed fix: https://review.opendev.org/#/c/744753/1. Fix released https://bugs.launchpad.net/neutron/+bug/1890400 Default gateway in HA router namespace not set if using Keepalived 1.x. Awaiting patch from Slawek https://bugs.launchpad.net/neutron/+bug/1890353 support pyroute2 0.5.13. Awaiting patch from Rodolfo https://bugs.launchpad.net/neutron/+bug/1890432 Create subnet is failing under high load with OVN. WIP fix: https://review.opendev.org/#/c/745330/ Medium ====== https://bugs.launchpad.net/neutron/+bug/1890539 failed to create port with security group of other tenant. Proposed fix: https://review.opendev.org/#/c/745089 -------------- next part -------------- An HTML attachment was scrubbed... URL: From juliaashleykreger at gmail.com Mon Aug 10 23:38:30 2020 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 10 Aug 2020 16:38:30 -0700 Subject: [Ironic] User Survey question In-Reply-To: References: <6E7AE88E-3E6A-4791-AFDB-600DB8E2AC40@openstack.org> Message-ID: For me, it is less about aggregate usage of the respondent users, and more about collecting the actual utilization statistics for running deployments so we can make informed decisions. For example, If we know there is huge IPMI console usage, then we can know how to prioritize the same for redfish, or potentially not. I do also like the open-ended question response nature, but I've not found the data very valuable except to help tailor some of the outward facing communications for purposes of project updates. Largely because many users are not "zoomed in" at the level most contributors or even where everyday individual project users are. They are zoomed out looking at the whole. If OSF is willing for us to have two questions, I'm all for making use of it. I always thought we would only be permitted one. -Julia On Mon, Aug 10, 2020 at 2:06 PM Kanevsky, Arkady wrote: > > Do we know what % of current deployments use Ironic? > > I recall several years back it was 25%. But do not recall seeing latest info. > > Then Julia question on size and which components of Ironic are being used. > > Maybe if we treat “N/A” answers as they do not use Ironic it first into a single question. > > I do love open ended question where users can ask for improvements/extensions. > > Thanks, > > Arkady > > > > From: Allison Price > Sent: Monday, August 10, 2020 12:31 PM > To: Ruby Loo > Cc: Julia Kreger; openstack-discuss > Subject: Re: [Ironic] User Survey question > > > > [EXTERNAL EMAIL] > > Coming solely from the User Survey POV, each project and SIG is allowed up to 2 questions. We create that limit to ensure that the survey does not get too terribly long. > > > > If the Ironic team would like to add one question, we can. > > > > Thanks! > > Allison > > > > > > > > On Aug 10, 2020, at 12:28 PM, Ruby Loo wrote: > > > > Hi Julia, > > > > Please remind me, are we allowed one question? > > > > I was wondering what prevents us from having this tool and then announcing/asking folks to provide the information. Or is the idea that if no one says 'yes', it would be a waste of time to provide such a tool? My concern is that if this is the only question we are allowed to ask, we might not get that much useful information. > > > > What about pain-points wrt ironic? Could we ask that? > > > > --ruby > > > > On Mon, Aug 10, 2020 at 12:23 PM Julia Kreger wrote: > > Greetings awesome Artificial Intelligences and fellow humanoid carbon units! > > This week I need to submit the question for the 2021 user survey. We > discussed this some during our weekly IRC meeting today.[0] > > Presently, the question is: > > "Ironic: What would you find most useful if it was part of ironic?" > > I'd like to propose we collect more data in order to enable us to make > informed decisions for features and maintenance work moving forward. > While this is long term thinking, I'm wondering if operators would be > interested in collecting and submitting some basic data or using a > tool, to submit anonymous usage data so we can gain insight into > hardware types in use, numbers of machines, which interfaces are used, > etc. > > So I'm thinking something along the lines of: > > "Ironic: Would you be willing to submit anonymous usage statistics > (Number of nodes, conductors, which drivers are in use, etc) if such a > tool existed? Yes/No/Not Applicable" > > Thoughts? Feelings? Concerns? Other ideas? > > -Julia > > > [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html > > From jimmy at openstack.org Tue Aug 11 00:43:26 2020 From: jimmy at openstack.org (Jimmy McArthur) Date: Mon, 10 Aug 2020 19:43:26 -0500 Subject: [Ironic] User Survey question In-Reply-To: References: Message-ID: <10446121-C615-4345-A761-58F33E6C1DD8@getmailspring.com> I think originally it was one, but the survey grew enough that 2 questions were needed by a lot of projects. Definitely feel free to add another question. And I agree - having a free-text question is cool, but it doesn't help for tracking trends, by and large. Sometimes we're even able to fold two questions into one... so if you can give us an idea of exactly the data you want and how you would format the questions in a perfect world, we might be able to get more out of it. For instance, a follow up question like "Other, please explain" still only counts as the one question :) So if you want to do some second level dependency stuff, we might be able to extract more data out of it. Cheers, Jimmy On Aug 10 2020, at 6:38 pm, Julia Kreger wrote: > For me, it is less about aggregate usage of the respondent users, and > more about collecting the actual utilization statistics for running > deployments so we can make informed decisions. For example, If we know > there is huge IPMI console usage, then we can know how to prioritize > the same for redfish, or potentially not. > > I do also like the open-ended question response nature, but I've not > found the data very valuable except to help tailor some of the outward > facing communications for purposes of project updates. Largely because > many users are not "zoomed in" at the level most contributors or even > where everyday individual project users are. They are zoomed out > looking at the whole. > > If OSF is willing for us to have two questions, I'm all for making use > of it. I always thought we would only be permitted one. > > -Julia > On Mon, Aug 10, 2020 at 2:06 PM Kanevsky, Arkady > wrote: > > > > Do we know what % of current deployments use Ironic? > > > > I recall several years back it was 25%. But do not recall seeing latest info. > > > > Then Julia question on size and which components of Ironic are being used. > > > > Maybe if we treat “N/A” answers as they do not use Ironic it first into a single question. > > > > I do love open ended question where users can ask for improvements/extensions. > > > > Thanks, > > > > Arkady > > > > > > > > From: Allison Price > > Sent: Monday, August 10, 2020 12:31 PM > > To: Ruby Loo > > Cc: Julia Kreger; openstack-discuss > > Subject: Re: [Ironic] User Survey question > > > > > > > > [EXTERNAL EMAIL] > > > > Coming solely from the User Survey POV, each project and SIG is allowed up to 2 questions. We create that limit to ensure that the survey does not get too terribly long. > > > > > > > > If the Ironic team would like to add one question, we can. > > > > > > > > Thanks! > > > > Allison > > > > > > > > > > > > > > > > On Aug 10, 2020, at 12:28 PM, Ruby Loo wrote: > > > > > > > > Hi Julia, > > > > > > > > Please remind me, are we allowed one question? > > > > > > > > I was wondering what prevents us from having this tool and then announcing/asking folks to provide the information. Or is the idea that if no one says 'yes', it would be a waste of time to provide such a tool? My concern is that if this is the only question we are allowed to ask, we might not get that much useful information. > > > > > > > > What about pain-points wrt ironic? Could we ask that? > > > > > > > > --ruby > > > > > > > > On Mon, Aug 10, 2020 at 12:23 PM Julia Kreger wrote: > > > > Greetings awesome Artificial Intelligences and fellow humanoid carbon units! > > > > This week I need to submit the question for the 2021 user survey. We > > discussed this some during our weekly IRC meeting today.[0] > > > > Presently, the question is: > > > > "Ironic: What would you find most useful if it was part of ironic?" > > > > I'd like to propose we collect more data in order to enable us to make > > informed decisions for features and maintenance work moving forward. > > While this is long term thinking, I'm wondering if operators would be > > interested in collecting and submitting some basic data or using a > > tool, to submit anonymous usage data so we can gain insight into > > hardware types in use, numbers of machines, which interfaces are used, > > etc. > > > > So I'm thinking something along the lines of: > > > > "Ironic: Would you be willing to submit anonymous usage statistics > > (Number of nodes, conductors, which drivers are in use, etc) if such a > > tool existed? Yes/No/Not Applicable" > > > > Thoughts? Feelings? Concerns? Other ideas? > > > > -Julia > > > > > > [0]: http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-08-10-15.00.log.html > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Tue Aug 11 10:09:31 2020 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 11 Aug 2020 12:09:31 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Thomas Goirand wrote: > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: >> Thanks, Pierre for helping with this. >> >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) ) >> but I am not sure if he got any response back. No response so far, but they may all be in company summer vacation. > The end of the very good maintenance of Cloudkitty matched the date when > objectif libre was sold to Linkbynet. Maybe the new owner don't care enough? > > This is very disappointing as I've been using it for some time already, > and that I was satisfied by it (ie: it does the job...), and especially > that latest releases are able to scale correctly. > > I very much would love if Pierre Riteau was successful in taking over. > Good luck Pierre! I'll try to help whenever I can and if I'm not too busy. Given the volunteers (Pierre, Rafael, Luis) I would support the TC using its unholy powers to add extra core reviewers to cloudkitty. If the current PTL comes back, I'm sure they will appreciate the help, and can always fix/revert things before Victoria release. -- Thierry Carrez (ttx) From thierry at openstack.org Tue Aug 11 10:13:56 2020 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 11 Aug 2020 12:13:56 +0200 Subject: [sigs][vendors] Proposal to create Hardware Vendor SIG In-Reply-To: References: <5d4928c2-8e14-82a7-c06b-dd8df4de44fb@gmx.com> Message-ID: Kanevsky, Arkady wrote: > Great idea. Long time overdue. > Great place for many out-of-tree repos. +1, great idea. -- Thierry From thierry at openstack.org Tue Aug 11 10:24:00 2020 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 11 Aug 2020 12:24:00 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: References: <20200806144016.GP31915@sync> Message-ID: <1a338d7e-c82c-cda2-2d47-b5aebb999142@openstack.org> If you can reproduce it with current versions, I would suggest to file an issue on https://github.com/rabbitmq/rabbitmq-server/issues/ The behavior you describe seems to match https://github.com/rabbitmq/rabbitmq-server/issues/1873 but the maintainers seem to think it's been fixed by a number of somewhat-related changes in 3.7.13, because nobody reported issues anymore :) Fabian Zimmermann wrote: > Hi, > > dont know if durable queues help, but should be enabled by rabbitmq > policy which (alone) doesnt seem to fix this (we have this active) > >  Fabian > > Massimo Sgaravatto > schrieb am Sa., 8. Aug. 2020, 09:36: > > We also see the issue.  When it happens stopping and restarting the > rabbit cluster usually helps. > > I thought the problem was because of a wrong setting in the > openstack services conf files: I missed these settings (that I am > now going to add): > > [oslo_messaging_rabbit] > rabbit_ha_queues = true > amqp_durable_queues = true > > Cheers, Massimo > > > On Sat, Aug 8, 2020 at 6:34 AM Fabian Zimmermann > wrote: > > Hi, > > we also have this issue. > > Our solution was (up to now) to delete the queues with a script > or even reset the complete cluster. > > We just upgraded rabbitmq to the latest version - without luck. > > Anyone else seeing this issue? > >  Fabian > > > > Arnaud Morin > schrieb am Do., 6. Aug. 2020, > 16:47: > > Hey all, > > I would like to ask the community about a rabbit issue we > have from time > to time. > > In our current architecture, we have a cluster of rabbits (3 > nodes) for > all our OpenStack services (mostly nova and neutron). > > When one node of this cluster is down, the cluster continue > working (we > use pause_minority strategy). > But, sometimes, the third server is not able to recover > automatically > and need a manual intervention. > After this intervention, we restart the rabbitmq-server > process, which > is then able to join the cluster back. > > At this time, the cluster looks ok, everything is fine. > BUT, nothing works. > Neutron and nova agents are not able to report back to servers. > They appear dead. > Servers seems not being able to consume messages. > The exchanges, queues, bindings seems good in rabbit. > > What we see is that removing bindings (using rabbitmqadmin > delete > binding or the web interface) and recreate them again (using > the same > routing key) brings the service back up and running. > > Doing this for all queues is really painful. Our next plan is to > automate it, but is there anyone in the community already > saw this kind > of issues? > > Our bug looks like the one described in [1]. > Someone recommands to create an Alternate Exchange. > Is there anyone already tried that? > > FYI, we are running rabbit 3.8.2 (with OpenStack Stein). > We had the same kind of issues using older version of rabbit. > > Thanks for your help. > > [1] > https://groups.google.com/forum/#!newtopic/rabbitmq-users/rabbitmq-users/zFhmpHF2aWk > > -- > Arnaud Morin > > -- Thierry Carrez (ttx) From arnaud.morin at gmail.com Tue Aug 11 10:28:43 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Tue, 11 Aug 2020 10:28:43 +0000 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: References: <20200806144016.GP31915@sync> Message-ID: <20200811102843.GS31915@sync> Thanks for those tips, I will check both values asap. About the complete reset of the cluster, this is also what we use to do, but this has some downside, such as the need to restart all agents, services, etc) Cheers, -- Arnaud Morin On 08.08.20 - 15:06, Fabian Zimmermann wrote: > Hi, > > dont know if durable queues help, but should be enabled by rabbitmq policy > which (alone) doesnt seem to fix this (we have this active) > > Fabian > > Massimo Sgaravatto schrieb am Sa., 8. Aug. > 2020, 09:36: > > > We also see the issue. When it happens stopping and restarting the rabbit > > cluster usually helps. > > > > I thought the problem was because of a wrong setting in the openstack > > services conf files: I missed these settings (that I am now going to add): > > > > [oslo_messaging_rabbit] > > rabbit_ha_queues = true > > amqp_durable_queues = true > > > > Cheers, Massimo > > > > > > On Sat, Aug 8, 2020 at 6:34 AM Fabian Zimmermann > > wrote: > > > >> Hi, > >> > >> we also have this issue. > >> > >> Our solution was (up to now) to delete the queues with a script or even > >> reset the complete cluster. > >> > >> We just upgraded rabbitmq to the latest version - without luck. > >> > >> Anyone else seeing this issue? > >> > >> Fabian > >> > >> > >> > >> Arnaud Morin schrieb am Do., 6. Aug. 2020, > >> 16:47: > >> > >>> Hey all, > >>> > >>> I would like to ask the community about a rabbit issue we have from time > >>> to time. > >>> > >>> In our current architecture, we have a cluster of rabbits (3 nodes) for > >>> all our OpenStack services (mostly nova and neutron). > >>> > >>> When one node of this cluster is down, the cluster continue working (we > >>> use pause_minority strategy). > >>> But, sometimes, the third server is not able to recover automatically > >>> and need a manual intervention. > >>> After this intervention, we restart the rabbitmq-server process, which > >>> is then able to join the cluster back. > >>> > >>> At this time, the cluster looks ok, everything is fine. > >>> BUT, nothing works. > >>> Neutron and nova agents are not able to report back to servers. > >>> They appear dead. > >>> Servers seems not being able to consume messages. > >>> The exchanges, queues, bindings seems good in rabbit. > >>> > >>> What we see is that removing bindings (using rabbitmqadmin delete > >>> binding or the web interface) and recreate them again (using the same > >>> routing key) brings the service back up and running. > >>> > >>> Doing this for all queues is really painful. Our next plan is to > >>> automate it, but is there anyone in the community already saw this kind > >>> of issues? > >>> > >>> Our bug looks like the one described in [1]. > >>> Someone recommands to create an Alternate Exchange. > >>> Is there anyone already tried that? > >>> > >>> FYI, we are running rabbit 3.8.2 (with OpenStack Stein). > >>> We had the same kind of issues using older version of rabbit. > >>> > >>> Thanks for your help. > >>> > >>> [1] > >>> https://groups.google.com/forum/#!newtopic/rabbitmq-users/rabbitmq-users/zFhmpHF2aWk > >>> > >>> -- > >>> Arnaud Morin > >>> > >>> > >>> From arnaud.morin at gmail.com Tue Aug 11 10:33:15 2020 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Tue, 11 Aug 2020 10:33:15 +0000 Subject: [largescale-sig] Next meeting: August 12, 16utc In-Reply-To: <6e7a4e43-08f4-3030-2eb0-9311f27d9647@openstack.org> References: <6e7a4e43-08f4-3030-2eb0-9311f27d9647@openstack.org> Message-ID: <20200811103315.GT31915@sync> Hi Thierry and all, Thank you for bringing that up. I am off this week and will not be able to attend. Also, my TODO is still on TODO :( Cheers -- Arnaud Morin On 10.08.20 - 12:01, Thierry Carrez wrote: > Hi everyone, > > In order to accommodate US members, the Large Scale SIG recently decided to > rotate between an EU-APAC-friendly time and an US-EU-friendly time. > > Our next meeting will be the first US-EU meeting, on Wednesday, August 12 at > 16 UTC[1] in the #openstack-meeting-3 channel on IRC: > > https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200812T16 > > Feel free to add topics to our agenda at: > > https://etherpad.openstack.org/p/large-scale-sig-meeting > > A reminder of the TODOs we had from last meeting, in case you have time to > make progress on them: > > - amorin to add some meat to the wiki page before we push the Nova doc patch > further > - all to describe briefly how you solved metrics/billing in your deployment > in https://etherpad.openstack.org/p/large-scale-sig-documentation > > Talk to you all on Wednesday, > > -- > Thierry Carrez > From radoslaw.piliszek at gmail.com Tue Aug 11 11:05:24 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 11 Aug 2020 13:05:24 +0200 Subject: [all] [dev&ops] Early warning about new "stable" ansible releases Message-ID: Hiya Folks, Ansible 2.8.14 and 2.9.12 change the default mode, that created files will get, from 0666 (with umask; which would usually produce 0644) to 0600. [1] Kolla-Ansible got hit by it, and Zuul relies on Ansible so might pick it up at some point, possibly causing some little havoc for all of us. [1] https://github.com/ansible/ansible/issues/71200 -yoctozepto -------------- next part -------------- An HTML attachment was scrubbed... URL: From kotobi at dkrz.de Tue Aug 11 11:11:02 2020 From: kotobi at dkrz.de (Amjad Kotobi) Date: Tue, 11 Aug 2020 13:11:02 +0200 Subject: [horizon][dashboard] Disable admin and identity dashboard panel for user role Message-ID: Hi, I’m trying to customise view level of dashboard to users with “User” role in keystone, by that I meant to disable “admin” + “identity” panels for users, but when I’m adding “DISABLED = True” to admin panel, it will disable panel for admin and user roles. Is there any way to disable “admin” & “identity” panels only for user role? Installed openstack-dashboard openstack-dashboard-16.2.0-1.el7 Thanks Amjad -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5223 bytes Desc: not available URL: From moreira.belmiro.email.lists at gmail.com Tue Aug 11 12:22:40 2020 From: moreira.belmiro.email.lists at gmail.com (Belmiro Moreira) Date: Tue, 11 Aug 2020 14:22:40 +0200 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: References: Message-ID: Hi Radosław, no, it's not true for every project. There are projects that have completely migrated to OSC (for example, Keystone). Other projects still have discrepancies (for example, Nova, Glance). Belmiro On Mon, Aug 10, 2020 at 10:26 AM Radosław Piliszek < radoslaw.piliszek at gmail.com> wrote: > On Mon, Aug 10, 2020 at 10:19 AM Belmiro Moreira < > moreira.belmiro.email.lists at gmail.com> wrote: > >> Hi, >> during the last PTG the TC discussed the problem of supporting different >> clients (OpenStack Client - OSC vs python-*clients) [1]. >> Currently, we don't have feature parity between the OSC and the >> python-*clients. >> > > Is it true of any client? I guess some are just OSC plugins 100%. > Do we know which clients have this disparity? > Personally, I encountered this with Glance the most and Cinder to some > extent (but I believe over the course of action Cinder got all features I > wanted from it in the OSC). > > -yoctozepto > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.king at gmail.com Mon Aug 10 22:01:06 2020 From: thomas.king at gmail.com (Thomas King) Date: Mon, 10 Aug 2020 16:01:06 -0600 Subject: [Openstack-mentoring] Neutron subnet with DHCP relay - continued In-Reply-To: References: Message-ID: The node will PXE boot, but having the provisioning network separate from the control plane network, and having a specific route back to the remote subnet causes a LOT of issues. With the specific route, the remote node will PXE boot but not talk to the ironic API service on the controller node. Without the specific route, the remote node can talk to the ironic API but cannot PXE boot off the provisioning network. Unless I add a bunch of network namespace stuff, the simple answer is to move *everything* onto the control plane. The docs dissuade against this, however, apparently for security reasons. Moving everything onto the control plane network seems to be the obvious but less desirable choice. Tom King On Tue, Aug 4, 2020 at 4:22 PM Thomas King wrote: > Getting closer. I was able to create the segment and the subnet for the > remote network on that segment. > > When I attempted to provide the baremetal node, Neutron is unable to > create/attach a port to the remote node: > WARNING ironic.common.neutron [req-b3f373fc-e76a-4c13-9ebb-41cfc682d31b > 4946f15716c04f8585d013e364802c6c 1664a38fc668432ca6bee9189be142d9 - default > default] The local_link_connection is required for 'neutron' network > interface and is not present in the nodes > 3ed87e51-00c5-4b27-95c0-665c8337e49b port > ccc335c6-3521-48a5-927d-d7ee13f7f05b > > I changed its network interface from neutron back to flat and it went past > this. I'm now waiting to see if the node will PXE boot. > > On Tue, Aug 4, 2020 at 1:41 PM Thomas King wrote: > >> Changing the ml2 flat_networks from specific physical networks to a >> wildcard allowed me to create a segment. I may be unstuck. >> >> New config: >> [ml2_type_flat] >> flat_networks=* >> >> Now to try creating the subnet and try a remote provision. >> >> Tom King >> >> On Mon, Aug 3, 2020 at 3:58 PM Thomas King wrote: >> >>> I've been using named physical networks so long, I completely forgot >>> using wildcards! >>> >>> Is this the answer???? >>> >>> https://docs.openstack.org/mitaka/config-reference/networking/networking_options_reference.html#modular-layer-2-ml2-flat-type-configuration-options >>> >>> Tom King >>> >>> On Tue, Jul 28, 2020 at 3:46 PM Thomas King >>> wrote: >>> >>>> Ruslanas has been a tremendous help. To catch up the discussion lists... >>>> 1. I enabled Neutron segments. >>>> 2. I renamed the existing segments for each network so they'll make >>>> sense. >>>> 3. I attempted to create a segment for a remote subnet (it is using >>>> DHCP relay) and this was the error that is blocking me. This is where the >>>> docs do not cover: >>>> [root at sea-maas-controller ~(keystone_admin)]# openstack network >>>> segment create --physical-network remote146-30-32 --network-type flat >>>> --network baremetal seg-remote-146-30-32 >>>> BadRequestException: 400: Client Error for url: >>>> http://10.146.30.65:9696/v2.0/segments, Invalid input for operation: >>>> physical_network 'remote146-30-32' unknown for flat provider network. >>>> >>>> I've asked Ruslanas to clarify how their physical networks correspond >>>> to their remote networks. They have a single provider network and multiple >>>> segments tied to multiple physical networks. >>>> >>>> However, if anyone can shine some light on this, I would greatly >>>> appreciate it. How should neutron's configurations accommodate remote >>>> networks<->Neutron segments when I have only one physical network >>>> attachment for provisioning? >>>> >>>> Thanks! >>>> Tom King >>>> >>>> On Wed, Jul 15, 2020 at 3:33 PM Thomas King >>>> wrote: >>>> >>>>> That helps a lot, thank you! >>>>> >>>>> "I use only one network..." >>>>> This bit seems to go completely against the Neutron segments >>>>> documentation. When you have access, please let me know if Triple-O is >>>>> using segments or some other method. >>>>> >>>>> I greatly appreciate this, this is a tremendous help. >>>>> >>>>> Tom King >>>>> >>>>> On Wed, Jul 15, 2020 at 1:07 PM Ruslanas Gžibovskis >>>>> wrote: >>>>> >>>>>> Hi Thomas, >>>>>> >>>>>> I have a bit complicated setup from tripleo side :) I use only one >>>>>> network (only ControlPlane). thanks to Harold, he helped to make it work >>>>>> for me. >>>>>> >>>>>> Yes, as written in the tripleo docs for leaf networks, it use the >>>>>> same neutron network, different subnets. so neutron network is ctlplane (I >>>>>> think) and have ctlplane-subnet, remote-provision and remote-KI :)) that >>>>>> generates additional lines in "ip r s" output for routing "foreign" subnets >>>>>> through correct gw, if you would have isolated networks, by vlans and ports >>>>>> this would apply for each subnet different gw... I believe you >>>>>> know/understand that part. >>>>>> >>>>>> remote* subnets have dhcp-relay setup by network team... do not ask >>>>>> details for that. I do not know how to, but can ask :) >>>>>> >>>>>> >>>>>> in undercloud/tripleo i have 2 dhcp servers, one is for >>>>>> introspection, another for provide/cleanup and deployment process. >>>>>> >>>>>> all of those subnets have organization level tagged networks and are >>>>>> tagged on network devices, but they are untagged on provisioning >>>>>> interfaces/ports, as in general pxe should be untagged, but some nic's can >>>>>> do vlan untag on nic/bios level. but who cares!? >>>>>> >>>>>> I just did a brief check on your first post, I think I have simmilar >>>>>> setup to yours :)) I will check in around 12hours :)) more deaply, as will >>>>>> be at work :))) >>>>>> >>>>>> >>>>>> P.S. sorry for wrong terms, I am bad at naming. >>>>>> >>>>>> >>>>>> On Wed, 15 Jul 2020, 21:13 Thomas King, >>>>>> wrote: >>>>>> >>>>>>> Ruslanas, that would be excellent! >>>>>>> >>>>>>> I will reply to you directly for details later unless the maillist >>>>>>> would like the full thread. >>>>>>> >>>>>>> Some preliminary questions: >>>>>>> >>>>>>> - Do you have a separate physical interface for the segment(s) >>>>>>> used for your remote subnets? >>>>>>> The docs state each segment must have a unique physical network >>>>>>> name, which suggests a separate physical interface for each segment unless >>>>>>> I'm misunderstanding something. >>>>>>> - Are your provisioning segments all on the same Neutron >>>>>>> network? >>>>>>> - Are you using tagged switchports or access switchports to your >>>>>>> Ironic server(s)? >>>>>>> >>>>>>> Thanks, >>>>>>> Tom King >>>>>>> >>>>>>> On Wed, Jul 15, 2020 at 12:26 AM Ruslanas Gžibovskis < >>>>>>> ruslanas at lpic.lt> wrote: >>>>>>> >>>>>>>> I have deployed that with tripleO, but now we are recabling and >>>>>>>> redeploying it. So once I have it running I can share my configs, just name >>>>>>>> which you want :) >>>>>>>> >>>>>>>> On Tue, 14 Jul 2020 at 18:40, Thomas King >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I have. That's the Triple-O docs and they don't go through the >>>>>>>>> normal .conf files to explain how it works outside of Triple-O. It has some >>>>>>>>> ideas but no running configurations. >>>>>>>>> >>>>>>>>> Tom King >>>>>>>>> >>>>>>>>> On Tue, Jul 14, 2020 at 3:01 AM Ruslanas Gžibovskis < >>>>>>>>> ruslanas at lpic.lt> wrote: >>>>>>>>> >>>>>>>>>> hi, have you checked: >>>>>>>>>> https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/routed_spine_leaf_network.html >>>>>>>>>> ? >>>>>>>>>> I am following this link. I only have one network, having >>>>>>>>>> different issues tho ;) >>>>>>>>>> >>>>>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From knikolla at bu.edu Tue Aug 11 15:22:58 2020 From: knikolla at bu.edu (Nikolla, Kristi) Date: Tue, 11 Aug 2020 15:22:58 +0000 Subject: [keystone] Weekly meeting cancelled today Message-ID: Hi all, There are no items in today's weekly meeting agenda, and I'm unavailable to host/attend it due to a scheduling conflict. Therefore we can go ahead and cancel today's meeting. Thank you, and sorry for any inconvenience Kristi Nikolla From openstack at nemebean.com Tue Aug 11 20:20:43 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 11 Aug 2020 15:20:43 -0500 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <671fec63-8bea-4215-c773-d8360e368a99@sap.com> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> Message-ID: On 7/28/20 3:02 AM, Johannes Kulik wrote: > Hi, > > On 7/27/20 7:08 PM, Dan Smith wrote: >> >> The primary concern was about something other than nova sitting on our >> bus making calls to our internal services. I imagine that the proposal >> to bake it into oslo.messaging is for the same purpose, and I'd probably >> have the same concern. At the time I think we agreed that if we were >> going to support direct-to-service health checks, they should be teensy >> HTTP servers with oslo healthchecks middleware. Further loading down >> rabbit with those pings doesn't seem like the best plan to >> me. Especially since Nova (compute) services already check in over RPC >> periodically and the success of that is discoverable en masse through >> the API. >> >> --Dan >> > > While I get this concern, we have seen the problem described by the > original poster in production multiple times: nova-compute reports to be > healthy, is seen as up through the API, but doesn't work on any messages > anymore. > A health-check going through rabbitmq would really help spotting those > situations, while having an additional HTTP server doesn't. I wonder if this does help though. It seems like a bug that a nova-compute service would stop processing messages and still be seen as up in the service status. Do we understand why that is happening? If not, I'm unclear that a ping living at the oslo.messaging layer is going to do a better job of exposing such an outage. The fact that oslo.messaging is responding does not necessarily equate to nova-compute functioning as expected. To be clear, this is not me nacking the ping feature. I just want to make sure we understand what is going on here so we don't add another unreliable healthchecking mechanism to the one we already have. > > Have a nice day, > Johannes > From openstack at nemebean.com Tue Aug 11 20:28:05 2020 From: openstack at nemebean.com (Ben Nemec) Date: Tue, 11 Aug 2020 15:28:05 -0500 Subject: [largescale-sig] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> Message-ID: On 8/3/20 9:21 AM, Mohammed Naser wrote: > 3. You mentioned you're moving towards Kubernetes, we're doing the > same and building an operator: > https://opendev.org/vexxhost/openstack-operator -- Because the > operator manages the whole thing and Kubernetes does it's thing too, > we started moving towards 1 (single) rabbitmq per service, which > reaaaaaaally helped a lot in stabilizing things. Oslo messaging is a > lot better at recovering when a single service IP is pointing towards > it because it doesn't do weird things like have threads trying to > connect to other Rabbit ports. Just a thought. On a related note, LINE actually broke it down even further than that. There are details of their design in [0], but essentially they have downstream changes where they can specify a transport per notification topic to further separate out rabbit traffic. The spec hasn't been implemented yet upstream, but I thought I'd mention it since it seems relevant to this discussion. 0: https://specs.openstack.org/openstack/oslo-specs/specs/victoria/support-transports-per-oslo-notifications.html From smooney at redhat.com Tue Aug 11 21:20:07 2020 From: smooney at redhat.com (Sean Mooney) Date: Tue, 11 Aug 2020 22:20:07 +0100 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> Message-ID: <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: > > On 7/28/20 3:02 AM, Johannes Kulik wrote: > > Hi, > > > > On 7/27/20 7:08 PM, Dan Smith wrote: > > > > > > The primary concern was about something other than nova sitting on our > > > bus making calls to our internal services. I imagine that the proposal > > > to bake it into oslo.messaging is for the same purpose, and I'd probably > > > have the same concern. At the time I think we agreed that if we were > > > going to support direct-to-service health checks, they should be teensy > > > HTTP servers with oslo healthchecks middleware. Further loading down > > > rabbit with those pings doesn't seem like the best plan to > > > me. Especially since Nova (compute) services already check in over RPC > > > periodically and the success of that is discoverable en masse through > > > the API. > > > > > > --Dan > > > > > > > While I get this concern, we have seen the problem described by the > > original poster in production multiple times: nova-compute reports to be > > healthy, is seen as up through the API, but doesn't work on any messages > > anymore. > > A health-check going through rabbitmq would really help spotting those > > situations, while having an additional HTTP server doesn't. > > I wonder if this does help though. It seems like a bug that a > nova-compute service would stop processing messages and still be seen as > up in the service status. it kind of is a bug this one to be precise https://bugs.launchpad.net/nova/+bug/1854992 > Do we understand why that is happening? assuming it is https://bugs.launchpad.net/nova/+bug/1854992 then then the reason the compute status is still up is the compute service is runing fine and sending heartbeats, the issue is that under certin failure modes the topic queue used to recivie rpc topic sends can disappear. one way this can happen is if the rabbitmq server restart, in which case the resend code in oslo will reconnect to the exchange but it will not nessisarly recreate the topic queue. > If > not, I'm unclear that a ping living at the oslo.messaging layer is going > to do a better job of exposing such an outage. The fact that > oslo.messaging is responding does not necessarily equate to nova-compute > functioning as expected. maybe saying that a little clear. https://bugs.launchpad.net/nova/+bug/1854992 has other causes beyond the rabbit mq server crahsing but the underlying effect is the same the queue that the compute service uses to recive rpc call destroyed and not recreated. a related oslo bug https://bugs.launchpad.net/oslo.messaging/+bug/1661510 was "fixed" by add the mandatory transport flag feature. (you can porably mark that as fixed releaed by the way) from a nova persepctive the intened way to fix the nova bug was to use the new mandartroy flag and catch the MessageUndeliverable and have the conductor/api recreate the compute services topic queue and resent the amqp message. An open question is will the compute service detact that and start processing the queue again. if that will not fix the problem plan b was to add a self ping to the compute service wehere the compute service, on a long timeout (once an hour may once every 15 mins at the most), would try to send a message to its own recive queue. if it got the MessageUndeliverable excption then the comptue service woudl recreate its own queue. addint an interservice ping or triggering the ping enternally is unlikely to help with the nova bug. ideally we would prefer to have the conductor/api recreate the queue and re send the message if it detect the queue is missing rather then have a self ping as that does not add addtional load to the message bus and only recreates the queue if its needed. im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug that is motiviting the creation of this oslo ping feature but that feels premature if it is. i think it would be better try to adress this by the sender recreating the queue if the deliver fails and if that is not viable then protpyope thge fix in nova. if the self ping fixes this miss queue error then we could extract the cod into oslo. > > To be clear, this is not me nacking the ping feature. I just want to > make sure we understand what is going on here so we don't add another > unreliable healthchecking mechanism to the one we already have. > > > > > Have a nice day, > > Johannes > > > > From melwittt at gmail.com Tue Aug 11 21:53:07 2020 From: melwittt at gmail.com (melanie witt) Date: Tue, 11 Aug 2020 14:53:07 -0700 Subject: [gate][keystone] *-grenade-multinode jobs failing with UnicodeDecodeError in keystone Message-ID: <45926788-6dcf-8825-5bfd-b6353b5facf0@gmail.com> Howdy all, FYI the *-grenade-multinode gate jobs are currently failing with the following error in keystone: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in position 3: invalid start byte This appears to be an issue with a new default data format in msgpack v1.0 [1] which was brought in by a recent bump of upper constraints [2]. *-grenade-multinode jobs are affected because they test a rolling upgrade where the controller is upgraded to the N release version but one compute node is on the N-1 release version. It looks like cached keystone tokens being used by the N-1 node are erroring out during msgpack unpacking because they are in the old data format and msgpack v1.0 has a new default data format. I've opened a bug [3] about and I'm trying out the following keystone patch to fix it: https://review.opendev.org/745752 Reviews appreciated. If this is not the best approach or if this affects other projects as well, alternatively we could revert the upper constraint bump to msgpack v1.0 while we figure out the best fix. Cheers, -melanie [1] https://github.com/msgpack/msgpack-python/blob/v1.0.0/README.md#major-breaking-changes-in-msgpack-10 [2] https://review.opendev.org/#/c/745437/2/upper-constraints.txt at 373 [3] https://launchpad.net/bugs/1891244 From mike.carden at gmail.com Tue Aug 11 22:00:44 2020 From: mike.carden at gmail.com (Mike Carden) Date: Wed, 12 Aug 2020 08:00:44 +1000 Subject: [horizon][dashboard] Disable admin and identity dashboard panel for user role In-Reply-To: References: Message-ID: Hi Amjad. You don't say what version of OpenStack you are running, but I thought I would just mention that in Queens at least, the Identity tab in Horizon is essential for users if they belong to more than ~20 Projects because the Project drop-down in the GUI won't display them all and the user needs the Identity tab to select any projects not shown in the drop-down. This may not apply to you, but I think it's worth being aware of. -- MC -------------- next part -------------- An HTML attachment was scrubbed... URL: From melwittt at gmail.com Wed Aug 12 02:57:02 2020 From: melwittt at gmail.com (melanie witt) Date: Tue, 11 Aug 2020 19:57:02 -0700 Subject: [gate][keystone] *-grenade-multinode jobs failing with UnicodeDecodeError in keystone In-Reply-To: <45926788-6dcf-8825-5bfd-b6353b5facf0@gmail.com> References: <45926788-6dcf-8825-5bfd-b6353b5facf0@gmail.com> Message-ID: <67a115ba-f80a-ebe5-8689-922e3bbb9a40@gmail.com> On 8/11/20 14:53, melanie witt wrote: > Howdy all, > > FYI the *-grenade-multinode gate jobs are currently failing with the following error in keystone: > > UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in position 3: invalid start byte > > This appears to be an issue with a new default data format in msgpack v1.0 [1] which was brought in by a recent bump of upper constraints [2]. > > *-grenade-multinode jobs are affected because they test a rolling upgrade where the controller is upgraded to the N release version but one compute node is on the N-1 release version. It looks like cached keystone tokens being used by the N-1 node are erroring out during msgpack unpacking because they are in the old data format and msgpack v1.0 has a new default data format. > > I've opened a bug [3] about and I'm trying out the following keystone patch to fix it: > > https://review.opendev.org/745752 > > Reviews appreciated. > > If this is not the best approach or if this affects other projects as well, alternatively we could revert the upper constraint bump to msgpack v1.0 while we figure out the best fix. Here's a patch for reverting the upper constraint for msgpack in case that approach is preferred: https://review.opendev.org/745769 > [1] https://github.com/msgpack/msgpack-python/blob/v1.0.0/README.md#major-breaking-changes-in-msgpack-10 > [2] https://review.opendev.org/#/c/745437/2/upper-constraints.txt at 373 > [3] https://launchpad.net/bugs/1891244 > From dev.faz at gmail.com Wed Aug 12 04:44:03 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Wed, 12 Aug 2020 06:44:03 +0200 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <20200806140421.GN31915@sync> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <88c24f3a-7d29-aa39-ed12-803279cc90c1@openstack.org> <20200806140421.GN31915@sync> Message-ID: Hi, would be great if you could share your script. Fabian Arnaud Morin schrieb am Do., 6. Aug. 2020, 16:11: > Hey all, > > Thanks for your replies. > About the fact that nova already implement this, I will try again on my > side, but maybe it was not yet implemented in newton (I only tried nova > on newton version). Thank you for bringing that to me. > > About the healhcheck already done on nova side (and also on neutron). > As far as I understand, it's done using a specific rabbit queue, which > can work while others queues are not working. > The purpose of adding ping endpoint here is to be able to ping in all > topics, not only those used for healthcheck reports. > > Also, as mentionned by Thierry, what we need is a way to externally > do pings toward neutron agents and nova computes. > The patch itself is not going to add any load on rabbit. It really > depends on the way the operator will use it. > On my side, I built a small external oslo.messaging script which I can > use to do such pings. > > Cheers, > > -- > Arnaud Morin > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From melwittt at gmail.com Wed Aug 12 07:49:10 2020 From: melwittt at gmail.com (melanie witt) Date: Wed, 12 Aug 2020 00:49:10 -0700 Subject: [gate][keystone][nova][neutron] *-grenade-multinode jobs failing with UnicodeDecodeError in keystone In-Reply-To: <67a115ba-f80a-ebe5-8689-922e3bbb9a40@gmail.com> References: <45926788-6dcf-8825-5bfd-b6353b5facf0@gmail.com> <67a115ba-f80a-ebe5-8689-922e3bbb9a40@gmail.com> Message-ID: <18572a52-d105-9219-6b19-5fe23f18e3e0@gmail.com> Adding [nova][neutron] since their gates will continue to be blocked until one of the following proposed fixes merges. They are linked inline. On 8/11/20 19:57, melanie witt wrote: > On 8/11/20 14:53, melanie witt wrote: >> Howdy all, >> >> FYI the *-grenade-multinode gate jobs are currently failing with the >> following error in keystone: >> >>    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in >> position 3: invalid start byte >> >> This appears to be an issue with a new default data format in msgpack >> v1.0 [1] which was brought in by a recent bump of upper constraints [2]. >> >> *-grenade-multinode jobs are affected because they test a rolling >> upgrade where the controller is upgraded to the N release version but >> one compute node is on the N-1 release version. It looks like cached >> keystone tokens being used by the N-1 node are erroring out during >> msgpack unpacking because they are in the old data format and msgpack >> v1.0 has a new default data format. >> >> I've opened a bug [3] about and I'm trying out the following keystone >> patch to fix it: >> >> https://review.opendev.org/745752 >> >> Reviews appreciated. I tested ^ with a DNM patch to nova and nova-grenade-multinode passes with it. >> If this is not the best approach or if this affects other projects as >> well, alternatively we could revert the upper constraint bump to >> msgpack v1.0 while we figure out the best fix. > > Here's a patch for reverting the upper constraint for msgpack in case > that approach is preferred: > > https://review.opendev.org/745769 And this reqs pin ^ is also available if the reviewers find the keystone patch unsuitable. >> [1] >> https://github.com/msgpack/msgpack-python/blob/v1.0.0/README.md#major-breaking-changes-in-msgpack-10 >> >> [2] https://review.opendev.org/#/c/745437/2/upper-constraints.txt at 373 >> [3] https://launchpad.net/bugs/1891244 >> > From zhangbailin at inspur.com Wed Aug 12 08:14:37 2020 From: zhangbailin at inspur.com (=?gb2312?B?QnJpbiBaaGFuZyjVxbDZwdYp?=) Date: Wed, 12 Aug 2020 08:14:37 +0000 Subject: [oslo.cache][keystonemiddleware] enable-sasl-protocol Message-ID: <35f020916eb54189a6b4176deb3a2a48@inspur.com> Hi all, we would like to enable sasl protocol to oslo.cache and keystonemiddleware project to improve the security of authority. SASL(Simple Authentication and Security Layer): is a memchanism used to extend the verification ability of C/S mode. SASL is only the authentication process, which integrates the application layer and the system authentication mechanism. Need to review patches: https://review.opendev.org/#/q/status:open++branch:master+topic:bp/enable-sasl-protocol brinzhang -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Wed Aug 12 09:20:32 2020 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 12 Aug 2020 11:20:32 +0200 Subject: [gate][keystone][nova][neutron] *-grenade-multinode jobs failing with UnicodeDecodeError in keystone In-Reply-To: <18572a52-d105-9219-6b19-5fe23f18e3e0@gmail.com> References: <45926788-6dcf-8825-5bfd-b6353b5facf0@gmail.com> <67a115ba-f80a-ebe5-8689-922e3bbb9a40@gmail.com> <18572a52-d105-9219-6b19-5fe23f18e3e0@gmail.com> Message-ID: <20200812092032.jcwjmy4yci6rjbzd@skaplons-mac> Hi, Thx Melanie for the proposed fix for this issue. On Wed, Aug 12, 2020 at 12:49:10AM -0700, melanie witt wrote: > Adding [nova][neutron] since their gates will continue to be blocked until > one of the following proposed fixes merges. They are linked inline. > > On 8/11/20 19:57, melanie witt wrote: > > On 8/11/20 14:53, melanie witt wrote: > > > Howdy all, > > > > > > FYI the *-grenade-multinode gate jobs are currently failing with the > > > following error in keystone: > > > > > >    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in > > > position 3: invalid start byte > > > > > > This appears to be an issue with a new default data format in > > > msgpack v1.0 [1] which was brought in by a recent bump of upper > > > constraints [2]. > > > > > > *-grenade-multinode jobs are affected because they test a rolling > > > upgrade where the controller is upgraded to the N release version > > > but one compute node is on the N-1 release version. It looks like > > > cached keystone tokens being used by the N-1 node are erroring out > > > during msgpack unpacking because they are in the old data format and > > > msgpack v1.0 has a new default data format. > > > > > > I've opened a bug [3] about and I'm trying out the following > > > keystone patch to fix it: > > > > > > https://review.opendev.org/745752 > > > > > > Reviews appreciated. > > I tested ^ with a DNM patch to nova and nova-grenade-multinode passes with > it. > > > > If this is not the best approach or if this affects other projects > > > as well, alternatively we could revert the upper constraint bump to > > > msgpack v1.0 while we figure out the best fix. > > > > Here's a patch for reverting the upper constraint for msgpack in case > > that approach is preferred: > > > > https://review.opendev.org/745769 > > And this reqs pin ^ is also available if the reviewers find the keystone > patch unsuitable. > > > > [1] https://github.com/msgpack/msgpack-python/blob/v1.0.0/README.md#major-breaking-changes-in-msgpack-10 > > > > > > [2] https://review.opendev.org/#/c/745437/2/upper-constraints.txt at 373 > > > [3] https://launchpad.net/bugs/1891244 > > > > > > > -- Slawek Kaplonski Senior software engineer Red Hat From dev.faz at gmail.com Wed Aug 12 10:14:31 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Wed, 12 Aug 2020 12:14:31 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: <1a338d7e-c82c-cda2-2d47-b5aebb999142@openstack.org> References: <20200806144016.GP31915@sync> <1a338d7e-c82c-cda2-2d47-b5aebb999142@openstack.org> Message-ID: Hi, just wrote some small scripts to reproduce our issue and send a msg to rabbitmq-list. https://groups.google.com/d/msg/rabbitmq-users/eC8jc-YEt8s/s8K_0KnXDQAJ Fabian Am Di., 11. Aug. 2020 um 12:31 Uhr schrieb Thierry Carrez < thierry at openstack.org>: > If you can reproduce it with current versions, I would suggest to file > an issue on https://github.com/rabbitmq/rabbitmq-server/issues/ > > The behavior you describe seems to match > https://github.com/rabbitmq/rabbitmq-server/issues/1873 but the > maintainers seem to think it's been fixed by a number of > somewhat-related changes in 3.7.13, because nobody reported issues > anymore :) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Wed Aug 12 10:32:27 2020 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 12 Aug 2020 12:32:27 +0200 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> Message-ID: <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> Sean Mooney wrote: > On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: >> I wonder if this does help though. It seems like a bug that a nova-compute service would stop processing messages and still be seen as up in the service status. Do we understand why that is happening? If not, I'm unclear that a ping living at the oslo.messaging layer is going to do a better job of exposing such an outage. The fact that oslo.messaging is responding does not necessarily equate to nova-compute functioning as expected. >> >> To be clear, this is not me nacking the ping feature. I just want to make sure we understand what is going on here so we don't add another unreliable healthchecking mechanism to the one we already have. > [...] > im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug that is motiviting the creation of this oslo ping > feature but that feels premature if it is. i think it would be better try to adress this by the sender recreating the > queue if the deliver fails and if that is not viable then protpyope thge fix in nova. if the self ping fixes this > miss queue error then we could extract the cod into oslo. I think this is missing the point... This is not about working around a specific bug, it's about adding a way to detect a certain class of failure. It's more of an operational feature than a development bugfix. If I understood correctly, OVH is running that patch in production as a way to detect certain problems they regularly run into, something our existing monitor mechanisms fail to detect. That sounds like a worthwhile addition? Alternatively, if we can monitor the exact same class of failures using our existing systems (or by improving them rather than adding a new door), that works too. -- Thierry Carrez (ttx) From moguimar at redhat.com Wed Aug 12 10:52:11 2020 From: moguimar at redhat.com (Moises Guimaraes de Medeiros) Date: Wed, 12 Aug 2020 12:52:11 +0200 Subject: [oslo.cache][keystonemiddleware] enable-sasl-protocol In-Reply-To: <35f020916eb54189a6b4176deb3a2a48@inspur.com> References: <35f020916eb54189a6b4176deb3a2a48@inspur.com> Message-ID: Hi Brin, Thanks for the patches! I've dropped a few reviews already. Feel free to reach me also on #openstack-oslo if you have any questions. moguimar On Wed, Aug 12, 2020 at 10:16 AM Brin Zhang(张百林) wrote: > Hi all, we would like to enable sasl protocol to oslo.cache and > keystonemiddleware > > project to improve the security of authority. > > SASL(Simple Authentication and Security Layer): is a memchanism used to > extend the verification ability of C/S mode. SASL is only the > authentication process, which integrates the application layer and the > system authentication mechanism. > > > > Need to review patches: > https://review.opendev.org/#/q/status:open++branch:master+topic:bp/enable-sasl-protocol > > > > > > > > brinzhang > > > -- Moisés Guimarães Software Engineer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Wed Aug 12 11:05:28 2020 From: smooney at redhat.com (Sean Mooney) Date: Wed, 12 Aug 2020 12:05:28 +0100 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> Message-ID: <4fc5e9d57172a73608e8fdf7e70ff569dca5dfd4.camel@redhat.com> On Wed, 2020-08-12 at 12:32 +0200, Thierry Carrez wrote: > Sean Mooney wrote: > > On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: > > > I wonder if this does help though. It seems like a bug that a nova-compute service would stop processing messages > > > and still be seen as up in the service status. Do we understand why that is happening? If not, I'm unclear that a > > > ping living at the oslo.messaging layer is going to do a better job of exposing such an outage. The fact that > > > oslo.messaging is responding does not necessarily equate to nova-compute functioning as expected. > > > > > > To be clear, this is not me nacking the ping feature. I just want to make sure we understand what is going on here > > > so we don't add another unreliable healthchecking mechanism to the one we already have. > > > > [...] > > im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug that is motiviting the creation of this oslo > > ping > > feature but that feels premature if it is. i think it would be better try to adress this by the sender recreating > > the > > queue if the deliver fails and if that is not viable then protpyope thge fix in nova. if the self ping fixes this > > miss queue error then we could extract the cod into oslo. > > I think this is missing the point... This is not about working around a > specific bug, it's about adding a way to detect a certain class of > failure. It's more of an operational feature than a development bugfix. right but we are concerned that there will be a negitive perfromance impact to adding it and it wont detect the one bug we are aware of of this type in a way that we could not also detect by using the mandtory flag. nova already has a heartbeat that the agents send to the conducto to report they are still alive. this ping would work in the opisite direction by reaching out to the compute node over the rpc bus. but that would only detect teh vailure mode if the pic use the topic queue and it could only fix it if recreating the queue via the conducor is a viable solution if it is using the mandataory flag and just recreating it is a better solution since we dont need to ping constantly in the background. if we get teh excpeiton we create the queue and retransmit. the compute manger does not resubscribe to the topic when the queue is recreated automaticaly then the new ping feature wont really help. we would need the comptue service or any other service that subsibse to the topic queue to try to ping its own topic queue and if that fails recreate the subsribtion/queue. as far as i am ware that is not what the fature is proposing > > If I understood correctly, OVH is running that patch in production as a > way to detect certain problems they regularly run into, something our > existing monitor mechanisms fail to detect. That sounds like a > worthwhile addition? im not sure what failure mode it will detect. if they can define that then it would help with understanding if this is worthwhile or not. > > Alternatively, if we can monitor the exact same class of failures using > our existing systems (or by improving them rather than adding a new > door), that works too. we can monitor the exitance of the queue at least form the rabbitmq api(its disable by defualt but just enable the rabbit-managment plugin) but im not sure what there current issue this is trying to solve is. > From radoslaw.piliszek at gmail.com Wed Aug 12 11:45:06 2020 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Wed, 12 Aug 2020 13:45:06 +0200 Subject: [tc][oslo] Move etcd3gw to OpenStack? Message-ID: Hey, Folks! I see it has been kinda proposed already [1] so that's mostly why I am asking about that now. >From what I understand, etcd3gw is our best bet when trying to get coordination with etcd3. However, it has recently released a broken release [2] due to no testing (not to mention gating with tooz). I think it could benefit from OpenDev's existing tooling. And since OpenStack is an important client of it and OpenStack preferring this client, it might be wise to put it in that namespace already. I guess the details would have to be discussed with dims (current owner) himself but he seemed happy about it in [1]. I'm notifying oslo as well as this would probably live best finally under oslo governance. Please let me know if any of the above is not true, so that I can amend my knowledge. :-) [1] https://github.com/dims/etcd3-gateway/issues/29 [2] https://bugs.launchpad.net/kolla-ansible/+bug/1891314 -yoctozepto -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwm2012 at gmail.com Wed Aug 12 13:03:17 2020 From: pwm2012 at gmail.com (pwm) Date: Wed, 12 Aug 2020 21:03:17 +0800 Subject: DNS server instead of /etc/hosts file on Infra Server In-Reply-To: References: Message-ID: Hi, Plan to use PowerDNS server instead of the /etc/hosts file for resolving. Has anyone done this before? The PowerDNS support MySQL DB backend and a frontend GUI PowerDNS Admin which allows centralized easy maintenance. Thanks On Sun, Aug 9, 2020 at 11:54 PM pwm wrote: > Hi, > Anyone interested in replacing the /etc/hosts file entry with a DNS server > on the openstack-ansible deployment? > > Thank you > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Wed Aug 12 13:23:09 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 12 Aug 2020 08:23:09 -0500 Subject: [Release-job-failures] Release of openstack/python-glanceclient for ref refs/tags/3.1.2 failed In-Reply-To: References: Message-ID: On 8/12/20 4:36 AM, zuul at openstack.org wrote: > Build failed. > > - openstack-upload-github-mirror https://zuul.opendev.org/t/openstack/build/cba5dda29e8744059d637a97f358c59f : SUCCESS in 43s > - release-openstack-python https://zuul.opendev.org/t/openstack/build/b9628cf4f28d4bea95844539295ff520 : SUCCESS in 2m 54s > - announce-release https://zuul.opendev.org/t/openstack/build/9f9a5910815247049bf02ab612781620 : FAILURE in 24m 20s > - propose-update-constraints https://zuul.opendev.org/t/openstack/build/44353ec832794af2965e7e2d05d63442 : SUCCESS in 3m 28s announce-release job appears to have failed due to a temporary network issue accessing PyPi packages. Since the release announcement is not critical, no further action is needed. Sean From jonathan.rosser at rd.bbc.co.uk Wed Aug 12 13:28:17 2020 From: jonathan.rosser at rd.bbc.co.uk (Jonathan Rosser) Date: Wed, 12 Aug 2020 14:28:17 +0100 Subject: DNS server instead of /etc/hosts file on Infra Server In-Reply-To: References: Message-ID: <7db29753-4710-a979-fe71-67a829fa55c3@rd.bbc.co.uk> Openstack-Ansible already supports optionally using the unbound dns server instead of managing /etc/hosts. Join #openstack-ansible on IRC if you need any help. Regards, Jonathan. On 12/08/2020 14:03, pwm wrote: > Hi, > Plan to use PowerDNS server instead of the /etc/hosts file for > resolving. Has anyone done this before? > The PowerDNS support MySQL DB backend and a frontend GUI PowerDNS > Admin which allows centralized easy maintenance. > > Thanks > > On Sun, Aug 9, 2020 at 11:54 PM pwm > wrote: > > Hi, > Anyone interested in replacing the /etc/hosts file entry with a > DNS server on the openstack-ansible deployment? > > Thank you > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Wed Aug 12 14:10:46 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 12 Aug 2020 10:10:46 -0400 Subject: [tc] monthly meeting summary Message-ID: Hi everyone, Here’s a summary of what happened in our TC monthly meeting last Thursday, August 6. # ATTENDEES (LINES SAID) - mnaser (100) - gmann (43) - diablo_rojo (20) - jungleboyj (16) - belmoreira (10) - evrardjp (8) - fungi (6) - zaneb (4) - knikolla (4) - njohnston (3) # MEETING SUMMARY 1. Rollcall (mnaser, 14:00:21) 2. Follow up on past action items (mnaser, 14:02:16) - https://review.opendev.org/#/c/744995/ (diablo_rojo, 14:03:14) - http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016336.html (gmann, 14:03:24) 3. OpenStack User-facing APIs and CLIs (belmoreira) (mnaser, 14:25:52) 4. W cycle goal selection start (mnaser, 14:34:39) 5. Completion of retirement cleanup (gmann) (mnaser, 14:40:48) - https://etherpad.opendev.org/p/tc-retirement-cleanup (mnaser, 14:41:02) - https://review.opendev.org/#/c/739291/1 (gmann, 14:42:17) - https://review.opendev.org/#/q/topic:cleanup-retirement+(status:open+OR+status:merged) (gmann, 14:42:51) # ACTION ITEMS - TC members to follow up and review "Resolution to define distributed leadership for projects" - mnaser schedule session with sig-arch and k8s steering committee - gmann continue to audit and clean-up tags - mnaser propose change to implement weekly meetings - njohnston and mugsie to work on getting goals groomed/proposed for W cycle - belmoreira start discussion around openstack user-facing apis & clis - gmann to merge changes to properly retire projects To read the full logs of the meeting, please refer to http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-08-06-14.00.log.html Thank you, Mohammed -- Mohammed Naser VEXXHOST, Inc. From mnaser at vexxhost.com Wed Aug 12 14:22:53 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 12 Aug 2020 10:22:53 -0400 Subject: [largescale-sig] RPC ping In-Reply-To: <20200806141132.GO31915@sync> References: <20200727095744.GK31915@sync> <20200806141132.GO31915@sync> Message-ID: On Thu, Aug 6, 2020 at 10:11 AM Arnaud Morin wrote: > > Hi Mohammed, > > 1 - That's something we would also like, but it's beyond the patch I > propose. > I need this patch not only for kubernetes, but also for monitoring my > legagy openstack agents running outside of k8s. > > 2 - Yes, latest version of rabbitmq is better on that point, but we > still see some weird issue (I will ask the community about it in another > topic). > > 3 - Thanks for this operator, we'll take a look! > By saying 1 rabbit per service, I understand 1 server, not 1 cluster, > right? > That sounds risky if you lose the server. The controllers are pretty stable and if a controller dies, Kubernetes will take care of restarting the pod somewhere else and everything will reconnect and things will be happy again. > I suppose you dont do that for the database? One database cluster per service, with 'old-school' replication because no one really does true multimaster in Galera with OpenStack anyways. > 4 - Nice, how to you monitor those consumptions? Using rabbit management > API? Prometheus RabbitMQ exporter, now migrating to the native one shipping in the new RabbitMQ releases. > Cheers, > > -- > Arnaud Morin > > On 03.08.20 - 10:21, Mohammed Naser wrote: > > I have a few operational suggestions on how I think we could do this best: > > > > 1. I think exposing a healthcheck endpoint that _actually_ runs the > > ping and responds with a 200 OK makes a lot more sense in terms of > > being able to run it inside something like Kubernetes, you end up with > > a "who makes the ping and who responds to it" type of scenario which > > can be tricky though I'm sure we can figure that out > > 2. I've found that newer releases of RabbitMQ really help with those > > un-usable queues after a split, I haven't had any issues at all with > > newer releases, so that could be something to help your life be a lot > > easier. > > 3. You mentioned you're moving towards Kubernetes, we're doing the > > same and building an operator: > > https://opendev.org/vexxhost/openstack-operator -- Because the > > operator manages the whole thing and Kubernetes does it's thing too, > > we started moving towards 1 (single) rabbitmq per service, which > > reaaaaaaally helped a lot in stabilizing things. Oslo messaging is a > > lot better at recovering when a single service IP is pointing towards > > it because it doesn't do weird things like have threads trying to > > connect to other Rabbit ports. Just a thought. > > 4. In terms of telemetry and making sure you avoid that issue, we > > track the consumption rates of queues inside OpenStack. OpenStack > > consumption rate should be constant and never growing, anytime it > > grows, we instantly detect that something is fishy. However, the > > other issue comes in that when you restart any openstack service, it > > 'forgets' all it's existing queues and then you have a set of building > > up queues until they automatically expire which happens around 30 > > minutes-ish, so it makes that alarm of "things are not being consumed" > > a little noisy if you're restarting services > > > > Sorry for the wall of super unorganized text, all over the place here > > but thought I'd chime in with my 2 cents :) > > > > On Mon, Jul 27, 2020 at 6:04 AM Arnaud Morin wrote: > > > > > > Hey all, > > > > > > TLDR: I propose a change to oslo_messaging to allow doing a ping over RPC, > > > this is useful to monitor liveness of agents. > > > > > > > > > Few weeks ago, I proposed a patch to oslo_messaging [1], which is adding a > > > ping endpoint to RPC dispatcher. > > > It means that every openstack service which is using oslo_messaging RPC > > > endpoints (almosts all OpenStack services and agents - e.g. neutron > > > server + agents, nova + computes, etc.) will then be able to answer to a > > > specific "ping" call over RPC. > > > > > > I decided to propose this patch in my company mainly for 2 reasons: > > > 1 - we are struggling monitoring our nova compute and neutron agents in a > > > correct way: > > > > > > 1.1 - sometimes our agents are disconnected from RPC, but the python process > > > is still running. > > > 1.2 - sometimes the agent is still connected, but the queue / binding on > > > rabbit cluster is not working anymore (after a rabbit split for > > > example). This one is very hard to debug, because the agent is still > > > reporting health correctly on neutron server, but it's not able to > > > receive messages anymore. > > > > > > > > > 2 - we are trying to monitor agents running in k8s pods: > > > when running a python agent (neutron l3-agent for example) in a k8s pod, we > > > wanted to find a way to monitor if it is still live of not. > > > > > > > > > Adding a RPC ping endpoint could help us solve both these issues. > > > Note that we still need an external mechanism (out of OpenStack) to do this > > > ping. > > > We also think it could be nice for other OpenStackers, and especially > > > large scale ops. > > > > > > Feel free to comment. > > > > > > > > > [1] https://review.opendev.org/#/c/735385/ > > > > > > > > > -- > > > Arnaud Morin > > > > > > > > > > > > -- > > Mohammed Naser > > VEXXHOST, Inc. -- Mohammed Naser VEXXHOST, Inc. From dev.faz at gmail.com Wed Aug 12 14:25:49 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Wed, 12 Aug 2020 16:25:49 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: References: <20200806144016.GP31915@sync> <1a338d7e-c82c-cda2-2d47-b5aebb999142@openstack.org> Message-ID: Hi, just could prove, that "durable queues" seem to workaround the issue. If I enable durable queues, im no longer able to reproduce my issue. Afaik durable queues have downsides - esp if a node fails and the queue is not (jet) synced. Anyone information about this? Fabian Am Mi., 12. Aug. 2020 um 12:14 Uhr schrieb Fabian Zimmermann < dev.faz at gmail.com>: > Hi, > > just wrote some small scripts to reproduce our issue and send a msg to > rabbitmq-list. > > https://groups.google.com/d/msg/rabbitmq-users/eC8jc-YEt8s/s8K_0KnXDQAJ > > Fabian > > > Am Di., 11. Aug. 2020 um 12:31 Uhr schrieb Thierry Carrez < > thierry at openstack.org>: > >> If you can reproduce it with current versions, I would suggest to file >> an issue on https://github.com/rabbitmq/rabbitmq-server/issues/ >> >> The behavior you describe seems to match >> https://github.com/rabbitmq/rabbitmq-server/issues/1873 but the >> maintainers seem to think it's been fixed by a number of >> somewhat-related changes in 3.7.13, because nobody reported issues >> anymore :) >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Wed Aug 12 14:30:00 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 12 Aug 2020 10:30:00 -0400 Subject: [tc] weekly summary Message-ID: Hi everyone, Here’s an update for what happened in the OpenStack TC this week. You can get more information by checking for changes in openstack/governance repository. We've also included a few references to some important mailing list threads that you should check out. # Patches ## Open Reviews - Create starter-kit:kubernetes-in-virt tag https://review.opendev.org/736369 - Move towards dual office hours https://review.opendev.org/745201 - Clean up expired i18n SIG extra-ATCs https://review.opendev.org/745565 - Resolution to define distributed leadership for projects https://review.opendev.org/744995 - Move towards single office hour https://review.opendev.org/745200 - Add legacy repository validation https://review.opendev.org/737559 - Drop all exceptions for legacy validation https://review.opendev.org/745403 - [draft] Add assert:supports-standalone https://review.opendev.org/722399 - Clean up expired i18n SIG extra-ATCs https://review.opendev.org/745565 - Add legacy repository validation https://review.opendev.org/737559 - Pierre Riteau as CloudKitty PTL for Victoria https://review.opendev.org/745653 - Resolution to define distributed leadership for projects https://review.opendev.org/744995 - Create starter-kit:kubernetes-in-virt tag https://review.opendev.org/736369 - Move towards dual office hours https://review.opendev.org/745201 - Move towards single office hour https://review.opendev.org/745200 ## Project Updates - Deprecate os_congress project https://review.opendev.org/742533 - Add Ceph iSCSI charm to OpenStack charms https://review.opendev.org/744480 - Add Keystone Kerberos charm to OpenStack charms https://review.opendev.org/743769 - Add python-dracclient to be owned by Hardware Vendor SIG https://review.opendev.org/745564 ## General Changes - Reverse sort series in selected goals https://review.opendev.org/744897 - Declare supported runtimes for Wallaby release https://review.opendev.org/743847 - Sort SIG names in repo owner list https://review.opendev.org/745563 - Drop neutron-vpnaas from legacy projects https://review.opendev.org/745401 ## Abandoned Changes - Migrate testing to ubuntu focal https://review.opendev.org/740851 # Email Threads - CloudKitty Status: http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016171.html - OSC vs python-*clients: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016409.html - Proposed Wallaby Schedule: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016391.html - New Office Hour Plans: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016372.html # Other Reminders - Cycle-Trailing Release Deadline Aug 13 Thanks for reading! Mohammed & Kendall -- Mohammed Naser VEXXHOST, Inc. From dev.faz at gmail.com Wed Aug 12 15:03:40 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Wed, 12 Aug 2020 17:03:40 +0200 Subject: [largescale-sig] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> <20200806141132.GO31915@sync> Message-ID: Hi, Am Mi., 12. Aug. 2020 um 16:30 Uhr schrieb Mohammed Naser < mnaser at vexxhost.com>: > On Thu, Aug 6, 2020 at 10:11 AM Arnaud Morin > wrote: > The controllers are pretty stable and if a controller dies, Kubernetes > will take care of restarting the pod somewhere else and everything > will reconnect and things will be happy again. > sounds really interesting. Do you have any docs how to use / do a poc of this setup? Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Wed Aug 12 15:21:31 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 12 Aug 2020 10:21:31 -0500 Subject: [all] Proposed Wallaby cycle schedule In-Reply-To: <2e56de68-c416-e3ea-f3da-caaf9399287d@gmx.com> References: <2e56de68-c416-e3ea-f3da-caaf9399287d@gmx.com> Message-ID: <0083db2a-0ef7-99fa-0c45-fd170f7d7902@gmx.com> > > The current thinking is it will likely take place in May (nothing is > set, just an educated guess, so please don't use that for any other > planning). So for the sake of figuring out the release schedule, we are > targeting a release date in early May. Hopefully this will then align > well with event plans. > > I have a proposed release schedule up for review here: > > https://review.opendev.org/#/c/744729/ > > For ease of viewing (until the job logs are garbage collected), you can > see the rendered schedule here: > > https://0e6b8aeca433e85b429b-46fd243db6dc394538bd0555f339eba5.ssl.cf1.rackcdn.com/744729/3/check/openstack-tox-docs/4f76901/docs/wallaby/schedule.html > > > There are always outside conflicts, but I think this has aligned mostly > well with major holidays. But please feel free to comment on the patch > if you see any major issues that we may have not considered. > One more update to this. Some concerns were raised around alignment with the planned Ubuntu release schedule. Plus some general sentiment for wanting to be closer to a 6 month schedule. As an alternative option, I have proposed a 26 week option: https://review.opendev.org/#/c/745911/ This would mean there would be a largish gap between when the X release starts and when we might hold the PTG for that development. That could be good or bad. Depending on the in-person event situation, it is also unknown if we would need to wait for a larger scheduled event, or if we would be able to hold a virtual event sooner. So lot's of unknowns. Getting community feedback on these options would be useful. If one schedule or the other seems better to you, please add comments to the patches. Here is a rendered schedule for the 26 week option: https://f7c086752d1ed6ae5f02-3fd01ef5e4a590ae96edf7e9bfcef60c.ssl.cf1.rackcdn.com/745911/2/check/openstack-tox-docs/5be238a/docs/wallaby/schedule.html Thanks! Sean From openstack at nemebean.com Wed Aug 12 15:50:21 2020 From: openstack at nemebean.com (Ben Nemec) Date: Wed, 12 Aug 2020 10:50:21 -0500 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> Message-ID: <16a3adf0-2f51-dd7d-c729-7b27f1593980@nemebean.com> On 8/12/20 5:32 AM, Thierry Carrez wrote: > Sean Mooney wrote: >> On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: >>> I wonder if this does help though. It seems like a bug that a >>> nova-compute service would stop processing messages and still be seen >>> as up in the service status. Do we understand why that is happening? >>> If not, I'm unclear that a ping living at the oslo.messaging layer is >>> going to do a better job of exposing such an outage. The fact that >>> oslo.messaging is responding does not necessarily equate to >>> nova-compute functioning as expected. >>> >>> To be clear, this is not me nacking the ping feature. I just want to >>> make sure we understand what is going on here so we don't add another >>> unreliable healthchecking mechanism to the one we already have. >> [...] >> im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug >> that is motiviting the creation of this oslo ping >> feature but that feels premature if it is. i think it would be better >> try to adress this by the sender recreating the >> queue if the deliver fails and if that is not viable then protpyope >> thge fix in nova. if the self ping fixes this >> miss queue error then we could extract the cod into oslo. > > I think this is missing the point... This is not about working around a > specific bug, it's about adding a way to detect a certain class of > failure. It's more of an operational feature than a development bugfix. > > If I understood correctly, OVH is running that patch in production as a > way to detect certain problems they regularly run into, something our > existing monitor mechanisms fail to detect. That sounds like a > worthwhile addition? Okay, I don't think I was aware that this was already being used. If someone already finds it useful and it's opt-in then I'm not inclined to block it. My main concern was that we were adding a feature that didn't actually address the problem at hand. I _would_ feel better about it if someone could give an example of a type of failure this is detecting that is missed by other monitoring methods though. Both because having a concrete example of a use case for the feature is good, and because if it turns out that the problems this is detecting are things like the Nova bug Sean is talking about (which I don't think this would catch anyway, since the topic is missing and there's nothing to ping) then there may be other changes we can/should make to improve things. > > Alternatively, if we can monitor the exact same class of failures using > our existing systems (or by improving them rather than adding a new > door), that works too. > From thierry at openstack.org Wed Aug 12 16:14:58 2020 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 12 Aug 2020 18:14:58 +0200 Subject: [largescale-sig] Next meeting: August 12, 16utc In-Reply-To: <6e7a4e43-08f4-3030-2eb0-9311f27d9647@openstack.org> References: <6e7a4e43-08f4-3030-2eb0-9311f27d9647@openstack.org> Message-ID: We just held the meeting, it was very short, as only mdelavergne and myself were on. None of the expected US-based recruits joined. We'll likely have to beat a larger drum for the next US-EU meeting in 4 weeks. Meeting logs at: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-08-12-16.00.html TODOs: - amorin to add some meat to the wiki page before we push the Nova doc patch further - all to describe briefly how you solved metrics/billing in your deployment in https://etherpad.openstack.org/p/large-scale-sig-documentation Next meetings: Aug 26, 8:00UTC; Sep 9, 16:00UTC (#openstack-meeting-3) -- Thierry Carrez (ttx) From mnaser at vexxhost.com Wed Aug 12 16:56:12 2020 From: mnaser at vexxhost.com (Mohammed Naser) Date: Wed, 12 Aug 2020 12:56:12 -0400 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez wrote: > > Thomas Goirand wrote: > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: > >> Thanks, Pierre for helping with this. > >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) ) > >> but I am not sure if he got any response back. > > No response so far, but they may all be in company summer vacation. > > > The end of the very good maintenance of Cloudkitty matched the date when > > objectif libre was sold to Linkbynet. Maybe the new owner don't care enough? > > > > This is very disappointing as I've been using it for some time already, > > and that I was satisfied by it (ie: it does the job...), and especially > > that latest releases are able to scale correctly. > > > > I very much would love if Pierre Riteau was successful in taking over. > > Good luck Pierre! I'll try to help whenever I can and if I'm not too busy. > > Given the volunteers (Pierre, Rafael, Luis) I would support the TC using > its unholy powers to add extra core reviewers to cloudkitty. https://review.opendev.org/#/c/745653 is currently merging and fungi will be adding Pierre as a core. Thank you for helping. > If the current PTL comes back, I'm sure they will appreciate the help, > and can always fix/revert things before Victoria release. > > -- > Thierry Carrez (ttx) > -- Mohammed Naser VEXXHOST, Inc. From openstack at nemebean.com Wed Aug 12 17:02:34 2020 From: openstack at nemebean.com (Ben Nemec) Date: Wed, 12 Aug 2020 12:02:34 -0500 Subject: [tc][oslo] Move etcd3gw to OpenStack? In-Reply-To: References: Message-ID: <0c77a128-acea-75f3-faa2-b3d79c3991aa@nemebean.com> I'm fine with this for all the reasons mentioned below. It's not a high volume project so it shouldn't be a big problem to bring it into Oslo. Plus it would give us an excuse to make dims an Oslo core again. ;-) On 8/12/20 6:45 AM, Radosław Piliszek wrote: > Hey, Folks! > > I see it has been kinda proposed already [1] so that's mostly why I am > asking about that now. > > From what I understand, etcd3gw is our best bet when trying to get > coordination with etcd3. > However, it has recently released a broken release [2] due to no testing > (not to mention gating with tooz). > I think it could benefit from OpenDev's existing tooling. > And since OpenStack is an important client of it and OpenStack > preferring this client, it might be wise to put it in that namespace > already. > > I guess the details would have to be discussed with dims (current owner) > himself but he seemed happy about it in [1]. > > I'm notifying oslo as well as this would probably live best finally > under oslo governance. > > Please let me know if any of the above is not true, so that I can amend > my knowledge. :-) > > [1] https://github.com/dims/etcd3-gateway/issues/29 > [2] https://bugs.launchpad.net/kolla-ansible/+bug/1891314 > > -yoctozepto > From rafaelweingartner at gmail.com Wed Aug 12 17:06:53 2020 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Wed, 12 Aug 2020 14:06:53 -0300 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Awesome! Thank you guys for the help. We have few PRs open there that are ready (or close to be ready) to be merged. On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser wrote: > On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez > wrote: > > > > Thomas Goirand wrote: > > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: > > >> Thanks, Pierre for helping with this. > > >> > > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) < > justin.ferrieu at objectif-libre.com>) > > >> but I am not sure if he got any response back. > > > > No response so far, but they may all be in company summer vacation. > > > > > The end of the very good maintenance of Cloudkitty matched the date > when > > > objectif libre was sold to Linkbynet. Maybe the new owner don't care > enough? > > > > > > This is very disappointing as I've been using it for some time already, > > > and that I was satisfied by it (ie: it does the job...), and especially > > > that latest releases are able to scale correctly. > > > > > > I very much would love if Pierre Riteau was successful in taking over. > > > Good luck Pierre! I'll try to help whenever I can and if I'm not too > busy. > > > > Given the volunteers (Pierre, Rafael, Luis) I would support the TC using > > its unholy powers to add extra core reviewers to cloudkitty. > > https://review.opendev.org/#/c/745653 is currently merging and fungi will > be > adding Pierre as a core. > > Thank you for helping. > > > If the current PTL comes back, I'm sure they will appreciate the help, > > and can always fix/revert things before Victoria release. > > > > -- > > Thierry Carrez (ttx) > > > > > -- > Mohammed Naser > VEXXHOST, Inc. > > -- Rafael Weingärtner -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Wed Aug 12 20:37:05 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Wed, 12 Aug 2020 22:37:05 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: I have now received core reviewer privileges. Thank you to TC members for trusting us with the CloudKitty project. I would like to kick things off by resuming IRC meetings. They're set to run every two weeks (on odd weeks) on Monday at 1400 UTC in #cloudkitty. Is this a convenient time slot for all potential contributors to the project? On Wed, 12 Aug 2020 at 19:08, Rafael Weingärtner wrote: > > Awesome! Thank you guys for the help. > We have few PRs open there that are ready (or close to be ready) to be merged. > > On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser wrote: >> >> On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez wrote: >> > >> > Thomas Goirand wrote: >> > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: >> > >> Thanks, Pierre for helping with this. >> > >> >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) ) >> > >> but I am not sure if he got any response back. >> > >> > No response so far, but they may all be in company summer vacation. >> > >> > > The end of the very good maintenance of Cloudkitty matched the date when >> > > objectif libre was sold to Linkbynet. Maybe the new owner don't care enough? >> > > >> > > This is very disappointing as I've been using it for some time already, >> > > and that I was satisfied by it (ie: it does the job...), and especially >> > > that latest releases are able to scale correctly. >> > > >> > > I very much would love if Pierre Riteau was successful in taking over. >> > > Good luck Pierre! I'll try to help whenever I can and if I'm not too busy. >> > >> > Given the volunteers (Pierre, Rafael, Luis) I would support the TC using >> > its unholy powers to add extra core reviewers to cloudkitty. >> >> https://review.opendev.org/#/c/745653 is currently merging and fungi will be >> adding Pierre as a core. >> >> Thank you for helping. >> >> > If the current PTL comes back, I'm sure they will appreciate the help, >> > and can always fix/revert things before Victoria release. >> > >> > -- >> > Thierry Carrez (ttx) >> > >> >> >> -- >> Mohammed Naser >> VEXXHOST, Inc. >> > > > -- > Rafael Weingärtner From rafaelweingartner at gmail.com Wed Aug 12 20:40:57 2020 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Wed, 12 Aug 2020 17:40:57 -0300 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Sounds good to me. On Wed, Aug 12, 2020 at 5:37 PM Pierre Riteau wrote: > I have now received core reviewer privileges. Thank you to TC members > for trusting us with the CloudKitty project. > > I would like to kick things off by resuming IRC meetings. They're set > to run every two weeks (on odd weeks) on Monday at 1400 UTC in > #cloudkitty. Is this a convenient time slot for all potential > contributors to the project? > > On Wed, 12 Aug 2020 at 19:08, Rafael Weingärtner > wrote: > > > > Awesome! Thank you guys for the help. > > We have few PRs open there that are ready (or close to be ready) to be > merged. > > > > On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser > wrote: > >> > >> On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez > wrote: > >> > > >> > Thomas Goirand wrote: > >> > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: > >> > >> Thanks, Pierre for helping with this. > >> > >> > >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) < > justin.ferrieu at objectif-libre.com>) > >> > >> but I am not sure if he got any response back. > >> > > >> > No response so far, but they may all be in company summer vacation. > >> > > >> > > The end of the very good maintenance of Cloudkitty matched the date > when > >> > > objectif libre was sold to Linkbynet. Maybe the new owner don't > care enough? > >> > > > >> > > This is very disappointing as I've been using it for some time > already, > >> > > and that I was satisfied by it (ie: it does the job...), and > especially > >> > > that latest releases are able to scale correctly. > >> > > > >> > > I very much would love if Pierre Riteau was successful in taking > over. > >> > > Good luck Pierre! I'll try to help whenever I can and if I'm not > too busy. > >> > > >> > Given the volunteers (Pierre, Rafael, Luis) I would support the TC > using > >> > its unholy powers to add extra core reviewers to cloudkitty. > >> > >> https://review.opendev.org/#/c/745653 is currently merging and fungi > will be > >> adding Pierre as a core. > >> > >> Thank you for helping. > >> > >> > If the current PTL comes back, I'm sure they will appreciate the help, > >> > and can always fix/revert things before Victoria release. > >> > > >> > -- > >> > Thierry Carrez (ttx) > >> > > >> > >> > >> -- > >> Mohammed Naser > >> VEXXHOST, Inc. > >> > > > > > > -- > > Rafael Weingärtner > -- Rafael Weingärtner -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.ramirez at opencloud.es Wed Aug 12 21:38:30 2020 From: luis.ramirez at opencloud.es (Luis Ramirez) Date: Wed, 12 Aug 2020 23:38:30 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Sounds good to me. El El mié, 12 ago 2020 a las 22:41, Rafael Weingärtner < rafaelweingartner at gmail.com> escribió: > Sounds good to me. > > On Wed, Aug 12, 2020 at 5:37 PM Pierre Riteau wrote: > >> I have now received core reviewer privileges. Thank you to TC members >> >> >> for trusting us with the CloudKitty project. >> >> >> >> >> >> I would like to kick things off by resuming IRC meetings. They're set >> >> >> to run every two weeks (on odd weeks) on Monday at 1400 UTC in >> >> >> #cloudkitty. Is this a convenient time slot for all potential >> >> >> contributors to the project? >> >> >> >> >> >> On Wed, 12 Aug 2020 at 19:08, Rafael Weingärtner >> >> >> wrote: >> >> >> > >> >> >> > Awesome! Thank you guys for the help. >> >> >> > We have few PRs open there that are ready (or close to be ready) to be >> merged. >> >> >> > >> >> >> > On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser >> wrote: >> >> >> >> >> >> >> >> On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez >> wrote: >> >> >> >> > >> >> >> >> > Thomas Goirand wrote: >> >> >> >> > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: >> >> >> >> > >> Thanks, Pierre for helping with this. >> >> >> >> > >> >> >> >> >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) < >> justin.ferrieu at objectif-libre.com>) >> >> >> >> > >> but I am not sure if he got any response back. >> >> >> >> > >> >> >> >> > No response so far, but they may all be in company summer vacation. >> >> >> >> > >> >> >> >> > > The end of the very good maintenance of Cloudkitty matched the >> date when >> >> >> >> > > objectif libre was sold to Linkbynet. Maybe the new owner don't >> care enough? >> >> >> >> > > >> >> >> >> > > This is very disappointing as I've been using it for some time >> already, >> >> >> >> > > and that I was satisfied by it (ie: it does the job...), and >> especially >> >> >> >> > > that latest releases are able to scale correctly. >> >> >> >> > > >> >> >> >> > > I very much would love if Pierre Riteau was successful in taking >> over. >> >> >> >> > > Good luck Pierre! I'll try to help whenever I can and if I'm not >> too busy. >> >> >> >> > >> >> >> >> > Given the volunteers (Pierre, Rafael, Luis) I would support the TC >> using >> >> >> >> > its unholy powers to add extra core reviewers to cloudkitty. >> >> >> >> >> >> >> >> https://review.opendev.org/#/c/745653 is currently merging and fungi >> will be >> >> >> >> adding Pierre as a core. >> >> >> >> >> >> >> >> Thank you for helping. >> >> >> >> >> >> >> >> > If the current PTL comes back, I'm sure they will appreciate the >> help, >> >> >> >> > and can always fix/revert things before Victoria release. >> >> >> >> > >> >> >> >> > -- >> >> >> >> > Thierry Carrez (ttx) >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Mohammed Naser >> >> >> >> VEXXHOST, Inc. >> >> >> >> >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > Rafael Weingärtner >> >> >> > > -- > Rafael Weingärtner > > > -- Br, Luis Rmz Blockchain, DevOps & Open Source Cloud Solutions Architect ---------------------------------------- Founder & CEO OpenCloud.es luis.ramirez at opencloud.es Skype ID: d.overload Hangouts: luis.ramirez at opencloud.es +34 911 950 123 / +39 392 1289553 / +49 152 26917722 -------------- next part -------------- An HTML attachment was scrubbed... URL: From melwittt at gmail.com Wed Aug 12 23:22:38 2020 From: melwittt at gmail.com (melanie witt) Date: Wed, 12 Aug 2020 16:22:38 -0700 Subject: [gate][keystone][nova][neutron] *-grenade-multinode jobs failing with UnicodeDecodeError in keystone In-Reply-To: <18572a52-d105-9219-6b19-5fe23f18e3e0@gmail.com> References: <45926788-6dcf-8825-5bfd-b6353b5facf0@gmail.com> <67a115ba-f80a-ebe5-8689-922e3bbb9a40@gmail.com> <18572a52-d105-9219-6b19-5fe23f18e3e0@gmail.com> Message-ID: On 8/12/20 00:49, melanie witt wrote: >>> I've opened a bug [3] about and I'm trying out the following keystone >>> patch to fix it: >>> >>> https://review.opendev.org/745752 >>> >>> Reviews appreciated. > > I tested ^ with a DNM patch to nova and nova-grenade-multinode passes > with it. The fix has merged and it is now safe to recheck your patches. Thank you all for the code reviews. Cheers, -melanie >>> [1] >>> https://github.com/msgpack/msgpack-python/blob/v1.0.0/README.md#major-breaking-changes-in-msgpack-10 >>> >>> [2] https://review.opendev.org/#/c/745437/2/upper-constraints.txt at 373 >>> [3] https://launchpad.net/bugs/1891244 From jayadityagupta11 at gmail.com Thu Aug 13 08:03:01 2020 From: jayadityagupta11 at gmail.com (jayaditya gupta) Date: Thu, 13 Aug 2020 10:03:01 +0200 Subject: [openstackclient] Implementing nova migration cmds in OSC Message-ID: Hello , i am trying to implement some nova migrations commands to openstackclient. Commands 1. migration-list : list all migrations ever happened 2. server-migration-list : Get the migrations list of specified server 3. server-migration-show : show currently going on migration of specified server. 4. live migration abort feature 5.live-migration-force-complete Please have a look at this patch : https://review.opendev.org/#/c/742210/ and share your insight, what should be the correct way to implement it ? should it be a root command or part of the openstack server migrate command? Best Regards Jayaditya Gupta -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Thu Aug 13 08:24:26 2020 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 13 Aug 2020 10:24:26 +0200 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <16a3adf0-2f51-dd7d-c729-7b27f1593980@nemebean.com> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> <16a3adf0-2f51-dd7d-c729-7b27f1593980@nemebean.com> Message-ID: Ben Nemec wrote: > On 8/12/20 5:32 AM, Thierry Carrez wrote: >> Sean Mooney wrote: >>> On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: >>>> I wonder if this does help though. It seems like a bug that a >>>> nova-compute service would stop processing messages and still be >>>> seen as up in the service status. Do we understand why that is >>>> happening? If not, I'm unclear that a ping living at the >>>> oslo.messaging layer is going to do a better job of exposing such an >>>> outage. The fact that oslo.messaging is responding does not >>>> necessarily equate to nova-compute functioning as expected. >>>> >>>> To be clear, this is not me nacking the ping feature. I just want to >>>> make sure we understand what is going on here so we don't add >>>> another unreliable healthchecking mechanism to the one we already have. >>> [...] >>> im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug >>> that is motiviting the creation of this oslo ping >>> feature but that feels premature if it is. i think it would be better >>> try to adress this by the sender recreating the >>> queue if the deliver fails and if that is not viable then protpyope >>> thge fix in nova. if the self ping fixes this >>> miss queue error then we could extract the cod into oslo. >> >> I think this is missing the point... This is not about working around >> a specific bug, it's about adding a way to detect a certain class of >> failure. It's more of an operational feature than a development bugfix. >> >> If I understood correctly, OVH is running that patch in production as >> a way to detect certain problems they regularly run into, something >> our existing monitor mechanisms fail to detect. That sounds like a >> worthwhile addition? > > Okay, I don't think I was aware that this was already being used. If > someone already finds it useful and it's opt-in then I'm not inclined to > block it. My main concern was that we were adding a feature that didn't > actually address the problem at hand. > > I _would_ feel better about it if someone could give an example of a > type of failure this is detecting that is missed by other monitoring > methods though. Both because having a concrete example of a use case for > the feature is good, and because if it turns out that the problems this > is detecting are things like the Nova bug Sean is talking about (which I > don't think this would catch anyway, since the topic is missing and > there's nothing to ping) then there may be other changes we can/should > make to improve things. Right. Let's wait for Arnaud to come back from vacation and confirm that (1) that patch is not a shot in the dark: it allows them to expose a class of issues in production (2) they fail to expose that same class of issues using other existing mechanisms, including those just suggested in this thread I just wanted to avoid early rejection of this health check ability on the grounds that the situation it exposes should just not happen. Or that, if enabled and heavily used, it would have a performance impact. -- Thierry Carrez (ttx) From pierre at stackhpc.com Thu Aug 13 11:35:42 2020 From: pierre at stackhpc.com (Pierre Riteau) Date: Thu, 13 Aug 2020 13:35:42 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Thank you both. I've merged a few patches to fix CI and finalise the Ussuri release (for example release notes were missing). I gave core reviewer privileges to Rafael and Luis. Let's try to merge patches with two +2 votes from now on. On Wed, 12 Aug 2020 at 23:38, Luis Ramirez wrote: > > Sounds good to me. > > El El mié, 12 ago 2020 a las 22:41, Rafael Weingärtner escribió: >> >> Sounds good to me. >> >> On Wed, Aug 12, 2020 at 5:37 PM Pierre Riteau wrote: >>> >>> I have now received core reviewer privileges. Thank you to TC members >>> >>> >>> for trusting us with the CloudKitty project. >>> >>> >>> >>> >>> >>> I would like to kick things off by resuming IRC meetings. They're set >>> >>> >>> to run every two weeks (on odd weeks) on Monday at 1400 UTC in >>> >>> >>> #cloudkitty. Is this a convenient time slot for all potential >>> >>> >>> contributors to the project? >>> >>> >>> >>> >>> >>> On Wed, 12 Aug 2020 at 19:08, Rafael Weingärtner >>> >>> >>> wrote: >>> >>> >>> > >>> >>> >>> > Awesome! Thank you guys for the help. >>> >>> >>> > We have few PRs open there that are ready (or close to be ready) to be merged. >>> >>> >>> > >>> >>> >>> > On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser wrote: >>> >>> >>> >> >>> >>> >>> >> On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez wrote: >>> >>> >>> >> > >>> >>> >>> >> > Thomas Goirand wrote: >>> >>> >>> >> > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: >>> >>> >>> >> > >> Thanks, Pierre for helping with this. >>> >>> >>> >> > >> >>> >>> >>> >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) ) >>> >>> >>> >> > >> but I am not sure if he got any response back. >>> >>> >>> >> > >>> >>> >>> >> > No response so far, but they may all be in company summer vacation. >>> >>> >>> >> > >>> >>> >>> >> > > The end of the very good maintenance of Cloudkitty matched the date when >>> >>> >>> >> > > objectif libre was sold to Linkbynet. Maybe the new owner don't care enough? >>> >>> >>> >> > > >>> >>> >>> >> > > This is very disappointing as I've been using it for some time already, >>> >>> >>> >> > > and that I was satisfied by it (ie: it does the job...), and especially >>> >>> >>> >> > > that latest releases are able to scale correctly. >>> >>> >>> >> > > >>> >>> >>> >> > > I very much would love if Pierre Riteau was successful in taking over. >>> >>> >>> >> > > Good luck Pierre! I'll try to help whenever I can and if I'm not too busy. >>> >>> >>> >> > >>> >>> >>> >> > Given the volunteers (Pierre, Rafael, Luis) I would support the TC using >>> >>> >>> >> > its unholy powers to add extra core reviewers to cloudkitty. >>> >>> >>> >> >>> >>> >>> >> https://review.opendev.org/#/c/745653 is currently merging and fungi will be >>> >>> >>> >> adding Pierre as a core. >>> >>> >>> >> >>> >>> >>> >> Thank you for helping. >>> >>> >>> >> >>> >>> >>> >> > If the current PTL comes back, I'm sure they will appreciate the help, >>> >>> >>> >> > and can always fix/revert things before Victoria release. >>> >>> >>> >> > >>> >>> >>> >> > -- >>> >>> >>> >> > Thierry Carrez (ttx) >>> >>> >>> >> > >>> >>> >>> >> >>> >>> >>> >> >>> >>> >>> >> -- >>> >>> >>> >> Mohammed Naser >>> >>> >>> >> VEXXHOST, Inc. >>> >>> >>> >> >>> >>> >>> > >>> >>> >>> > >>> >>> >>> > -- >>> >>> >>> > Rafael Weingärtner >>> >>> >> >> >> -- >> Rafael Weingärtner >> >> > -- > Br, > Luis Rmz > Blockchain, DevOps & Open Source Cloud Solutions Architect > ---------------------------------------- > Founder & CEO > OpenCloud.es > luis.ramirez at opencloud.es > Skype ID: d.overload > Hangouts: luis.ramirez at opencloud.es > +34 911 950 123 / +39 392 1289553 / +49 152 26917722 From rafaelweingartner at gmail.com Thu Aug 13 11:44:20 2020 From: rafaelweingartner at gmail.com (=?UTF-8?Q?Rafael_Weing=C3=A4rtner?=) Date: Thu, 13 Aug 2020 08:44:20 -0300 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Awesome, thanks! I will try to dedicate a few hours every week to review CloudKitty patches. On Thu, Aug 13, 2020 at 8:36 AM Pierre Riteau wrote: > Thank you both. > > I've merged a few patches to fix CI and finalise the Ussuri release > (for example release notes were missing). > I gave core reviewer privileges to Rafael and Luis. Let's try to merge > patches with two +2 votes from now on. > > On Wed, 12 Aug 2020 at 23:38, Luis Ramirez > wrote: > > > > Sounds good to me. > > > > El El mié, 12 ago 2020 a las 22:41, Rafael Weingärtner < > rafaelweingartner at gmail.com> escribió: > >> > >> Sounds good to me. > >> > >> On Wed, Aug 12, 2020 at 5:37 PM Pierre Riteau > wrote: > >>> > >>> I have now received core reviewer privileges. Thank you to TC members > >>> > >>> > >>> for trusting us with the CloudKitty project. > >>> > >>> > >>> > >>> > >>> > >>> I would like to kick things off by resuming IRC meetings. They're set > >>> > >>> > >>> to run every two weeks (on odd weeks) on Monday at 1400 UTC in > >>> > >>> > >>> #cloudkitty. Is this a convenient time slot for all potential > >>> > >>> > >>> contributors to the project? > >>> > >>> > >>> > >>> > >>> > >>> On Wed, 12 Aug 2020 at 19:08, Rafael Weingärtner > >>> > >>> > >>> wrote: > >>> > >>> > >>> > > >>> > >>> > >>> > Awesome! Thank you guys for the help. > >>> > >>> > >>> > We have few PRs open there that are ready (or close to be ready) to > be merged. > >>> > >>> > >>> > > >>> > >>> > >>> > On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser > wrote: > >>> > >>> > >>> >> > >>> > >>> > >>> >> On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez < > thierry at openstack.org> wrote: > >>> > >>> > >>> >> > > >>> > >>> > >>> >> > Thomas Goirand wrote: > >>> > >>> > >>> >> > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: > >>> > >>> > >>> >> > >> Thanks, Pierre for helping with this. > >>> > >>> > >>> >> > >> > >>> > >>> > >>> >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) < > justin.ferrieu at objectif-libre.com>) > >>> > >>> > >>> >> > >> but I am not sure if he got any response back. > >>> > >>> > >>> >> > > >>> > >>> > >>> >> > No response so far, but they may all be in company summer > vacation. > >>> > >>> > >>> >> > > >>> > >>> > >>> >> > > The end of the very good maintenance of Cloudkitty matched the > date when > >>> > >>> > >>> >> > > objectif libre was sold to Linkbynet. Maybe the new owner don't > care enough? > >>> > >>> > >>> >> > > > >>> > >>> > >>> >> > > This is very disappointing as I've been using it for some time > already, > >>> > >>> > >>> >> > > and that I was satisfied by it (ie: it does the job...), and > especially > >>> > >>> > >>> >> > > that latest releases are able to scale correctly. > >>> > >>> > >>> >> > > > >>> > >>> > >>> >> > > I very much would love if Pierre Riteau was successful in > taking over. > >>> > >>> > >>> >> > > Good luck Pierre! I'll try to help whenever I can and if I'm > not too busy. > >>> > >>> > >>> >> > > >>> > >>> > >>> >> > Given the volunteers (Pierre, Rafael, Luis) I would support the > TC using > >>> > >>> > >>> >> > its unholy powers to add extra core reviewers to cloudkitty. > >>> > >>> > >>> >> > >>> > >>> > >>> >> https://review.opendev.org/#/c/745653 is currently merging and > fungi will be > >>> > >>> > >>> >> adding Pierre as a core. > >>> > >>> > >>> >> > >>> > >>> > >>> >> Thank you for helping. > >>> > >>> > >>> >> > >>> > >>> > >>> >> > If the current PTL comes back, I'm sure they will appreciate the > help, > >>> > >>> > >>> >> > and can always fix/revert things before Victoria release. > >>> > >>> > >>> >> > > >>> > >>> > >>> >> > -- > >>> > >>> > >>> >> > Thierry Carrez (ttx) > >>> > >>> > >>> >> > > >>> > >>> > >>> >> > >>> > >>> > >>> >> > >>> > >>> > >>> >> -- > >>> > >>> > >>> >> Mohammed Naser > >>> > >>> > >>> >> VEXXHOST, Inc. > >>> > >>> > >>> >> > >>> > >>> > >>> > > >>> > >>> > >>> > > >>> > >>> > >>> > -- > >>> > >>> > >>> > Rafael Weingärtner > >>> > >>> > >> > >> > >> -- > >> Rafael Weingärtner > >> > >> > > -- > > Br, > > Luis Rmz > > Blockchain, DevOps & Open Source Cloud Solutions Architect > > ---------------------------------------- > > Founder & CEO > > OpenCloud.es > > luis.ramirez at opencloud.es > > Skype ID: d.overload > > Hangouts: luis.ramirez at opencloud.es > > +34 911 950 123 / +39 392 1289553 / +49 152 26917722 > -- Rafael Weingärtner -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Aug 13 12:14:06 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 13 Aug 2020 13:14:06 +0100 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> <16a3adf0-2f51-dd7d-c729-7b27f1593980@nemebean.com> Message-ID: <6e68d1a3cfc4efff91d3668bb53805dc469673c6.camel@redhat.com> On Thu, 2020-08-13 at 10:24 +0200, Thierry Carrez wrote: > Ben Nemec wrote: > > On 8/12/20 5:32 AM, Thierry Carrez wrote: > > > Sean Mooney wrote: > > > > On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: > > > > > I wonder if this does help though. It seems like a bug that a > > > > > nova-compute service would stop processing messages and still be > > > > > seen as up in the service status. Do we understand why that is > > > > > happening? If not, I'm unclear that a ping living at the > > > > > oslo.messaging layer is going to do a better job of exposing such an > > > > > outage. The fact that oslo.messaging is responding does not > > > > > necessarily equate to nova-compute functioning as expected. > > > > > > > > > > To be clear, this is not me nacking the ping feature. I just want to > > > > > make sure we understand what is going on here so we don't add > > > > > another unreliable healthchecking mechanism to the one we already have. > > > > > > > > [...] > > > > im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug > > > > that is motiviting the creation of this oslo ping > > > > feature but that feels premature if it is. i think it would be better > > > > try to adress this by the sender recreating the > > > > queue if the deliver fails and if that is not viable then protpyope > > > > thge fix in nova. if the self ping fixes this > > > > miss queue error then we could extract the cod into oslo. > > > > > > I think this is missing the point... This is not about working around > > > a specific bug, it's about adding a way to detect a certain class of > > > failure. It's more of an operational feature than a development bugfix. > > > > > > If I understood correctly, OVH is running that patch in production as > > > a way to detect certain problems they regularly run into, something > > > our existing monitor mechanisms fail to detect. That sounds like a > > > worthwhile addition? > > > > Okay, I don't think I was aware that this was already being used. If > > someone already finds it useful and it's opt-in then I'm not inclined to > > block it. My main concern was that we were adding a feature that didn't > > actually address the problem at hand. > > > > I _would_ feel better about it if someone could give an example of a > > type of failure this is detecting that is missed by other monitoring > > methods though. Both because having a concrete example of a use case for > > the feature is good, and because if it turns out that the problems this > > is detecting are things like the Nova bug Sean is talking about (which I > > don't think this would catch anyway, since the topic is missing and > > there's nothing to ping) then there may be other changes we can/should > > make to improve things. > > Right. Let's wait for Arnaud to come back from vacation and confirm that > > (1) that patch is not a shot in the dark: it allows them to expose a > class of issues in production > > (2) they fail to expose that same class of issues using other existing > mechanisms, including those just suggested in this thread > > I just wanted to avoid early rejection of this health check ability on > the grounds that the situation it exposes should just not happen. Or > that, if enabled and heavily used, it would have a performance impact. I think the inital push back from nova is we already have ping rpc function https://github.com/openstack/nova/blob/c6218428e9b29a2c52808ec7d27b4b21aadc0299/nova/baserpc.py#L55-L76 so if a geneirc metion called ping is added it will break nova. the reset of the push back is related to not haveing a concrete usecase, including concern over perfroamce consideration and external services potenailly acessing the rpc bus which is coniserd an internal api. e.g. we woudl not want an external monitoring solution connecting to the rpc bus and invoking arbitary RPC calls, ping is well pretty safe but form a design point of view while litening to notification is fine we dont want anything outside of the openstack services actully sending message on the rpc bus. so if this does actully detect somethign we can otherwise detect and the use cases involves using it within the openstack services not form an external source then i think that is fine but we proably need to use another name (alive? status?) or otherewise modify nova so that there is no conflict. > From dev.faz at gmail.com Thu Aug 13 13:13:45 2020 From: dev.faz at gmail.com (Fabian Zimmermann) Date: Thu, 13 Aug 2020 15:13:45 +0200 Subject: [nova][neutron][oslo][ops] rabbit bindings issue In-Reply-To: References: <20200806144016.GP31915@sync> <1a338d7e-c82c-cda2-2d47-b5aebb999142@openstack.org> Message-ID: Hi, just did some short tests today in our test-environment (without durable queues and without replication): * started a rally task to generate some load * kill-9-ed rabbitmq on one node * rally task immediately stopped and the cloud (mostly) stopped working after some debugging i found (again) exchanges which had bindings to queues, but these bindings didnt forward any msgs. Wrote a small script to detect these broken bindings and will now check if this is "reproducible" then I will try "durable queues" and "durable queues with replication" to see if this helps. Even if I would expect rabbitmq should be able to handle this without these "hidden broken bindings" This just FYI. Fabian -------------- next part -------------- An HTML attachment was scrubbed... URL: From moreira.belmiro.email.lists at gmail.com Thu Aug 13 13:15:41 2020 From: moreira.belmiro.email.lists at gmail.com (Belmiro Moreira) Date: Thu, 13 Aug 2020 15:15:41 +0200 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: References: Message-ID: Hi, we would really appreciate your comments on this. Especially the OSC team and all the project teams that are facing issues migrating their clients. Let us know, Belmiro On Mon, Aug 10, 2020 at 10:13 AM Belmiro Moreira < moreira.belmiro.email.lists at gmail.com> wrote: > Hi, > during the last PTG the TC discussed the problem of supporting different > clients (OpenStack Client - OSC vs python-*clients) [1]. > Currently, we don't have feature parity between the OSC and the > python-*clients. > > Different OpenStack projects invest in different clients. > This can be a huge problem for users/ops. Depending on the projects > deployed in their infrastructures, they need to use different clients for > different tasks. > It's confusing because of the partial implementation in the OSC. > > There was also the proposal to enforce new functionality only in the SDK > (and optionally the OSC) and not the project’s specific clients to stop > increasing the disparity between the two. > > We would like to understand first the problems and missing pieces that > projects are facing to move into OSC and help to overcome them. > Let us know. > > Belmiro, > on behalf of the TC > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015418.html > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ekuvaja at redhat.com Thu Aug 13 13:36:34 2020 From: ekuvaja at redhat.com (Erno Kuvaja) Date: Thu, 13 Aug 2020 14:36:34 +0100 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: References: Message-ID: On Thu, Aug 13, 2020 at 2:19 PM Belmiro Moreira < moreira.belmiro.email.lists at gmail.com> wrote: > Hi, > we would really appreciate your comments on this. > Especially the OSC team and all the project teams that are facing issues > migrating their clients. > > Let us know, > Belmiro > > In Glance perspective we already stated that we're more than happy to try endorsing osc again once it has stabilized the Images API facing code and maintained feature parity for a few cycles. Just stopping developing python-glanceclient will only result in no up-to-date stable client for Images API developed under OpenStack Governance. I really don't think forcing to fork python-glanceclient to keep development going outside of OpenStack Governance will be the better solution here. - jokke On Mon, Aug 10, 2020 at 10:13 AM Belmiro Moreira < > moreira.belmiro.email.lists at gmail.com> wrote: > >> Hi, >> during the last PTG the TC discussed the problem of supporting different >> clients (OpenStack Client - OSC vs python-*clients) [1]. >> Currently, we don't have feature parity between the OSC and the >> python-*clients. >> >> Different OpenStack projects invest in different clients. >> This can be a huge problem for users/ops. Depending on the projects >> deployed in their infrastructures, they need to use different clients for >> different tasks. >> It's confusing because of the partial implementation in the OSC. >> >> There was also the proposal to enforce new functionality only in the SDK >> (and optionally the OSC) and not the project’s specific clients to stop >> increasing the disparity between the two. >> >> We would like to understand first the problems and missing pieces that >> projects are facing to move into OSC and help to overcome them. >> Let us know. >> >> Belmiro, >> on behalf of the TC >> >> [1] >> http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015418.html >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwm2012 at gmail.com Thu Aug 13 13:38:20 2020 From: pwm2012 at gmail.com (pwm) Date: Thu, 13 Aug 2020 21:38:20 +0800 Subject: DNS server instead of /etc/hosts file on Infra Server In-Reply-To: <7db29753-4710-a979-fe71-67a829fa55c3@rd.bbc.co.uk> References: <7db29753-4710-a979-fe71-67a829fa55c3@rd.bbc.co.uk> Message-ID: Great will check it out. Thanks, Jonathan. On Wed, Aug 12, 2020 at 9:39 PM Jonathan Rosser < jonathan.rosser at rd.bbc.co.uk> wrote: > Openstack-Ansible already supports optionally using the unbound dns server > instead of managing > /etc/hosts. > > Join #openstack-ansible on IRC if you need any help. > > Regards, > Jonathan. > On 12/08/2020 14:03, pwm wrote: > > Hi, > Plan to use PowerDNS server instead of the /etc/hosts file for resolving. > Has anyone done this before? > The PowerDNS support MySQL DB backend and a frontend GUI PowerDNS Admin > which allows centralized easy maintenance. > > Thanks > > On Sun, Aug 9, 2020 at 11:54 PM pwm wrote: > >> Hi, >> Anyone interested in replacing the /etc/hosts file entry with a DNS >> server on the openstack-ansible deployment? >> >> Thank you >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From luis.ramirez at opencloud.es Thu Aug 13 13:59:49 2020 From: luis.ramirez at opencloud.es (Luis Ramirez) Date: Thu, 13 Aug 2020 15:59:49 +0200 Subject: [cloudkitty][tc] Cloudkitty abandoned? In-Reply-To: References: <173c942a17b.dfe050d2111458.180813585646259079@ghanshyammann.com> Message-ID: Great! I'll try to do the same. Br, Luis Rmz Blockchain, DevOps & Open Source Cloud Solutions Architect ---------------------------------------- Founder & CEO OpenCloud.es luis.ramirez at opencloud.es Skype ID: d.overload Hangouts: luis.ramirez at opencloud.es [image: ] +34 911 950 123 / [image: ]+39 392 1289553 / [image: ]+49 152 26917722 / Česká republika: +420 774 274 882 ----------------------------------------------------- El jue., 13 ago. 2020 a las 13:44, Rafael Weingärtner (< rafaelweingartner at gmail.com>) escribió: > Awesome, thanks! > I will try to dedicate a few hours every week to review CloudKitty patches. > > On Thu, Aug 13, 2020 at 8:36 AM Pierre Riteau wrote: > >> Thank you both. >> >> I've merged a few patches to fix CI and finalise the Ussuri release >> (for example release notes were missing). >> I gave core reviewer privileges to Rafael and Luis. Let's try to merge >> patches with two +2 votes from now on. >> >> On Wed, 12 Aug 2020 at 23:38, Luis Ramirez >> wrote: >> > >> > Sounds good to me. >> > >> > El El mié, 12 ago 2020 a las 22:41, Rafael Weingärtner < >> rafaelweingartner at gmail.com> escribió: >> >> >> >> Sounds good to me. >> >> >> >> On Wed, Aug 12, 2020 at 5:37 PM Pierre Riteau >> wrote: >> >>> >> >>> I have now received core reviewer privileges. Thank you to TC members >> >>> >> >>> >> >>> for trusting us with the CloudKitty project. >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> I would like to kick things off by resuming IRC meetings. They're set >> >>> >> >>> >> >>> to run every two weeks (on odd weeks) on Monday at 1400 UTC in >> >>> >> >>> >> >>> #cloudkitty. Is this a convenient time slot for all potential >> >>> >> >>> >> >>> contributors to the project? >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> On Wed, 12 Aug 2020 at 19:08, Rafael Weingärtner >> >>> >> >>> >> >>> wrote: >> >>> >> >>> >> >>> > >> >>> >> >>> >> >>> > Awesome! Thank you guys for the help. >> >>> >> >>> >> >>> > We have few PRs open there that are ready (or close to be ready) to >> be merged. >> >>> >> >>> >> >>> > >> >>> >> >>> >> >>> > On Wed, Aug 12, 2020 at 1:59 PM Mohammed Naser >> wrote: >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> >> On Tue, Aug 11, 2020 at 6:16 AM Thierry Carrez < >> thierry at openstack.org> wrote: >> >>> >> >>> >> >>> >> > >> >>> >> >>> >> >>> >> > Thomas Goirand wrote: >> >>> >> >>> >> >>> >> > > On 8/7/20 4:10 PM, Ghanshyam Mann wrote: >> >>> >> >>> >> >>> >> > >> Thanks, Pierre for helping with this. >> >>> >> >>> >> >>> >> > >> >> >>> >> >>> >> >>> >> > >> ttx has reached out to PTL (Justin Ferrieu (jferrieu) < >> justin.ferrieu at objectif-libre.com>) >> >>> >> >>> >> >>> >> > >> but I am not sure if he got any response back. >> >>> >> >>> >> >>> >> > >> >>> >> >>> >> >>> >> > No response so far, but they may all be in company summer >> vacation. >> >>> >> >>> >> >>> >> > >> >>> >> >>> >> >>> >> > > The end of the very good maintenance of Cloudkitty matched the >> date when >> >>> >> >>> >> >>> >> > > objectif libre was sold to Linkbynet. Maybe the new owner >> don't care enough? >> >>> >> >>> >> >>> >> > > >> >>> >> >>> >> >>> >> > > This is very disappointing as I've been using it for some time >> already, >> >>> >> >>> >> >>> >> > > and that I was satisfied by it (ie: it does the job...), and >> especially >> >>> >> >>> >> >>> >> > > that latest releases are able to scale correctly. >> >>> >> >>> >> >>> >> > > >> >>> >> >>> >> >>> >> > > I very much would love if Pierre Riteau was successful in >> taking over. >> >>> >> >>> >> >>> >> > > Good luck Pierre! I'll try to help whenever I can and if I'm >> not too busy. >> >>> >> >>> >> >>> >> > >> >>> >> >>> >> >>> >> > Given the volunteers (Pierre, Rafael, Luis) I would support the >> TC using >> >>> >> >>> >> >>> >> > its unholy powers to add extra core reviewers to cloudkitty. >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> >> https://review.opendev.org/#/c/745653 is currently merging and >> fungi will be >> >>> >> >>> >> >>> >> adding Pierre as a core. >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> >> Thank you for helping. >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> >> > If the current PTL comes back, I'm sure they will appreciate the >> help, >> >>> >> >>> >> >>> >> > and can always fix/revert things before Victoria release. >> >>> >> >>> >> >>> >> > >> >>> >> >>> >> >>> >> > -- >> >>> >> >>> >> >>> >> > Thierry Carrez (ttx) >> >>> >> >>> >> >>> >> > >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> >> -- >> >>> >> >>> >> >>> >> Mohammed Naser >> >>> >> >>> >> >>> >> VEXXHOST, Inc. >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >>> > >> >>> >> >>> >> >>> > >> >>> >> >>> >> >>> > -- >> >>> >> >>> >> >>> > Rafael Weingärtner >> >>> >> >>> >> >> >> >> >> >> -- >> >> Rafael Weingärtner >> >> >> >> >> > -- >> > Br, >> > Luis Rmz >> > Blockchain, DevOps & Open Source Cloud Solutions Architect >> > ---------------------------------------- >> > Founder & CEO >> > OpenCloud.es >> > luis.ramirez at opencloud.es >> > Skype ID: d.overload >> > Hangouts: luis.ramirez at opencloud.es >> > +34 911 950 123 / +39 392 1289553 / +49 152 26917722 >> > > > -- > Rafael Weingärtner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ashlee at openstack.org Thu Aug 13 14:07:40 2020 From: ashlee at openstack.org (Ashlee Ferguson) Date: Thu, 13 Aug 2020 09:07:40 -0500 Subject: Community Voting for The Virtual Summit Sessions is Open! Message-ID: <02EC11B6-6D57-4DFA-94B6-F920E90A4FF6@openstack.org> Community voting for the virtual Open Infrastructure Summit sessions is open! You can VOTE HERE , but what does that mean? Now that the Call for Presentations has closed, all submissions are available for community vote and input. After community voting closes, the volunteer Programming Committee members will receive the results to review to help them determine the final selections for Summit schedule. While community votes are meant to help inform the decision, Programming Committee members are expected to exercise judgment in their area of expertise and help ensure diversity of sessions and speakers. View full details of the session selection process . In order to vote, you need an OSF community membership. If you do not have an account, please create one by going to openstack.org/join . If you need to reset your password, you can do that here . Hurry, voting closes Monday, August 17 at 11:59pm Pacific Time. Don’t forget to Register for the Summit for free! Visit https://www.openstack.org/summit/2020/ for all other Summit-related information. Interested in sponsoring? Visit this page . If you have any questions, please email summit at openstack.org . Cheers, Ashlee Ashlee Ferguson Community & Events Coordinator OpenStack Foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: From jasowang at redhat.com Thu Aug 13 04:24:50 2020 From: jasowang at redhat.com (Jason Wang) Date: Thu, 13 Aug 2020 12:24:50 +0800 Subject: device compatibility interface for live migration with assigned devices In-Reply-To: <20200810074631.GA29059@joy-OptiPlex-7040> References: <20200727162321.7097070e@x1.home> <20200729080503.GB28676@joy-OptiPlex-7040> <20200804183503.39f56516.cohuck@redhat.com> <20200805021654.GB30485@joy-OptiPlex-7040> <2624b12f-3788-7e2b-2cb7-93534960bcb7@redhat.com> <20200805075647.GB2177@nanopsycho> <20200805093338.GC30485@joy-OptiPlex-7040> <20200805105319.GF2177@nanopsycho> <20200810074631.GA29059@joy-OptiPlex-7040> Message-ID: On 2020/8/10 下午3:46, Yan Zhao wrote: >> driver is it handled by? > It looks that the devlink is for network device specific, and in > devlink.h, it says > include/uapi/linux/devlink.h - Network physical device Netlink > interface, Actually not, I think there used to have some discussion last year and the conclusion is to remove this comment. It supports IB and probably vDPA in the future. > I feel like it's not very appropriate for a GPU driver to use > this interface. Is that right? I think not though most of the users are switch or ethernet devices. It doesn't prevent you from inventing new abstractions. Note that devlink is based on netlink, netlink has been widely used by various subsystems other than networking. Thanks > > Thanks > Yan > > From alifshit at redhat.com Thu Aug 13 14:30:49 2020 From: alifshit at redhat.com (Artom Lifshitz) Date: Thu, 13 Aug 2020 10:30:49 -0400 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: <1668118.VLH7GnMWUR@whitebase.usersys.redhat.com> References: <1668118.VLH7GnMWUR@whitebase.usersys.redhat.com> Message-ID: On Mon, Aug 10, 2020 at 4:40 AM Luigi Toscano wrote: > > On Monday, 10 August 2020 10:26:24 CEST Radosław Piliszek wrote: > > On Mon, Aug 10, 2020 at 10:19 AM Belmiro Moreira < > > > > moreira.belmiro.email.lists at gmail.com> wrote: > > > Hi, > > > during the last PTG the TC discussed the problem of supporting different > > > clients (OpenStack Client - OSC vs python-*clients) [1]. > > > Currently, we don't have feature parity between the OSC and the > > > python-*clients. > > > > Is it true of any client? I guess some are just OSC plugins 100%. > > Do we know which clients have this disparity? > > Personally, I encountered this with Glance the most and Cinder to some > > extent (but I believe over the course of action Cinder got all features I > > wanted from it in the OSC). > > As far as I know there is still a huge problem with microversion handling > which impacts some cinder features. It has been discussed in the past and > still present. Yeah, my understanding is that osc will never "properly" support microversions. Openstacksdk is the future in that sense, and my understanding is that the osc team is "porting" osc to use the sdk. Given these two thing, when we (Nova) talked about this with the osc folks, we decided that rather than catch up osc to python-novaclient, we'd rather focus our efforts on the sdk. I've been slowly doing that [1], starting with the earlier Nova microversions. The eventual long term goal is for the Nova team to *only* support the sdk, and drop python-novaclient entirely, but that's a long time away. [1] https://review.opendev.org/#/q/status:open+project:openstack/openstacksdk+branch:master+topic:story/2007929 > > > -- > Luigi > > > From moguimar at redhat.com Thu Aug 13 15:06:31 2020 From: moguimar at redhat.com (Moises Guimaraes de Medeiros) Date: Thu, 13 Aug 2020 17:06:31 +0200 Subject: [oslo] Proposing Lance Bragstad as oslo.cache core Message-ID: Hello everybody, It is my pleasure to propose Lance Bragstad (lbragstad) as a new member of the oslo.core core team. Lance has been a big contributor to the project and is known as a walking version of the Keystone documentation, which happens to be one of the biggest consumers of oslo.cache. Obviously we think he'd make a good addition to the core team. If there are no objections, I'll make that happen in a week. Thanks. -- Moisés Guimarães Software Engineer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Thu Aug 13 15:08:31 2020 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 13 Aug 2020 10:08:31 -0500 Subject: [oslo] Proposing Lance Bragstad as oslo.cache core In-Reply-To: References: Message-ID: <82be76d9-0898-63a7-d339-48e9f6db540c@gmx.com> On 8/13/20 10:06 AM, Moises Guimaraes de Medeiros wrote: > Hello everybody, > > It is my pleasure to propose Lance Bragstad (lbragstad) as a new > member of the oslo.core core team. > > Lance has been a big contributor to the project and is known as a > walking version of the Keystone documentation, which happens to be one > of the biggest consumers of oslo.cache. > > Obviously we think he'd make a good addition to the core team. If > there are no objections, I'll make that happen in a week. > +1! -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Aug 13 15:18:45 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 13 Aug 2020 16:18:45 +0100 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: References: <1668118.VLH7GnMWUR@whitebase.usersys.redhat.com> Message-ID: <9cbf9d69a9beb30d03af71e42a3e2446a516292a.camel@redhat.com> On Thu, 2020-08-13 at 10:30 -0400, Artom Lifshitz wrote: > On Mon, Aug 10, 2020 at 4:40 AM Luigi Toscano wrote: > > > > On Monday, 10 August 2020 10:26:24 CEST Radosław Piliszek wrote: > > > On Mon, Aug 10, 2020 at 10:19 AM Belmiro Moreira < > > > > > > moreira.belmiro.email.lists at gmail.com> wrote: > > > > Hi, > > > > during the last PTG the TC discussed the problem of supporting different > > > > clients (OpenStack Client - OSC vs python-*clients) [1]. > > > > Currently, we don't have feature parity between the OSC and the > > > > python-*clients. > > > > > > Is it true of any client? I guess some are just OSC plugins 100%. > > > Do we know which clients have this disparity? > > > Personally, I encountered this with Glance the most and Cinder to some > > > extent (but I believe over the course of action Cinder got all features I > > > wanted from it in the OSC). > > > > As far as I know there is still a huge problem with microversion handling > > which impacts some cinder features. It has been discussed in the past and > > still present. > > Yeah, my understanding is that osc will never "properly" support > microversions. it does already properly support micorversion the issue is not everyone agrees on what properly means. the behavior of the project clients was considered broken by many. it has been poirpose that we explcity allow a way to opt in to the auto negociation via a new "auto" sentaial value and i have also suggested that we should tag each comman with the minium microversion that parmater or command requires and decault to that minium based on teh arges you passed. both of those imporvement dont break the philosipy of providing stable cli behavior across cloud and would imporve the ux. defaulting to the minium microversion needed for the arguments passed would solve most of the ux issues and adding an auto sentical would resolve the rest while still keeping the correct microversion behvior it already has. the glance and cinder gaps are not really related to microverions by the way. its just that no one has done the work and cinder an glance have not require contiuptors to update osc as part of adding new features. nova has not required that either but there were some who worked on nova that cared enough about osc to mention it in code review or submit patches themsevles. the glance team does not really have the resouces to do that and the osc team does not have the resouce to maintain clis for all teams. so over tiem as service poject added new feature the gaps have increase since there were not people tyring to keep it in sync. > Openstacksdk is the future in that sense, and my > understanding is that the osc team is "porting" osc to use the sdk. > Given these two thing, when we (Nova) talked about this with the osc > folks, we decided that rather than catch up osc to python-novaclient, > we'd rather focus our efforts on the sdk. well that is not entirly a good caraterisation. we want to catch up osc too but the suggest was to support eveything in osc then it would be easier to add osc support since it just has to call the sdk functions. we did not say we dont want to close the gaps in osc. > I've been slowly doing that > [1], starting with the earlier Nova microversions. The eventual long > term goal is for the Nova team to *only* support the sdk, and drop > python-novaclient entirely, but that's a long time away. > > [1] https://review.opendev.org/#/q/status:open+project:openstack/openstacksdk+branch:master+topic:story/2007929 > > > > > > > -- > > Luigi > > > > > > > > From openstack at nemebean.com Thu Aug 13 15:28:12 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 13 Aug 2020 10:28:12 -0500 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: <6e68d1a3cfc4efff91d3668bb53805dc469673c6.camel@redhat.com> References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> <16a3adf0-2f51-dd7d-c729-7b27f1593980@nemebean.com> <6e68d1a3cfc4efff91d3668bb53805dc469673c6.camel@redhat.com> Message-ID: On 8/13/20 7:14 AM, Sean Mooney wrote: > On Thu, 2020-08-13 at 10:24 +0200, Thierry Carrez wrote: >> Ben Nemec wrote: >>> On 8/12/20 5:32 AM, Thierry Carrez wrote: >>>> Sean Mooney wrote: >>>>> On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: >>>>>> I wonder if this does help though. It seems like a bug that a >>>>>> nova-compute service would stop processing messages and still be >>>>>> seen as up in the service status. Do we understand why that is >>>>>> happening? If not, I'm unclear that a ping living at the >>>>>> oslo.messaging layer is going to do a better job of exposing such an >>>>>> outage. The fact that oslo.messaging is responding does not >>>>>> necessarily equate to nova-compute functioning as expected. >>>>>> >>>>>> To be clear, this is not me nacking the ping feature. I just want to >>>>>> make sure we understand what is going on here so we don't add >>>>>> another unreliable healthchecking mechanism to the one we already have. >>>>> >>>>> [...] >>>>> im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug >>>>> that is motiviting the creation of this oslo ping >>>>> feature but that feels premature if it is. i think it would be better >>>>> try to adress this by the sender recreating the >>>>> queue if the deliver fails and if that is not viable then protpyope >>>>> thge fix in nova. if the self ping fixes this >>>>> miss queue error then we could extract the cod into oslo. >>>> >>>> I think this is missing the point... This is not about working around >>>> a specific bug, it's about adding a way to detect a certain class of >>>> failure. It's more of an operational feature than a development bugfix. >>>> >>>> If I understood correctly, OVH is running that patch in production as >>>> a way to detect certain problems they regularly run into, something >>>> our existing monitor mechanisms fail to detect. That sounds like a >>>> worthwhile addition? >>> >>> Okay, I don't think I was aware that this was already being used. If >>> someone already finds it useful and it's opt-in then I'm not inclined to >>> block it. My main concern was that we were adding a feature that didn't >>> actually address the problem at hand. >>> >>> I _would_ feel better about it if someone could give an example of a >>> type of failure this is detecting that is missed by other monitoring >>> methods though. Both because having a concrete example of a use case for >>> the feature is good, and because if it turns out that the problems this >>> is detecting are things like the Nova bug Sean is talking about (which I >>> don't think this would catch anyway, since the topic is missing and >>> there's nothing to ping) then there may be other changes we can/should >>> make to improve things. >> >> Right. Let's wait for Arnaud to come back from vacation and confirm that >> >> (1) that patch is not a shot in the dark: it allows them to expose a >> class of issues in production >> >> (2) they fail to expose that same class of issues using other existing >> mechanisms, including those just suggested in this thread >> >> I just wanted to avoid early rejection of this health check ability on >> the grounds that the situation it exposes should just not happen. Or >> that, if enabled and heavily used, it would have a performance impact. > I think the inital push back from nova is we already have ping rpc function > https://github.com/openstack/nova/blob/c6218428e9b29a2c52808ec7d27b4b21aadc0299/nova/baserpc.py#L55-L76 > so if a geneirc metion called ping is added it will break nova. It occurred to me after I commented on the review that we have tempest running on oslo.messaging changes and it passed on the patch for this. I suppose it's possible that it broke some error handling in Nova that just isn't tested, but maybe the new ping could function as a cross-project replacement for the Nova ping? Anyway, it's still be to deduplicate the name, but I felt kind of dumb about having asked if it was tested when the test results were right in front of me. ;-) > > the reset of the push back is related to not haveing a concrete usecase, including concern over > perfroamce consideration and external services potenailly acessing the rpc bus which is coniserd an internal > api. e.g. we woudl not want an external monitoring solution connecting to the rpc bus and invoking arbitary > RPC calls, ping is well pretty safe but form a design point of view while litening to notification is fine > we dont want anything outside of the openstack services actully sending message on the rpc bus. I'm not concerned about the performance impact here. It's an optional feature, so anyone using it is choosing to take that hit. Having external stuff on the RPC bus is more of a gray area, but it's not like we can stop operators from doing that. I think it's probably better to provide a well-defined endpoint for them to talk to rather than have everyone implement their own slightly different RPC ping mechanism. The docs for this feature should be very explicit that this is the only thing external code should be calling. > > so if this does actully detect somethign we can otherwise detect and the use cases involves using it within > the openstack services not form an external source then i think that is fine but we proably need to use another > name (alive? status?) or otherewise modify nova so that there is no conflict. >> > If I understand your analysis of the bug correctly, this would have caught that type of outage after all since the failure was asymmetric. The compute node was still able to send its status updates to Nova, but wasn't receiving any messages. A ping would have detected that situation. From openstack at nemebean.com Thu Aug 13 15:28:43 2020 From: openstack at nemebean.com (Ben Nemec) Date: Thu, 13 Aug 2020 10:28:43 -0500 Subject: [oslo] Proposing Lance Bragstad as oslo.cache core In-Reply-To: References: Message-ID: <305f4be5-950d-97a6-63d0-fe59d56bfe8d@nemebean.com> +1! On 8/13/20 10:06 AM, Moises Guimaraes de Medeiros wrote: > Hello everybody, > > It is my pleasure to propose Lance Bragstad (lbragstad) as a new member > of the oslo.core core team. > > Lance has been a big contributor to the project and is known as a > walking version of the Keystone documentation, which happens to be one > of the biggest consumers of oslo.cache. > > Obviously we think he'd make a good addition to the core team. If there > are no objections, I'll make that happen in a week. > > Thanks. > > -- > > Moisés Guimarães > > Software Engineer > > Red Hat > > > From allison at openstack.org Thu Aug 13 15:28:55 2020 From: allison at openstack.org (Allison Price) Date: Thu, 13 Aug 2020 10:28:55 -0500 Subject: Running OpenStack? Take the 2020 User Survey now! Message-ID: <1E70817D-D322-4B3E-8940-762DBAA7708A@openstack.org> Hi everyone, There is only one week left before we are closing the 2020 OpenStack User Survey [1]! If you are running OpenStack, please take a few minutes to log your deployment—all information will remain anonymous unless you indicate otherwise. If you have completed a User Survey before, all you have to do is update your information and answer a few new questions. Anonymous feedback will be passed along to the upstream project teams, and anonymized data will be available in the analytics dashboard [2]. The deadline to add your deployment to this round of analysis is Thursday, August 20. Let me know if you have any questions or issues completing. Thanks! Allison [1] https://www.openstack.org/user-survey/survey-2020 [2] https://www.openstack.org/analytics -------------- next part -------------- An HTML attachment was scrubbed... URL: From abishop at redhat.com Thu Aug 13 15:42:30 2020 From: abishop at redhat.com (Alan Bishop) Date: Thu, 13 Aug 2020 08:42:30 -0700 Subject: [all][TC] OpenStack Client (OSC) vs python-*clients In-Reply-To: <9cbf9d69a9beb30d03af71e42a3e2446a516292a.camel@redhat.com> References: <1668118.VLH7GnMWUR@whitebase.usersys.redhat.com> <9cbf9d69a9beb30d03af71e42a3e2446a516292a.camel@redhat.com> Message-ID: On Thu, Aug 13, 2020 at 8:27 AM Sean Mooney wrote: > On Thu, 2020-08-13 at 10:30 -0400, Artom Lifshitz wrote: > > On Mon, Aug 10, 2020 at 4:40 AM Luigi Toscano > wrote: > > > > > > On Monday, 10 August 2020 10:26:24 CEST Radosław Piliszek wrote: > > > > On Mon, Aug 10, 2020 at 10:19 AM Belmiro Moreira < > > > > > > > > moreira.belmiro.email.lists at gmail.com> wrote: > > > > > Hi, > > > > > during the last PTG the TC discussed the problem of supporting > different > > > > > clients (OpenStack Client - OSC vs python-*clients) [1]. > > > > > Currently, we don't have feature parity between the OSC and the > > > > > python-*clients. > > > > > > > > Is it true of any client? I guess some are just OSC plugins 100%. > > > > Do we know which clients have this disparity? > > > > Personally, I encountered this with Glance the most and Cinder to > some > > > > extent (but I believe over the course of action Cinder got all > features I > > > > wanted from it in the OSC). > > > > > > As far as I know there is still a huge problem with microversion > handling > > > which impacts some cinder features. It has been discussed in the past > and > > > still present. > > > > Yeah, my understanding is that osc will never "properly" support > > microversions. > it does already properly support micorversion the issue is not everyone > agrees > on what properly means. the behavior of the project clients was considered > broken > by many. it has been poirpose that we explcity allow a way to opt in to > the auto negociation > via a new "auto" sentaial value and i have also suggested that we should > tag each comman with the minium > microversion that parmater or command requires and decault to that minium > based on teh arges you passed. > > both of those imporvement dont break the philosipy of providing stable cli > behavior across cloud and would > imporve the ux. defaulting to the minium microversion needed for the > arguments passed would solve most of the ux > issues and adding an auto sentical would resolve the rest while still > keeping the correct microversion behvior it > already has. > > the glance and cinder gaps are not really related to microverions by the > way. > its just that no one has done the work and cinder an glance have not > require contiuptors to update > Updates to osc from cinder's side are pretty much stalled due to lack of support for microversions. A patch for that was rejected and we've had trouble getting an update on a viable path forward. See comment in https://review.opendev.org/590807. Alan > osc as part of adding new features. nova has not required that either but > there were some who worked on nova > that cared enough about osc to mention it in code review or submit patches > themsevles. the glance team does > not really have the resouces to do that and the osc team does not have the > resouce to maintain clis for all teams. > > so over tiem as service poject added new feature the gaps have increase > since there were not people tyring to keep it in > sync. > > > Openstacksdk is the future in that sense, and my > > understanding is that the osc team is "porting" osc to use the sdk. > > Given these two thing, when we (Nova) talked about this with the osc > > folks, we decided that rather than catch up osc to python-novaclient, > > we'd rather focus our efforts on the sdk. > well that is not entirly a good caraterisation. we want to catch up osc too > but the suggest was to support eveything in osc then it would be easier to > add osc support > since it just has to call the sdk functions. we did not say we dont want > to close the gaps in osc. > > > I've been slowly doing that > > [1], starting with the earlier Nova microversions. The eventual long > > term goal is for the Nova team to *only* support the sdk, and drop > > python-novaclient entirely, but that's a long time away. > > > > [1] > https://review.opendev.org/#/q/status:open+project:openstack/openstacksdk+branch:master+topic:story/2007929 > > > > > > > > > > > -- > > > Luigi > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Thu Aug 13 16:07:18 2020 From: smooney at redhat.com (Sean Mooney) Date: Thu, 13 Aug 2020 17:07:18 +0100 Subject: [largescale-sig][nova][neutron][oslo] RPC ping In-Reply-To: References: <20200727095744.GK31915@sync> <3d238530-6c84-d611-da4c-553ba836fc02@nemebean.com> <671fec63-8bea-4215-c773-d8360e368a99@sap.com> <2af09e63936f75489946ea6b70c41d6e091531ee.camel@redhat.com> <7496bd35-856e-f48f-b6d8-65155b1777f1@openstack.org> <16a3adf0-2f51-dd7d-c729-7b27f1593980@nemebean.com> <6e68d1a3cfc4efff91d3668bb53805dc469673c6.camel@redhat.com> Message-ID: <65204b738f13fcea16b9b6d5a68149c89be73e6a.camel@redhat.com> On Thu, 2020-08-13 at 10:28 -0500, Ben Nemec wrote: > > On 8/13/20 7:14 AM, Sean Mooney wrote: > > On Thu, 2020-08-13 at 10:24 +0200, Thierry Carrez wrote: > > > Ben Nemec wrote: > > > > On 8/12/20 5:32 AM, Thierry Carrez wrote: > > > > > Sean Mooney wrote: > > > > > > On Tue, 2020-08-11 at 15:20 -0500, Ben Nemec wrote: > > > > > > > I wonder if this does help though. It seems like a bug that a > > > > > > > nova-compute service would stop processing messages and still be > > > > > > > seen as up in the service status. Do we understand why that is > > > > > > > happening? If not, I'm unclear that a ping living at the > > > > > > > oslo.messaging layer is going to do a better job of exposing such an > > > > > > > outage. The fact that oslo.messaging is responding does not > > > > > > > necessarily equate to nova-compute functioning as expected. > > > > > > > > > > > > > > To be clear, this is not me nacking the ping feature. I just want to > > > > > > > make sure we understand what is going on here so we don't add > > > > > > > another unreliable healthchecking mechanism to the one we already have. > > > > > > > > > > > > [...] > > > > > > im not sure https://bugs.launchpad.net/nova/+bug/1854992 is the bug > > > > > > that is motiviting the creation of this oslo ping > > > > > > feature but that feels premature if it is. i think it would be better > > > > >