From ngtech1ltd at gmail.com Sun Dec 1 00:09:37 2019 From: ngtech1ltd at gmail.com (ngtech1ltd at gmail.com) Date: Sun, 1 Dec 2019 02:09:37 +0200 Subject: Openstack - Rocky Install In-Reply-To: References: Message-ID: <014c01d5a7db$9f807510$de815f30$@gmail.com> The issue is with the python syntax. However since last week I had trouble understanding the basic installation steps and their order in the docs, I tried to use: https://www.server-world.info/en/note?os=CentOS_7 &p=openstack_rocky&f=2 As my installation refence for a minimal installation. It helped me a lot to install both rocky and train versions. Hope it helps, Eliezer ---- Eliezer Croitoru Linux System Administrator Mobile: +972-5-28704261 Email: ngtech1ltd at gmail.com From: doug schmidt Sent: Saturday, November 30, 2019 4:53 PM To: openstack-discuss at lists.openstack.org Subject: Openstack - Rocky Install Hi, I'm having an issue with httpd not starting on my install. I have installed rocky on a windows 10 laptop with virtualbox and centos 7 guests. So far I have 2 controllers and 3 compute nodes. I have followed the install docs for rocky. https://docs.openstack.org/install-guide/openstack-services.html#minimal-deployment-for-rocky Minimal services have been installed and configured.keystone, glance, nova, neutron are configured and running. I followed the install for horizon dashboard, and that is where I'm getting httpd.service failing. I have gone over the configuration a few times, but I can not find what the issue is. Any ideas of where to look or what to fix? Thanks for any help with this ------------------------ [root at openstack-cntr1 ~]# systemctl | egrep -i 'openstack|neutron|httpd' ● httpd.service loaded failed failed The Apache HTTP Server neutron-dhcp-agent.service loaded active running OpenStack Neutron DHCP Agent neutron-linuxbridge-agent.service loaded active running OpenStack Neutron Linux Bridge Agent neutron-metadata-agent.service loaded active running OpenStack Neutron Metadata Agent neutron-server.service loaded active running OpenStack Neutron Server openstack-glance-api.service loaded active running OpenStack Image Service (code-named Glance) API server openstack-glance-registry.service loaded active running OpenStack Image Service (code-named Glance) Registry server openstack-nova-api.service loaded active running OpenStack Nova API Server openstack-nova-conductor.service loaded active running OpenStack Nova Conductor Server openstack-nova-consoleauth.service loaded active running OpenStack Nova VNC console auth Server openstack-nova-novncproxy.service loaded active running OpenStack Nova NoVNC Proxy Server openstack-nova-scheduler.service loaded active running OpenStack Nova Scheduler Server [root at openstack-cntr1 ~]# systemctl start httpd.service Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details. [root at openstack-cntr1 ~]# systemctl status httpd.service ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d └─openstack-dashboard.conf Active: failed (Result: exit-code) since Sat 2019-11-30 09:45:46 EST; 15s ago Docs: man:httpd(8) man:apachectl(8) Process: 12855 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear -v0 (code=exited, status=1/FAILURE) Nov 30 09:45:46 openstack-cntr1 python[12855]: File "/usr/share/openstack-dashboard/openstack_dashboard/settings.py", line 376, in Nov 30 09:45:46 openstack-cntr1 python[12855]: from local.local_settings import * # noqa: F403,H303 Nov 30 09:45:46 openstack-cntr1 python[12855]: File "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py", line 399 Nov 30 09:45:46 openstack-cntr1 python[12855]: 'supported_vnic_types': ['*'], Nov 30 09:45:46 openstack-cntr1 python[12855]: ^ Nov 30 09:45:46 openstack-cntr1 python[12855]: IndentationError: unexpected indent Nov 30 09:45:46 openstack-cntr1 systemd[1]: httpd.service: control process exited, code=exited status=1 Nov 30 09:45:46 openstack-cntr1 systemd[1]: Failed to start The Apache HTTP Server. Nov 30 09:45:46 openstack-cntr1 systemd[1]: Unit httpd.service entered failed state. Nov 30 09:45:46 openstack-cntr1 systemd[1]: httpd.service failed. [root at openstack-cntr1 ~]# -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 11295 bytes Desc: not available URL: From skaplons at redhat.com Sun Dec 1 10:44:10 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Sun, 1 Dec 2019 11:44:10 +0100 Subject: State of the Gate In-Reply-To: References: <20191031211535.vk7rtiq3pvsb6j2t@skaplons-mac> Message-ID: <262BDACC-9139-4B66-9E5B-F95CF84BD324@redhat.com> Hi, > On 31 Oct 2019, at 23:03, Matt Riedemann wrote: > > On 10/31/2019 4:15 PM, Slawek Kaplonski wrote: >>> 3. CirrOS guest SSH issues >>> >>> There are several (some might be duplicates): >>> >>> http://status.openstack.org/elastic-recheck/index.html#1848078 >> This one is I think the same as we have reported in >> https://bugs.launchpad.net/neutron/+bug/1850557 >> Basically we noticed issues with dhcp after resize/migration/shelve of instance >> but I didn't have time to investigate it yet. > > Hmm, https://review.opendev.org/#/c/670591/ is new to devstack in Train and was backported to stable/stein. I wonder if that is too aggressive and is causing issues with operations where the guest is stopped and started, though for resize/migrate/shelve/unshelve the guest is destroyed on one host and re-spawned on another, so I would think that having a graceful shutdown for the guest wouldn't matter in those cases, unless it has to do with leaving the guest "dirty" somehow before transferring the root disk / creating a snapshot (in the case of shelve). Maybe we should bump that back up to 10 seconds? I finally spent some time on investigating this issue. I was able to reproduce it locally and I found that the problem is in openvswitch firewall. All is described in [1]. I just pushed patch which should fix this. It’s in [2]. [1] https://bugs.launchpad.net/neutron/+bug/1850557 [2] https://review.opendev.org/696794 > > -- > > Thanks, > > Matt > — Slawek Kaplonski Senior software engineer Red Hat From strigazi at gmail.com Sun Dec 1 13:49:58 2019 From: strigazi at gmail.com (Spyros Trigazis) Date: Sun, 1 Dec 2019 14:49:58 +0100 Subject: [Magnum] Virtual PTG planning In-Reply-To: <70f7d7bc-8862-241e-4a21-47e89e56c7e3@catalyst.net.nz> References: <2f35bc6c-b4bb-dbe7-c16d-ede34bc23914@catalyst.net.nz> <70f7d7bc-8862-241e-4a21-47e89e56c7e3@catalyst.net.nz> Message-ID: Hello, the etherpad is broken for me. I think an emoji did it. I have seen that in the past. Infra team resurrected it. Is it working for you? Cheers, Spyros On Mon, Nov 25, 2019 at 9:48 PM Feilong Wang wrote: > Hi team, > > After discussed with other team members, the virtual PTG is schedule on: > > 1st Session: 28th Nov 9:00AM-11:00AM UTC > > 2nd Session: 4th Dec 9:00AM-11:00AM UTC > > Please add your topics on > https://etherpad.openstack.org/p/magnum-ussuri-virtual-ptg-planning > Thanks. > > > On 19/11/19 10:46 AM, Feilong Wang wrote: > > Hi team, > > > > As we discussed on last weekly team meeting, we'd like to have a virtual > > PTG before the Xmas holiday to plan our work for the U release. The > > general idea is extending our current weekly meeting time from 1 hour to > > 2 hours and having 2 sessions with total 4 hours. My current proposal is > > as below, please reply if you have question or comments. Thanks. > > > > Pre discussion/Ideas collection: 20th Nov 9:00AM-10:00AM UTC > > > > 1st Session: 27th Nov 9:00AM-11:00AM UTC > > > > 2nd Session: 4th Dec 9:00AM-11:00AM UTC > > > > > -- > Cheers & Best regards, > Feilong Wang (王飞龙) > Head of R&D > Catalyst Cloud - Cloud Native New Zealand > -------------------------------------------------------------------------- > Tel: +64-48032246 > Email: flwang at catalyst.net.nz > Level 6, Catalyst House, 150 Willis Street, Wellington > -------------------------------------------------------------------------- > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From feilong at catalyst.net.nz Sun Dec 1 20:41:24 2019 From: feilong at catalyst.net.nz (Feilong Wang) Date: Mon, 2 Dec 2019 09:41:24 +1300 Subject: [Magnum] Virtual PTG planning In-Reply-To: References: <2f35bc6c-b4bb-dbe7-c16d-ede34bc23914@catalyst.net.nz> <70f7d7bc-8862-241e-4a21-47e89e56c7e3@catalyst.net.nz> Message-ID: I can't load it, the page is always in Loading status. On 2/12/19 2:49 AM, Spyros Trigazis wrote: > Hello, > > the etherpad is broken for me. I think an emoji did it. I have seen > that in the past. > Infra team resurrected it. > > Is it working for you?  > > Cheers, > Spyros > > On Mon, Nov 25, 2019 at 9:48 PM Feilong Wang > wrote: > > Hi team, > > After discussed with other team members, the virtual PTG is > schedule on: > > 1st Session:  28th Nov 9:00AM-11:00AM UTC > > 2nd Session: 4th Dec 9:00AM-11:00AM UTC > > Please add your topics on > https://etherpad.openstack.org/p/magnum-ussuri-virtual-ptg-planning > Thanks. > > > On 19/11/19 10:46 AM, Feilong Wang wrote: > > Hi team, > > > > As we discussed on last weekly team meeting, we'd like to have a > virtual > > PTG before the Xmas holiday to plan our work for the U release. The > > general idea is extending our current weekly meeting time from 1 > hour to > > 2 hours and having 2 sessions with total 4 hours. My current > proposal is > > as below, please reply if you have question or comments. Thanks. > > > > Pre discussion/Ideas collection:   20th Nov  9:00AM-10:00AM UTC > > > > 1st Session:  27th Nov 9:00AM-11:00AM UTC > > > > 2nd Session: 4th Dec 9:00AM-11:00AM UTC > > > > > -- > Cheers & Best regards, > Feilong Wang (王飞龙) > Head of R&D > Catalyst Cloud - Cloud Native New Zealand > -------------------------------------------------------------------------- > Tel: +64-48032246 > Email: flwang at catalyst.net.nz > Level 6, Catalyst House, 150 Willis Street, Wellington > -------------------------------------------------------------------------- > > > -- Cheers & Best regards, Feilong Wang (王飞龙) Head of R&D Catalyst Cloud - Cloud Native New Zealand -------------------------------------------------------------------------- Tel: +64-48032246 Email: flwang at catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington -------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From sshnaidm at redhat.com Sun Dec 1 20:43:44 2019 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Sun, 1 Dec 2019 22:43:44 +0200 Subject: [tripleo][ironic][ansible][openstack-ansible][ansible-sig] Ironic/Baremetal Ansible modules In-Reply-To: References: <826531574942720@sas8-ed615920eca2.qloud-c.yandex.net> <126374281574955397@iva5-2a2172cb7cff.qloud-c.yandex.net> Message-ID: The meeting will take place in IRC #openstack-ansible-sig at 15.00 UTC Tuesday 03 Dec. Please feel free to join. Thanks On Fri, Nov 29, 2019 at 5:02 PM Sagi Shnaidman wrote: > Let's choose an appropriate time in the etherpad [1], starting from Tue > next week to avoid a short notice. I'll send an update with exact time in > Sunday 01 Dec. > Thanks > > [1] https://etherpad.openstack.org/p/ironic-ansible-modules > > On Thu, Nov 28, 2019 at 5:48 PM Dmitriy Rabotyagov > wrote: > >> We have SIG meetings pretty ocasionally nowadays and it was a while since >> the last one. But we used to held them on Firdays at 2pm UTC in >> #openstack-ansible-sig IRC channel. >> >> >> 28.11.2019, 14:47, "Dmitry Tantsur" : >> >> Hi, >> >> On Thu, Nov 28, 2019 at 1:07 PM Dmitriy Rabotyagov >> wrote: >> >> Hi, >> >> I feel that this might be a good topic to discuss in terms of ansible SIG >> [1] and it can become the new home. >> So maybe we can plan a meeting for closer and prodictive discussion? >> >> >> I think it's a great idea. Do you have any formal meetings yet or should >> we maybe schedule a separate one? >> >> Dmitry >> >> >> >> [1] https://etherpad.openstack.org/p/ansible-sig >> >> >> 28.11.2019, 11:45, "Mark Goddard" : >> > On Wed, 27 Nov 2019 at 17:58, Sagi Shnaidman >> wrote: >> >> Hi, all >> >> >> >> in the light of finding the new home place for openstack related >> ansible modules [1] I'd like to discuss the best strategy to create Ironic >> ansible modules. Existing Ironic modules in Ansible repo don't cover even >> half of Ironic functionality, don't fit current needs and definitely >> require an additional work. There are a few topics that require attention >> and better be solved before modules are written to save additional work. We >> prepared an etherpad [2] with all these questions and if you have ideas or >> suggestions on how it should look you're welcome to update it. >> >> We'd like to decide the final place for them, name conventions (the >> most complex one!), what they should look like and how better to implement. >> >> Anybody interested in Ansible and baremetal management in Openstack, >> you're more than welcome to contribute. >> > >> > Thanks for raising this, we're definitely missing some key things for >> > ironic. I added a couple of roles and modules that we developed for >> > kayobe to the etherpad. Would be happy to contribute them to the >> > collection. >> > >> >> Thanks >> >> >> >> [1] https://review.opendev.org/#/c/684740/ >> >> [2] https://etherpad.openstack.org/p/ironic-ansible-modules >> >> >> >> -- >> >> Best regards >> >> Sagi Shnaidman >> >> -- >> Kind Regards, >> Dmitriy Rabotyagov >> >> >> >> >> >> -- >> Kind Regards, >> Dmitriy Rabotyagov >> >> > > > -- > Best regards > Sagi Shnaidman > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From feilong at catalyst.net.nz Sun Dec 1 20:43:43 2019 From: feilong at catalyst.net.nz (feilong) Date: Mon, 2 Dec 2019 09:43:43 +1300 Subject: [Magnum] Virtual PTG planning In-Reply-To: References: <2f35bc6c-b4bb-dbe7-c16d-ede34bc23914@catalyst.net.nz> <70f7d7bc-8862-241e-4a21-47e89e56c7e3@catalyst.net.nz> Message-ID: <684e7e05-83d4-3179-cc20-54daefea776c@catalyst.net.nz> But the good thing is I still have the tag opened on my laptop, so here you are: Magnum Ussuri Virtual PTG Two days: * Thursday        28 November 0900-1100 UTC * Wednesday     4 December 0900-1100 UTC Attendees:     flwang     brtknr     strigazi Topics: - Containerized master * - https://drive.google.com/open?id=10Qx-BQwv-JSXSSDTdFOQI_PnU2tL4JM9 * - How customisable is the master from the user point of view? Lingxian has done a PoC https://github.com/lingxiankong/magnum/tree/k8s-on-k8s just FYI, it does need more work, but just wanna give you guys an idea * - Catalyst Cloud care about this feature as a public cloud provider, how about CERN and StackHPC? * - It looks interesting, but at the moment we won't benefit very much from a brutal change. What about a new driver? See this llink https://github.com/lingxiankong/magnum/tree/k8s-on-k8s it will be a new driver. And user can choose which kind of cluster by using different driver/template * go for it * - Is there a demand for this feature from you as a Cloud Provider POV or is this mirroring GCP model? Not just mirroring, this feature can dramitically reduce the cluster cost since user won't have to pay the master node cost. Dos this allow a master to be shared between multiple clusters? No, there will be a root cluster managed by clouder provider, and user's cluster master node will be run on the root cluster as pods. so yes :) It is consolidation of resources. No, it's different. it's not a master being shared. still disagree but ok, it is a detail on describing Right, we don't have to go to far in this session. From our POV, there is a greater demand for master resize. With this way, master resizing will be easy. Do you have a diagram or just code? Download this https://drive.google.com/open?id=10Qx-BQwv-JSXSSDTdFOQI_PnU2tL4JM9. We are not opposed to this, its a cool idea, but would like to see it in action. Was there a live demo at the summit??? YesCool, I'll wait for the videos... * there are things missing, like the master IP. I (love? irony) that it is in ubuntu so fully incompatible with we have now. It's just a PoC, we can switch to FC30, it's not a problem I think. Lingxian did this with Ubuntu, just because he's more familiar with Ubuntu and Ansible, that doesn't mean we will go for this eventually. * - Catalyst Cloud will slowly and carefully do the upstream * I can not gurantee we will be onboard. That is why we have drivers. Totally understand. - Cinder CSI - remove in-tree Cinder       Similar work in kubespray https://github.com/kubernetes-sigs/kubespray/pull/5184/files       this is a small patch, no? Is anyone depending on cinder? let's take a patch. Catalyst Cloud is using Cinder, I think CERN and StackHPC are not, right?       Yes looks like it from the kubespray PR.        Caveat: For the moment, only Cinder v3 is supported by the CSI Driver. Not sure if this is a blocker? Is this the block-storage endpoint? For master branch, i don't think this is a blocker. Ok.So I will propose a patch for CSI support.  Yes. I am also happy to work on the patch next week. Sold, it's all yours. That was easy... +1 - All addons in a helm chart     - Traefik - this one should be easy. We need an ower for this.CERN can take this. Wonderful.     - What else?     - autoscaler     - OCCM     - keystone webhook     - kubernetes dashboard     - flannel     - calico     - Isn't kustomize the cool kid in the block these days? There are no community ready recipes for this.     As for the Helm, personally, I'd like to see a refactoring so that we can remove those shell scripts from https://github.com/openstack/magnum/tree/master/magnum/drivers/common/templates/kubernetes/helm Do you think we can drop those scripts by only have one and using a loop to install charts?????? It won't be a loop. It will one umbrella chart. I don't care TBH, but I don't want to see we're using Helm and still have to maintain such a lot bash scripts. Which makes me hard to see the value of switching to Helm, comparing the old way we're using with tons of bash scripts. One chart to rule them all. It would be fantastic, will CERN take this??????????????????????????????????????????????? ;) - Support for containerd?       https://review.opendev.org/#/c/695210/ CERN has this. I'm excited about this! Will allow us to run Kata containers. +1+100 I'm talking about Kata tonight in London, anyone fancy coming? https://www.meetup.com/OpenInfra-London/events/266230014/?rv=wm1&_xtd=gatlbWFpbF9jbGlja9oAJDExYzZhNjcwLTRjMWItNDc3OC05NTM2LWU2MDhiMTQxZWY2MQ&_af=event&_af_eid=266230014 Buy me a flight ticket from Wellington to London please - sure, my bank account detail, i can email you later. - Master resize       Another bonus maybe got by this is dropping the (discovery service dependency - what is this?) If we want to support master resizing, that probably means we will use token to bootstrap etcd cluster, and with that way, we can drop the dependency for the https://discovery.etcd.io       Isn't this irrelevant with the containerized master? Yes and No. Though with the containerized master, the master resizing could be easy. But we still want to allow user to resize a VM-based masters Catalyst needs this? We do have customers asked this. +1. I would say the public cloud customers are more picky than private cloud users. You guys can argue that ：）They are not. Try having customers that can ask for anything and they know it is free. LOL.       After discussing the better containrized magnum, we (CERN) will take a look into rebuiding the master and moving the master to the OPS tenant. Not sure how far will take the resizing.Into Ops tenant, but still being VM? yes. OK. Then what's the benefit? users dont' touch or see the vm, OPS have access to it and can do inrenventions. The change is small, "don't create the stack in user's tenant, but in OPS" OK, I can see the point from CERN's PoV. You don't benefit from this? Not much, because user still need to pay (I don't think you saw what i said)for the VM and it's making harder for billing. The master will be owned by the OPS project. How is the user being charged? But the master nodes will still occupy resource, no? The containerzed masters cluster is not occupying resources? It is, but far less than several c2r4 VMs. How come? if you need 4 cores in VMs you still need cores in the k8s cluster. Overcommit the vms if needed. But as pods, we can use small 'flavor' for master nodes and have more for HA. I don't see it, sorry, maybe it is your billing. From the resources perspective is almost the same.Yes, you raised a good point. I will calculate it again. That said, there is still value in containerized master. * - CLI*       At the moment, `openstack coe cluster list` AS AN ADMIN shows all clusters from all projects. When you try to do `openstack coe cluster config project_from_another_tenant`, you hit a 404. This does not seem right.??????????????????????? this 100% not an upstream bug. |      (os) [bkunwar at cloudadmin000 ~]$ openstack coe cluster config k8s-radarad-2 --dir ~/.kube/ --force| |Cluster 20b15ecd-9748-433a-b52c-09c2bbf7f603 could not be found (HTTP 404) (Request-ID: req-d9eaddf5-ef46-449a-b622-c6da7e26ecf3)| ah, ok makes sense, it is a feature. You can change the policy or add admin in all projects.Yes, by default, as admin, you shouldn't get the right to access any other's cluster(say customer's cluster) OK understood. - Replace heapster with metrics-server. Isn't this already done?       https://github.com/kubernetes-retired/heapster/blob/master/docs/deprecation.md Isn't this done? heapster is running but metrics server is used by the kubectl top command. Just drop heapster? Yep, we should drop heapster and probably install metrics-server by default. Thoughts? ok +2. I will take this. - Magum UI      Catalyst Cloud just done some big improvements for the Magnum UI and we would like to contribute them back. I will add you guys as reviewer for that. Just a heads up.      Cool! I have just figured out how to look at the magnum-ui plugin on dev Horizon so will be fun to put that to use. - Worker node upgrade by replacement * New upgrade in progress or reconciling status * Use resize API to drop nodes from one nodegroup to anpther (one-by-one / batch) * do this from inside the cluster with a daemon/pod running in the master so you can drain first. It would be great. We can extend the magnum-auto-healer as magnum-controller * maybe new controller and then merge if needed? Also works for me. * Would this work for upgrade from Atomic to CoreOS? * not initially, it could be possible. But when I started to support both Atomic and CoreOS in the same driver (we - who? I think me, you, feilong, it was IRC) changed our mind. * I am in two minds about this -  on one hand, clusters are disposable... they are not pets. Every elaborate service I know on k8s never uprgades, only replaces. many small clusters. This is the most stable pattern. You just need to figure out the proxy in front of the cluster. * Not tried this but if a new nodegroup has a different image specified and it uses fedora-coreos label instead of fedora-atomic, what would happen. It could work. Cool, this should be good enough. - Removing unused drivers and labels    - e.g. kube_version... what is this for? nothing but the values in cluster show    - Do we still need a CoreOS driver? Fedora Ironic Driver? Anyone using the Mesos and DCOS drivers?  we don't swarm_fedora_atomic_v1? v2 for us Does v2 still work? We can call for owner/maintainers for each inactive driver, if there is no maintainer for volunteer, then we can move them into "contrib", thoughts? Moving it to contrib would not lighten the code base. I was thinking of culling them. Cant remember the last time anyone asked a question about these drivers +1? I don't know. We can revisit this one later. Seems we don't have a quick conclustion. 👍     - ACTIONS * - *Containerized master* - Catalyst * - *Cinder CSI - remove in-tree Cinder* - StackHPC, do this with helm from the start ✅ * - *All addons in a helm chart* - CERN will look at this, StackHPC can help with... also bump up chart versions and check compatibility. * - *Master resize* - Catalyst, as part of the containerized solution? I will start with a spec, maybe separate * - *Worker node replacement** *- CERN * - *Magnum UI* - Catalyst * - *Containerd* - CERN * - *Drop Heapster* - Catalyst * * On 2/12/19 9:41 AM, Feilong Wang wrote: > > I can't load it, the page is always in Loading status. > > > On 2/12/19 2:49 AM, Spyros Trigazis wrote: >> Hello, >> >> the etherpad is broken for me. I think an emoji did it. I have seen >> that in the past. >> Infra team resurrected it. >> >> Is it working for you?  >> >> Cheers, >> Spyros >> >> On Mon, Nov 25, 2019 at 9:48 PM Feilong Wang > > wrote: >> >> Hi team, >> >> After discussed with other team members, the virtual PTG is >> schedule on: >> >> 1st Session:  28th Nov 9:00AM-11:00AM UTC >> >> 2nd Session: 4th Dec 9:00AM-11:00AM UTC >> >> Please add your topics on >> https://etherpad.openstack.org/p/magnum-ussuri-virtual-ptg-planning >> Thanks. >> >> >> On 19/11/19 10:46 AM, Feilong Wang wrote: >> > Hi team, >> > >> > As we discussed on last weekly team meeting, we'd like to have >> a virtual >> > PTG before the Xmas holiday to plan our work for the U release. The >> > general idea is extending our current weekly meeting time from >> 1 hour to >> > 2 hours and having 2 sessions with total 4 hours. My current >> proposal is >> > as below, please reply if you have question or comments. Thanks. >> > >> > Pre discussion/Ideas collection:   20th Nov  9:00AM-10:00AM UTC >> > >> > 1st Session:  27th Nov 9:00AM-11:00AM UTC >> > >> > 2nd Session: 4th Dec 9:00AM-11:00AM UTC >> > >> > >> -- >> Cheers & Best regards, >> Feilong Wang (王飞龙) >> Head of R&D >> Catalyst Cloud - Cloud Native New Zealand >> -------------------------------------------------------------------------- >> Tel: +64-48032246 >> Email: flwang at catalyst.net.nz >> Level 6, Catalyst House, 150 Willis Street, Wellington >> -------------------------------------------------------------------------- >> >> >> > -- > Cheers & Best regards, > Feilong Wang (王飞龙) > Head of R&D > Catalyst Cloud - Cloud Native New Zealand > -------------------------------------------------------------------------- > Tel: +64-48032246 > Email: flwang at catalyst.net.nz > Level 6, Catalyst House, 150 Willis Street, Wellington > -------------------------------------------------------------------------- -- Cheers & Best regards, Feilong Wang (王飞龙) ------------------------------------------------------ Senior Cloud Software Engineer Tel: +64-48032246 Email: flwang at catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington ------------------------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr at ham.ie Sun Dec 1 21:21:05 2019 From: gr at ham.ie (Graham Hayes) Date: Sun, 1 Dec 2019 13:21:05 -0800 Subject: [Usage][Designate] Salve Bind servers In-Reply-To: <1960112069.4062341.1574888348325@mail.yahoo.com> References: <1960112069.4062341.1574888348325.ref@mail.yahoo.com> <1960112069.4062341.1574888348325@mail.yahoo.com> Message-ID: <9cce3cc5-b7e2-98da-0ab2-6f2511d4ac5d@ham.ie> On 27/11/2019 12:59, Steven Relf wrote: > Hi list.... >   > I'm looking at making use of Designate, and have so far managed to get > designate configured and working against a single bind9 server. >   > The issue I'm having, and this could be due to not understanding bind > config well, is how do I configure other bind servers to be slaves of > the server used by designate. I keep running into an issue where the > slave doesn’t want to do a transfer and errors with "received notify for > zone 'domain': not authoritative" > I'm assuming this has something to do with the slave not knowing about > the zone. >   > So the question, do I have to have all my bind servers hooked upto > designate, and managed with a mdns service, or am I missing something > obvious. >   Yes, when using Designate and Bind, Designate (or in this case the designate-mdns server) is the "master" DNS server. As the bind nodes are configured to pull the data from Designate, they will not allow other servers to pull from them. To add this, you need to add extra "targets" in your pools.yaml file for the pool you are adding the servers to see [1] for an example. Thanks, Graham 1 - https://opendev.org/openstack/designate/src/branch/master/etc/designate/pools.yaml.sample-bind > Rgds > Steve. >   > The future has already arrived. It's just not evenly distributed yet - > William Gibson From emiller at genesishosting.com Sun Dec 1 22:39:13 2019 From: emiller at genesishosting.com (Eric K. Miller) Date: Sun, 1 Dec 2019 16:39:13 -0600 Subject: [neutron] DNS resolution delay after VM launch Message-ID: <046E9C0290DD9149B106B72FC9156BEA047714C9@gmsxchsvr01.thecreation.com> Hi, I'm trying to resolve an issue that happens about 30% of the time. DVR is being used in this environment. The environment is deployed with Kolla Ansible on CentOS 7 using Stein (the latest Kolla Ansible installer). Physical servers have the latest CentOS 7 patches through November 30th, 2019. This seems to happen on new and existing projects, including newly created subnets, and, so far, it seems to be only related to VM launches. Delaying the time between a subnet creation and the VM launch makes no difference in the behavior. When the problem occurs, after a VM is launched, it takes a longer-than-normal time to boot. This would usually indicate a DNS issue or possible a DHCP issue. After the OS boots (after 60 seconds or so), I can login, but with a delayed login (another 20 seconds or so) due to the DNS issue I'm diagnosing. DHCP works fine - no issues with IP assignment, and the /etc/resolv.conf entries are set to the routers' two private IPs. No DHCP errors in the dmesg output. I can connect to both respective qdhcp network namespaces where dnsmasq is running for the two distributed routers, immediately after the subnet is created (in our test case, it is 192.168.99.0/24), and can resolve names using "nslookup 192.168.99.1" and "nslookup 192.168.99.2" instantly. So, dnsmasq is working properly. A floating IP is assigned to this VM, which is used for the following SSH sessions. After SSH'ing to the VM, I can ping the 192.168.99.1 and 192.168.99.2 addresses, so it does not appear to be a network issue (ICMP works), unless there is a firewall rule either in iptables or OVS that is blocking UDP/TCP port 53 for DNS requests, which would be odd, since the egress security group assigned to this VM is unrestricted. I can also ping external IPs from the VM (Internet-accessible IPs), so the routers are working. I can also perform nslookups against external DNS servers without issue, such as "nslookup 1.1.1.1". If I direct nslookup to the internal DNS servers using "nslookup 192.168.99.1", the lookup times out. We normally use CentOS images, but I tried an Ubuntu 19.04 image and the same problem occurs often. Now, what is ODD - the problem goes away after about 3 minutes! DNS lookups against the internal DNS servers (the router IPs) start working perfectly. So something is taking a while to get configured, but only about 30% of the time. If, during the creation of the subnet, we specify --dns-nameserver options, such as: --dns-nameserver 1.1.1.1 the issue NEVER occurs. So it seems to be limited to DNS traffic between the VM and the dnsmasq service in the qdhcp namespace. All traffic in this environment is switched (we are not routing VXLAN traffic, for example), so there can be no physical network interference at Layers 3 or higher. The problem has also occurred on multiple hosts in the environment (the VM has been launched on various hosts with the same issue). Any ideas how to diagnose this? I'm assuming it has something to do with the iptables or OVS configuration, but if someone knows specifically what could cause this, such as a specific rule that is supposed to be created for DNS traffic to the internal routers, I could dive into the these configurations. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralonsoh at redhat.com Mon Dec 2 06:45:39 2019 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Mon, 02 Dec 2019 06:45:39 +0000 Subject: [neutron] Proposing Jakub Libosvar as Neutron core reviewer In-Reply-To: <16244BF0-3D00-4621-93CF-B32D3E3F5234@redhat.com> References: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> <16244BF0-3D00-4621-93CF-B32D3E3F5234@redhat.com> Message-ID: <5c92fc891352718e7d787fe203d9ec5ba6a03542.camel@redhat.com> Of course this proposal have my +1. I will say that, in the past, it was a pleasure to learn how Neutron works from him. On Fri, 2019-11-29 at 13:31 -0500, Nate Johnston wrote: > Wholehearted +1. Jakub is a great reference on many topics and he will be an essential bridge > helping OVN’s merge into Neutron be sustainable now and in the future. > > Nate > > > On Nov 28, 2019, at 4:03 AM, Slawek Kaplonski wrote: > > > > Hi neutrinos, > > > > We already started process of migrating networking-ovn driver to be one of in-tree neutron > > drivers. Blueprint for that is [1]. > > As part of this process I today proposed to include networking-ovn-core group into neutron-core > > group. Mail about it can be found at [2]. > > One of persons in networking-ovn-group is Jakub Libosvar who was Neutron core for very long time > > in the past. He knows very well not only ovn related code but also have great knowledge about > > all Neutron code base. > > So I would like to propose to Jakub as Neutron core reviewer again as he will be back working on > > neutron again now, after ovn will be in-tree driver. > > What do You think about it? > > I will wait for Your opinions for 1 week from now. Thx for all Your comments about it. > > > > [1] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge > > [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011240.html > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > > > From skaplons at redhat.com Mon Dec 2 08:58:11 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Mon, 2 Dec 2019 09:58:11 +0100 Subject: [neutron] Including networking-ovn-core team into neutron-core In-Reply-To: References: Message-ID: Hi, As I didn’t saw any points against this, I just included networking-ovn-core group into neutron-core group. Welcome networking-ovn cores in neutron cores team now :) > On 28 Nov 2019, at 09:26, Slawek Kaplonski wrote: > > Hi neutrinos, > > We recently merged spec [1] and now we are starting to moving networking-ovn code to be in neutron tree. > Blueprint to track progress of this is on [2]. > As a consequence of this merge of code, I think that we need to also include networking-ovn-core team into neutron-core to give people who are cores on networking-ovn today ability to merge ovn related patches in neutron after this migration will be done. > Of course current networking-ovn cores should only approve patches related to ovn driver, and not approve patches related to different areas of neutron code. > So if there will be no objections for that until the end of this week, I will include networking-ovn-core group in neutron-core. > > [1] https://review.opendev.org/#/c/658414/ > [2] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge > > — > Slawek Kaplonski > Senior software engineer > Red Hat > — Slawek Kaplonski Senior software engineer Red Hat From pierre at stackhpc.com Mon Dec 2 09:34:28 2019 From: pierre at stackhpc.com (Pierre Riteau) Date: Mon, 2 Dec 2019 10:34:28 +0100 Subject: [blazar] IRC meetings cancelled this week Message-ID: Hello, Due to travel, both Blazar IRC meetings are cancelled this week. Cheers, Pierre From lucasagomes at gmail.com Mon Dec 2 09:59:37 2019 From: lucasagomes at gmail.com (Lucas Alvares Gomes) Date: Mon, 2 Dec 2019 09:59:37 +0000 Subject: [neutron] Including networking-ovn-core team into neutron-core In-Reply-To: References: Message-ID: Thank you all very much! On Mon, Dec 2, 2019 at 9:08 AM Slawek Kaplonski wrote: > > Hi, > > As I didn’t saw any points against this, I just included networking-ovn-core group into neutron-core group. > Welcome networking-ovn cores in neutron cores team now :) > > > On 28 Nov 2019, at 09:26, Slawek Kaplonski wrote: > > > > Hi neutrinos, > > > > We recently merged spec [1] and now we are starting to moving networking-ovn code to be in neutron tree. > > Blueprint to track progress of this is on [2]. > > As a consequence of this merge of code, I think that we need to also include networking-ovn-core team into neutron-core to give people who are cores on networking-ovn today ability to merge ovn related patches in neutron after this migration will be done. > > Of course current networking-ovn cores should only approve patches related to ovn driver, and not approve patches related to different areas of neutron code. > > So if there will be no objections for that until the end of this week, I will include networking-ovn-core group in neutron-core. > > > > [1] https://review.opendev.org/#/c/658414/ > > [2] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From sean.mcginnis at gmx.com Mon Dec 2 10:05:57 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 2 Dec 2019 04:05:57 -0600 Subject: [all] Nominations for the "V" release name In-Reply-To: <20191115152903.GA29931@sm-workstation> References: <20191115152903.GA29931@sm-workstation> Message-ID: <20191202100557.GA363626@sm-workstation> Just a quick reminder for everyone. We are down to the last week to submit names to be included in the V release cycle naming. We already have a few great names on the list for the V release naming poll! If you have any other good names that start with V related to the British Columbia geographic region, please add it to the list. The actual polling to select the official name for the V release will start one week from today. Sean On Fri, Nov 15, 2019 at 09:29:03AM -0600, Sean McGinnis wrote: > Hey everyone, > > There is ongoing discussion about changing our release naming process, but for > the time being we are going to stick with what we have been doing. That means > it's time to start thinking about the "V" release name! > > The next developer event will take place in Vancouver, BC. The geographic > location for this release will be things starting with "V" in the British > Columbia province. > > The nomination period is now open. Please add suitable names to > https://wiki.openstack.org/wiki/Release_Naming/V_Proposals. We will accept > nominations until December 6, 2019 23:59:59 UTC. > > A recap of our current naming rules: > > * Each release name must start with the letter of the ISO basic Latin > alphabet following the initial letter of the previous release, starting > with the initial release of "Austin". After "Z", the next name should > start with "A" again. > > * The name must be composed only of the 26 characters of the ISO basic > Latin alphabet. Names which can be transliterated into this character > set are also acceptable. > > * The name must refer to the physical or human geography of the region > encompassing the location of the OpenStack design summit for the > corresponding release. The exact boundaries of the geographic region > under consideration must be declared before the opening of nominations, > as part of the initiation of the selection process. > > * The name must be a single word with a maximum of 10 characters. Words > that describe the feature should not be included, so "Foo City" or "Foo > Peak" would both be eligible as "Foo". > > Names which do not meet these criteria but otherwise sound really cool > should be added to a separate section of the wiki page and the TC may > make an exception for one or more of them to be considered in the > Condorcet poll. The naming official is responsible for presenting the > list of exceptional names for consideration to the TC before the poll opens. > > > Additional information about the release naming process can be found here: > > https://governance.openstack.org/tc/reference/release-naming.html > > Looking forward to having a name for our next release! > > Sean > From arnaud.morin at gmail.com Mon Dec 2 10:06:24 2019 From: arnaud.morin at gmail.com (Arnaud Morin) Date: Mon, 2 Dec 2019 10:06:24 +0000 Subject: [neutron][nova][large scale SIG] Rootwrap daemon and privsep In-Reply-To: <43282856-ccd3-fb30-01a2-47e6a1814a06@openstack.org> References: <20191129155335.GC27522@sync> <20191129155633.GD27522@sync> <43282856-ccd3-fb30-01a2-47e6a1814a06@openstack.org> Message-ID: <20191202100624.GE27522@sync> Hello, Thanks Thierry, it helps a lot, things look clearer now. Still, one point is still confused in my mind. Now that we are supposed to use privsep in most cases, services (nova, neutron and others) are supposed to start some sort of privsep daemon (something called privsep-helper if I believe what I see running on my infra). With what you said, I suspect that this privsep-helper is bootstrapped through a rootwrap execution. First question is: is the daemon mode of rootwrap still needed? Maybe not if the whole nova code is using the privsep mechanism now? Maybe it is if some part of the code has not yet moved on privsep? Moreover, if I take nova for exemple, I can see two privsep daemon running on my compute node: - one with context nova.privsep.sys_admin_pctxt - second with context vif_plug_linux_bridge.privsep.vif_plug Do you know why is it like that? Thanks, -- Arnaud Morin On 29.11.19 - 18:17, Thierry Carrez wrote: > Arnaud Morin wrote: > > [...] I'd like to understand the difference with privsep daemon. > > > > Is privsep a new daemon which is supposed to replace the rootwrap one? > > Is privsep being launch after rootwrap? > > IS privsep enabled by default, so I should not care about rootwrap at > > all? > > I can help with that, since I originally created rootwrap. > > Rootwrap is a privilege escalation control mechanism. It serves as a way to > filter what the service user on the machine can execute as root via sudo. > The idea is that sudoers files do not provide enough granularity, so instead > of trying to describe what is allowed and what is not in sudoers file, we > basically allow calling "sudo rootwrap command" and let rootwrap figure it > out. Rootwrap reads a number of (root-owned) configuration files and decides > to allow calling the command or not. > > There are two issues with this mechanism. The first is that the performance > is not great, as first you run a python executable (rootwrap), which in turn > spawns another process if the command is allowed. If you do that for > hundreds of "ip" calls as you set up networking in neutron, this can add up > pretty fast. > > The second issue is that rootwrap is only as secure as its configuration. If > for example you configure rootwrap to allow the 'nova' user to run the > "chmod" command, well that's basically the same as allowing it run anything > as root. You have to use advanced filters to further refine what it can > actually do based on command parameter analysis, and there is only so much > you can control that way. > > Rootwrap-daemon is a way to partially help with the first issue. Rather than > calling a new rootwrap Python process every time a command needs to be > called, you maintain a long-running rootwrap process that will process all > requests. It significantly improves performance, but it adds inter-process > communication complexity (never great in a security system). And it does > nothing to address the second issue. > > Privsep is the "right" way of addressing both issues. Rather than having the > code try to call shell commands as root, privsep allows the code to call > Python functions as root. This solves the performance issue, as you don't > have the overhead of a separate Python process + shell process every time > you want to change the ownership of a file, you can just call a Python > function that will call os.chown() and get ear to syscall efficiency. It > also solves the granularity issue, by allowing to call a function that will > only do what you want to do, rather than have to find a way to filter > parameters so that the command you call cannot be abused to do other things. > > The main issue with privsep is that it requires changing the code. You have > to set it up in every project (it's now done for most), but then every place > the service calls utils.execute(command, run_as_root=True) needs to be > changed to call a privileged Python function instead. > > The second issue with privsep is that it still needs root to start. The way > this is usually done is by using rootwrap itself to bootstrap privsep... > which can be confusing. There are obviously other ways to start the process > as root, but since most of those services still make use of rootwrap anyway, > that is what continues to be used for the initial bootstrapping. > > Ideally services would be completely transitioned to privsep, and we would > discontinue rootwrap. > > Hoping this helps, > > -- > Thierry Carrez (ttx) > From frode.nordahl at canonical.com Mon Dec 2 10:29:35 2019 From: frode.nordahl at canonical.com (Frode Nordahl) Date: Mon, 2 Dec 2019 11:29:35 +0100 Subject: [charms] IRC meetings and the holiday season Message-ID: Hello all, We are entering the holiday season, and as such we will stop our IRC meeting activity until we enter the next year. The next meeting will be held Monday January 13th 2020. I want to use this opportunity to say thank you to all activity and contributions to the charms project this year, and I look forward to what the next year might bring. Cheers! -- Frode Nordahl From ignaziocassano at gmail.com Mon Dec 2 10:33:11 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 11:33:11 +0100 Subject: [magnum][stein] error Message-ID: Hello, I've juste installed magnum on centos 7 stein, when magnu-api starts it does not log under /var/log/magnum but under /var/log/messages and it reports the following error: magnum-api: error: [Errno 32] Broken pipe Please , help me on this thread. Regards Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Mon Dec 2 11:19:06 2019 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 2 Dec 2019 12:19:06 +0100 Subject: [neutron][nova][large scale SIG] Rootwrap daemon and privsep In-Reply-To: <20191202100624.GE27522@sync> References: <20191129155335.GC27522@sync> <20191129155633.GD27522@sync> <43282856-ccd3-fb30-01a2-47e6a1814a06@openstack.org> <20191202100624.GE27522@sync> Message-ID: Arnaud Morin wrote: > [...] > Still, one point is still confused in my mind. > > Now that we are supposed to use privsep in most cases, services (nova, > neutron and others) are supposed to start some sort of privsep daemon > (something called privsep-helper if I believe what I see running on my > infra). > With what you said, I suspect that this privsep-helper is bootstrapped > through a rootwrap execution. Yes. It obviously depends on packaging and how they start services, but by default this is how privsep-helper is started. > First question is: is the daemon mode of rootwrap still needed? > Maybe not if the whole nova code is using the privsep mechanism now? > Maybe it is if some part of the code has not yet moved on privsep? I'll defer to nova experts, but yes, it's a trade-off that depends on how much has already been migrated, and how often the remaining rootwrap commands are called. Looking at nova compute node it only has a couple of rootwrap filters left[1], but maybe the performance loss of dropping daemon mode there is acceptable. [1] https://opendev.org/openstack/nova/src/branch/master/etc/nova/rootwrap.d/compute.filters > Moreover, if I take nova for exemple, I can see two privsep daemon > running on my compute node: > - one with context nova.privsep.sys_admin_pctxt > - second with context vif_plug_linux_bridge.privsep.vif_plug > > Do you know why is it like that? Privsep is finer-grained than rootwrap. Rather than just being root or non-root, you can require specific capabilities. See [2] for an example of such context definition. Privsep-helper starts as root and drops to the required capabilities to run with the minimum-needed privileges. That requires separate executables, one for each context. Specific libraries (like os_brick) or plugins can also come with their own privsep context. [2] https://opendev.org/openstack/nova/src/branch/master/nova/privsep/__init__.py Reading the initial spec[3] can give you extra context on how the change was driven. The current implementation has not strayed far away from that initial concept: [3] https://specs.openstack.org/openstack/oslo-specs/specs/liberty/privsep.html -- Thierry Carrez (ttx) From rosmaita.fossdev at gmail.com Mon Dec 2 13:18:12 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 2 Dec 2019 08:18:12 -0500 Subject: [cinder] Anastasiya accepted for Outreachy In-Reply-To: <16eadd0c11e.10a571b60328058.1259845006172962881@ghanshyammann.com> References: <20191126221030.GA107149@sm-workstation> <16eadd0c11e.10a571b60328058.1259845006172962881@ghanshyammann.com> Message-ID: <72b3d7b5-dfef-a931-c66e-8d0ac7feb5a5@gmail.com> On 11/27/19 12:02 PM, Ghanshyam Mann wrote: > ---- On Tue, 26 Nov 2019 16:10:30 -0600 Sean McGinnis wrote ---- > > On Tue, Nov 26, 2019 at 02:49:04PM -0500, Brian Rosmaita wrote: > > > On 11/26/19 12:45 PM, Sofia Enriquez wrote: > > > > Hi Cinder team, > > > > > > > > I'd like to announce that Anastasiya will be working with us improving > > > > the Tempest coverage this round. The internship schedule starts on Dec. > > > > 3, 2019, to March 3, 2020. Feel free to reach her on IRC /as anastzhyr/ > > > > if something comes up. > > > > > > Congratulations, Anastasiya! Improving tempest coverage is one of our > > > priorities for Ussuri, so I'm really glad you'll be working on this topic. > > > > > > Also, special thanks to you, Sofi, for acting as Anastasiya's mentor. > > > > > > > Totally agree! Welcome Anastasiya and thank you Sofi! > > Thanks. One question on the scope of Tempest test coverage. Is it for overall Tempest coverage for all projects or just Cinder Tempest tests? Our plan is to improve the tests in the cinder-tempest-plugin so that we can test some advanced scenarios more thoroughly. We'd certainly welcome reviews from the QA team to make sure that the code that lands meets tempest standards and could be incorporated easily into the main tempest suite if appropriate, but our plan is to selfishly have Anastasiya focus on Cinder during her internship. cheers, brian > > -gmann > > > > > Sean > > > > > From ignaziocassano at gmail.com Mon Dec 2 15:08:07 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 16:08:07 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate Message-ID: hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine Thanks Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Mon Dec 2 15:11:06 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Mon, 2 Dec 2019 15:11:06 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> Hi Ignazio, What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. Best bharat > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: > > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine > > Thanks > Ignazio From bharat at stackhpc.com Mon Dec 2 15:11:06 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Mon, 2 Dec 2019 15:11:06 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> Hi Ignazio, What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. Best bharat > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: > > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine > > Thanks > Ignazio From mthode at mthode.org Mon Dec 2 15:16:49 2019 From: mthode at mthode.org (Matthew Thode) Date: Mon, 2 Dec 2019 09:16:49 -0600 Subject: [requirements][os-ken][neutron] update of tinyrpc from 1.0.3 to 1.0.4 breaks neutron-grenade Message-ID: <20191202151649.zqqzq7vcwlfj5jtv@mthode.org> The only explicit depenency I can find is for os-ken +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ | Repository | Filename | Line | Text | +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ | openstack/os-ken | requirements.txt | 13 | tinyrpc>=0.6 # RPC library, BGP speaker(net_cntl) | | openstack/requirements | openstack_requirements/tests/files/upper-constraints.txt | 187 | tinyrpc===0.5 | | openstack/upstream-institute-virtual-environment | elements/upstream-training/static/tmp/requirements.txt | 292 | tinyrpc==1.0.3 | +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ Though it's included in constraints for a bunch of neutron packages though lower-constraints.txt +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ | Repository | Filename | Line | Text | +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ | openstack/dragonflow | lower-constraints.txt | 151 | tinyrpc==0.8 | | openstack/networking-bagpipe | lower-constraints.txt | 143 | tinyrpc==0.6 | | openstack/networking-baremetal | lower-constraints.txt | 139 | tinyrpc==0.8 | | openstack/networking-bgpvpn | lower-constraints.txt | 165 | tinyrpc==0.6 | | openstack/networking-generic-switch | lower-constraints.txt | 134 | tinyrpc==0.6 | | openstack/networking-hyperv | lower-constraints.txt | 131 | tinyrpc==0.6 | | openstack/networking-l2gw | lower-constraints.txt | 137 | tinyrpc==0.6 | | openstack/networking-odl | lower-constraints.txt | 192 | tinyrpc==0.8 | | openstack/networking-ovn | lower-constraints.txt | 145 | tinyrpc==0.6 | | openstack/networking-powervm | lower-constraints.txt | 138 | tinyrpc==0.6 | | openstack/networking-sfc | lower-constraints.txt | 146 | tinyrpc==0.8 | | openstack/neutron | lower-constraints.txt | 147 | tinyrpc==0.6 | | openstack/neutron-dynamic-routing | lower-constraints.txt | 141 | tinyrpc==0.6 | | openstack/neutron-fwaas | lower-constraints.txt | 141 | tinyrpc==0.6 | | openstack/neutron-vpnaas | lower-constraints.txt | 144 | tinyrpc==0.6 | | openstack/os-ken | lower-constraints.txt | 144 | tinyrpc==0.6 | | openstack/os-ken | requirements.txt | 13 | tinyrpc>=0.6 # RPC library, BGP speaker(net_cntl) | | openstack/requirements | openstack_requirements/tests/files/upper-constraints.txt | 187 | tinyrpc===0.5 | | openstack/requirements | upper-constraints.txt | 270 | tinyrpc===1.0.3 | | openstack/tricircle | lower-constraints.txt | 149 | tinyrpc==0.6 | | openstack/upstream-institute-virtual-environment | elements/upstream-training/static/tmp/requirements.txt | 292 | tinyrpc==1.0.3 | | x/flame | upper-constraints.txt | 208 | tinyrpc===0.9.3 | | x/networking-arista | lower-constraints.txt | 146 | tinyrpc==0.6 | | x/networking-fujitsu | lower-constraints.txt | 142 | tinyrpc==0.8 | | x/networking-omnipath | lower-constraints.txt | 144 | tinyrpc==0.6 | | x/networking-opencontrail | lower-constraints.txt | 149 | tinyrpc==0.6 | | x/networking-vsphere | lower-constraints.txt | 142 | tinyrpc==0.6 | | x/neutron-interconnection | lower-constraints.txt | 145 | tinyrpc==0.6 | | x/tap-as-a-service | lower-constraints.txt | 139 | tinyrpc==0.8 | +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ https://github.com/mbr/tinyrpc/compare/1.0.3...1.0.4 isn't too big but does seem to have a refactor in it. If you want to depend on a requirements patch for testing, I've created https://review.opendev.org/692915 for testing the update. Thanks, -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From rosmaita.fossdev at gmail.com Mon Dec 2 15:25:06 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Mon, 2 Dec 2019 10:25:06 -0500 Subject: [cinder][ops][extended-maintenance-sig][public-cloud-sig][enterprise-wg] Cinder to EOL some branches In-Reply-To: <7e8b4158-89b2-6207-6f06-215782e0b126@est.tech> References: <3c5fd6e6-8ae4-d300-71a7-97b22431cb3b@gmail.com> <7e8b4158-89b2-6207-6f06-215782e0b126@est.tech> Message-ID: <70f9cd7b-535d-8f6b-dbe2-3578bd21c346@gmail.com> On 11/27/19 12:02 PM, Elõd Illés wrote: > Hi, > > First of all, sorry for the late response. > > About EOLing Ocata and Pike: Extended Maintenance was formed just to > have a common place for interested parties, vendors, operators, to push > bugfixes to. Currently these branches are in a good shape, check / gate > jobs work and as far as I see there are a couple of backports, too (not > too many, though). So hopefully it's not a waste of efforts. Why don't > we keep them open as long as the CI works and there are patches? Of > course, whenever e.g. Ocata branch / CI becomes unmaintainable (zuul v3 > incompatibilities) or there aren't anyone who fixes the issues there, we > can put it to EOL then. Quick question: the driverfixes/mitaka and driverfixes/newton gates are broken now. Do you have any interest in getting these working, or are you only interested in stable/ocata and stable/pike? > I write this, because my employer supports Extended Maintenance, and I > usually try to fix CI issues on stable branches and reviewing patches > there. Thanks for following this strategy, it's good for the community, and we appreciate it. > So maybe I can be some help here, too. Please consider leaving > branches in Extended Maintenance open as long as they are in a good > shape and there are bugfixes coming. This is good feedback. As I said in the original email, we don't mind keeping these open as long as people are actually using them. I'll put a proposal on this week's Cinder meeting agenda to wait to EOL ocata and pike until their gates break and no one steps up to fix them, instead of proactively EOL'ing them now. Do let me know your position on the driverfixes branches right away, though. thanks, brian > > Thanks, > > Előd > > > On 2019. 11. 25. 20:21, Brian Rosmaita wrote: >> This is a courtesy notice that having received no responses to my >> email of 28 October [0] proposing to EOL some currently open Cinder >> branches, and following the policy articulated in [1], at today's >> Virtual PTG meeting the Cinder project team has decided to put the >> following stable branches into the End of Life state: >>   driverfixes/mitaka >>   driverfixes/newton >>   stable/ocata >>   stable/pike >> >> I will submit the paperwork to get this process moving one week from >> today (2 December 2019). >> >> >> cheers, >> brian >> >> [0] >> http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010385.html >> [1] >> https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-life >> > From ignaziocassano at gmail.com Mon Dec 2 15:26:30 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 16:26:30 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> Message-ID: The network plugin is flannel . I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? Thanks Ignazio Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: > Hi Ignazio, > > What version of kube_tag are you trying to run? What network plugin? I > suggest using `heat_container_agent_tag: train-stable` as its easier to > debug. > > > Best > > bharat > > > On 2 Dec 2019, at 15:08, Ignazio Cassano > wrote: > > > > hello, I've just installed stein with magnum on centos 7. I am creating > a kubernetes cluster but it is running since 22 minutes and I think it will > not work because the heat stack is waiting OSSoftwareDeployment completes > > I presume something is not working on heat with magnum. I ran several > stack with Software Deployment without magnum, and they worked fine > > > > Thanks > > Ignazio > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ildiko.vancsa at gmail.com Mon Dec 2 15:28:12 2019 From: ildiko.vancsa at gmail.com (Ildiko Vancsa) Date: Mon, 2 Dec 2019 10:28:12 -0500 Subject: Open Infrastructure Summit and PTG Shanghai - edge recap Message-ID: <753EAF6E-625F-40FA-9C8D-6C53B77B1304@gmail.com> Hi, I wrote up a quick summary about edge activities at the Open Infrastructure Summit and PTG in Shanghai to give a brief summary to people who were there and to catch up those who couldn’t attend this time. You can access the content here: http://lists.openstack.org/pipermail/edge-computing/2019-December/000664.html Please let me know if you have any questions or comments. Thanks, Ildikó (IRC: ildikov on Freenode) From bharat at stackhpc.com Mon Dec 2 15:29:05 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Mon, 2 Dec 2019 15:29:05 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> Message-ID: <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. Also make sure cluster_user_trust=True inside your magnum.conf. > On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: > > The network plugin is flannel . > I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? > I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? > Thanks > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar > ha scritto: > Hi Ignazio, > > What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. > > > Best > > bharat > > > On 2 Dec 2019, at 15:08, Ignazio Cassano > wrote: > > > > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes > > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine > > > > Thanks > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Mon Dec 2 15:29:05 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Mon, 2 Dec 2019 15:29:05 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> Message-ID: <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. Also make sure cluster_user_trust=True inside your magnum.conf. > On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: > > The network plugin is flannel . > I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? > I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? > Thanks > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar > ha scritto: > Hi Ignazio, > > What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. > > > Best > > bharat > > > On 2 Dec 2019, at 15:08, Ignazio Cassano > wrote: > > > > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes > > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine > > > > Thanks > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Dec 2 15:35:07 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 16:35:07 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Message-ID: Thanks, cluster_user_trust is right in my conf. Now I am going to play wth heat_container_agent labels. Many thanks. I'll send updates. Ignazio Il giorno lun 2 dic 2019 alle ore 16:29 Bharat Kunwar ha scritto: > Yes you can specify comma separated `labels` when you create a cluster > template/cluster from the dashboard. > > e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. > > Also make sure cluster_user_trust=True inside your magnum.conf. > > On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: > > The network plugin is flannel . > I do not kow what is kube_tag . I read that with it I can specify the > kubernetes version, right ? > I am using openstack dashboard for creating my cluster: where I can > specify heat_container_agent_tag: train-stable in the dashboard ? > Thanks > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar > ha scritto: > >> Hi Ignazio, >> >> What version of kube_tag are you trying to run? What network plugin? I >> suggest using `heat_container_agent_tag: train-stable` as its easier to >> debug. >> >> >> Best >> >> bharat >> >> > On 2 Dec 2019, at 15:08, Ignazio Cassano >> wrote: >> > >> > hello, I've just installed stein with magnum on centos 7. I am creating >> a kubernetes cluster but it is running since 22 minutes and I think it will >> not work because the heat stack is waiting OSSoftwareDeployment completes >> > I presume something is not working on heat with magnum. I ran several >> stack with Software Deployment without magnum, and they worked fine >> > >> > Thanks >> > Ignazio >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From knikolla at bu.edu Mon Dec 2 15:26:48 2019 From: knikolla at bu.edu (Nikolla, Kristi) Date: Mon, 2 Dec 2019 15:26:48 +0000 Subject: [mitaka][keystone] Authentication over keycloak server possible? In-Reply-To: <1713979160.118784.1575037172218@ox.dhbw-mannheim.de> References: <1713979160.118784.1575037172218@ox.dhbw-mannheim.de> Message-ID: <0D6D20BA-D766-4008-B490-68217797D6FB@bu.edu> Hi Michael, It is possible to use Keycloak as an identity provider and federate over SAML 2.0 or OpenID Connect. Please see this documentation for more details https://docs.openstack.org/keystone/latest/admin/federation/configure_federation.html#keystone-as-a-service-provider-sp There are a few improvements in later versions with regards to federation, so I would advise upgrading if possible. Most importantly, Mitaka has been End Of Life, and hence unsupported since 2017-04-10. But outside of that, you’re good to go. Best, Kristi From: Michael Stang Date: Friday, November 29, 2019 at 9:25 AM To: "openstack-discuss at lists.openstack.org" Subject: [mitaka][keystone] Authentication over keycloak server possible? Hi, we have an OpenStack Mitaka installation running (yes I know it's pretty old ;-) ) at our lab and would like to use the keycloak-server from the central IT for authentication. So I would like to know if it is already possible in mitaka to use this external keycloak server or if this only possible in a later OpenStack version? Maybe anyone know and if yes is there any documentation how to do it? Was searching for it but found not much about it by now... Thanks :) Kind regards Michael Michael Stang Laboringenieur, Dipl. Inf. (FH) Duale Hochschule Baden-Württemberg Mannheim Baden-Wuerttemberg Cooperative State University Mannheim ZeMath Zentrum für mathematisch-naturwissenschaftliches Basiswissen Fachbereich Informatik, Fakultät Technik Coblitzallee 1-9 68163 Mannheim michael.stang at dhbw-mannheim.de http://www.mannheim.dhbw.de [cid:7e0b1716-5d14-479c-b601-30379f07106a] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 28324 bytes Desc: image001.jpg URL: From ignaziocassano at gmail.com Mon Dec 2 15:49:02 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 16:49:02 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Message-ID: Hello, the stack went fine with paramenter you suggested....many many thanks. I have got further issues : 1) /var/log/magnum on controllers in emply. Logs are sent on /var/log/messages 2) in var log messages I have a lot of errors like the following: ec 2 16:47:15 tst2-osctrl01 magnum-api: Traceback (most recent call last): Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 568, in process_request Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.finish_request(request, client_address) Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.RequestHandlerClass(request, client_address, self) Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 651, in __init__ Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.finish() Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 710, in finish Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.wfile.close() Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/socket.py", line 279, in close Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.flush() Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/socket.py", line 303, in flush Dec 2 16:47:15 tst2-osctrl01 magnum-api: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 401, in sendall Dec 2 16:47:15 tst2-osctrl01 magnum-api: tail = self.send(data, flags) Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 395, in send Dec 2 16:47:15 tst2-osctrl01 magnum-api: return self._send_loop(self.fd.send, data, flags) Dec 2 16:47:15 tst2-osctrl01 magnum-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 382, in _send_loop Dec 2 16:47:15 tst2-osctrl01 magnum-api: return send_method(data, *args) Dec 2 16:47:15 tst2-osctrl01 magnum-api: error: [Errno 32] Broken pipe I deplpyed magnum on 3 controllers with a pacemaker cluster and haproxy. I can use dashboard and command line for magnum so I do not understand why I got the above errors. Regards Ignazio Il giorno lun 2 dic 2019 alle ore 16:29 Bharat Kunwar ha scritto: > Yes you can specify comma separated `labels` when you create a cluster > template/cluster from the dashboard. > > e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. > > Also make sure cluster_user_trust=True inside your magnum.conf. > > On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: > > The network plugin is flannel . > I do not kow what is kube_tag . I read that with it I can specify the > kubernetes version, right ? > I am using openstack dashboard for creating my cluster: where I can > specify heat_container_agent_tag: train-stable in the dashboard ? > Thanks > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar > ha scritto: > >> Hi Ignazio, >> >> What version of kube_tag are you trying to run? What network plugin? I >> suggest using `heat_container_agent_tag: train-stable` as its easier to >> debug. >> >> >> Best >> >> bharat >> >> > On 2 Dec 2019, at 15:08, Ignazio Cassano >> wrote: >> > >> > hello, I've just installed stein with magnum on centos 7. I am creating >> a kubernetes cluster but it is running since 22 minutes and I think it will >> not work because the heat stack is waiting OSSoftwareDeployment completes >> > I presume something is not working on heat with magnum. I ran several >> stack with Software Deployment without magnum, and they worked fine >> > >> > Thanks >> > Ignazio >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Mon Dec 2 15:49:26 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Mon, 2 Dec 2019 09:49:26 -0600 Subject: [neutron][nova][large scale SIG] Rootwrap daemon and privsep In-Reply-To: References: <20191129155335.GC27522@sync> <20191129155633.GD27522@sync> <43282856-ccd3-fb30-01a2-47e6a1814a06@openstack.org> <20191202100624.GE27522@sync> Message-ID: On 12/2/2019 5:19 AM, Thierry Carrez wrote: > I'll defer to nova experts, but yes, it's a trade-off that depends on > how much has already been migrated, and how often the remaining rootwrap > commands are called. Looking at nova compute node it only has a couple > of rootwrap filters left[1], but maybe the performance loss of dropping > daemon mode there is acceptable. > > [1] > https://opendev.org/openstack/nova/src/branch/master/etc/nova/rootwrap.d/compute.filters I want to say mikal converted everything native to nova from rootwrap to privsep and that was completed in Train: https://docs.openstack.org/releasenotes/nova/train.html#security-issues "The transition from rootwrap (or sudo) to privsep has been completed for nova. The only case where rootwrap is still used is to start privsep helpers. All other rootwrap configurations for nova may now be removed." Looking at what's in the compute.filters file looks like it's all stuff for os-brick, but I though os-brick was fully using privsep natively as well? Maybe it's just a matter of someone working on this TODO: https://opendev.org/openstack/nova/src/branch/master/etc/nova/rootwrap.d/compute.filters#L16 -- Thanks, Matt From pawel.konczalski at everyware.ch Mon Dec 2 15:53:09 2019 From: pawel.konczalski at everyware.ch (Pawel Konczalski) Date: Mon, 2 Dec 2019 16:53:09 +0100 Subject: Magnum: coe cluster can not be deleted (DELETE_FAILED) Message-ID: <157614a8-9f22-2728-4489-70a870dc9302@everyware.ch> Hello, i try to delete a broken magnum (v7.1.1.dev21) cluster but neither the cluster or the stack entries can be deleted. Any idea how to force / fix this? BR Pawel openstack coe cluster list | grep DELETE_FAILED openstack coe cluster delete ae5c347c-3ba8-4d39-9125-0bdadef253b4 openstack coe cluster show ae5c347c-3ba8-4d39-9125-0bdadef253b4 | status               | DELETE_FAILED ... |--- | status_reason        | Resource DELETE failed: JSONDecodeError: resources.kube_masters.resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0) ... | faults               | {'1': 'JSONDecodeError: resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0)', 'kube_masters': 'JSONDecodeError: resources.kube_masters.resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0)', '0': 'JSONDecodeError: resources[0].resources.api_pool_member: Expecting value: line 1 column 1 (char 0)', 'api_pool_member': 'JSONDecodeError: resources.api_pool_member: Expecting value: line 1 column 1 (char 0)', 'etcd_pool_member': 'JSONDecodeError: resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0)'} | openstack stack list | grep DELETE_FAILED openstack stack delete 2a907fc4-7b40-43f6-9254-d4084bd056b8 t Payload           DELETE: ResourceGroup "kube_masters" [85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec] Stack "slu-k8s-cluster2-6xiqrpfe7jd6" [2a907fc4-7b40-43f6-9254-d4084bd056b8] 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource Traceback (most recent call last): 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource   File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", line 924, in _action_recorder 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource     yield 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource   File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", line 2034, in delete 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource *action_args) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource   File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/scheduler.py", line 346, in wrapper 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource     step = next(subtask) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource   File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", line 986, in action_handler_task 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource     done = check(handler_data) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource   File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 596, in check_delete_complete 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource     return self._check_status_complete(self.DELETE) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource   File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 463, in _check_status_complete 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource action=action) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource ResourceFailure: JSONDecodeError: resources.kube_masters.resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource # openstack stack list --nested  | grep 7390a9b1d4be4d75b4bd08ab8107e4ff | 77848f9e-ff0c-4f7a-9fd7-8ed9e979b998 | test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd-1-tpqatosey33u | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED   | 2019-05-13T15:28:04Z | 2019-12-02T15:30:53Z | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED   | 2019-05-13T15:28:03Z | 2019-12-02T15:30:53Z | 2a907fc4-7b40-43f6-9254-d4084bd056b8 | | 4c91c53b-42f4-41ca-a97c-49d9261069d6 | test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd-0-hm2ucynq6hkc | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED   | 2019-05-13T15:28:03Z | 2019-12-02T15:30:54Z | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | | 2a907fc4-7b40-43f6-9254-d4084bd056b8 | test-k8s-cluster2-6xiqrpfe7jd6 | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED   | 2019-05-13T15:24:45Z | 2019-12-02T15:30:53Z | None                                 | -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5227 bytes Desc: not available URL: From michael.stang at dhbw-mannheim.de Mon Dec 2 15:57:41 2019 From: michael.stang at dhbw-mannheim.de (Michael Stang) Date: Mon, 2 Dec 2019 16:57:41 +0100 (CET) Subject: [mitaka][keystone] Authentication over keycloak server possible? In-Reply-To: <0D6D20BA-D766-4008-B490-68217797D6FB@bu.edu> References: <1713979160.118784.1575037172218@ox.dhbw-mannheim.de> <0D6D20BA-D766-4008-B490-68217797D6FB@bu.edu> Message-ID: <286895854.126790.1575302261864@ox.dhbw-mannheim.de> Hi Kristi, great, many thanks for the link, I will give this a try :-) Kind regrads, Michael "Nikolla, Kristi" hat am 2. Dezember 2019 um 16:26 geschrieben: > > Hi Michael, > > > > It is possible to use Keycloak as an identity provider and federate over SAML 2.0 or OpenID Connect. Please see this documentation for more details https://docs.openstack.org/keystone/latest/admin/federation/configure_federation.html#keystone-as-a-service-provider-sp https://docs.openstack.org/keystone/latest/admin/federation/configure_federation.html#keystone-as-a-service-provider-sp > > > > There are a few improvements in later versions with regards to federation, so I would advise upgrading if possible. Most importantly, Mitaka has been End Of Life, and hence unsupported since 2017-04-10. But outside of that, you’re good to go. > > > > Best, > > Kristi > > > > > > From: Michael Stang > Date: Friday, November 29, 2019 at 9:25 AM > To: "openstack-discuss at lists.openstack.org" > Subject: [mitaka][keystone] Authentication over keycloak server possible? > > > > Hi, > > we have an OpenStack Mitaka installation running (yes I know it's pretty old ;-) ) at our lab and would like to use the keycloak-server from the central IT for authentication. > > So I would like to know if it is already possible in mitaka to use this external keycloak server or if this only possible in a later OpenStack version? Maybe anyone know and if yes is there any documentation how to do it? Was searching for it but found not much about it by now... > > Thanks :) > > Kind regards > > Michael > > > > Michael Stang > Laboringenieur, Dipl. Inf. (FH) > > Duale Hochschule Baden-Württemberg Mannheim > Baden-Wuerttemberg Cooperative State University Mannheim > ZeMath Zentrum für mathematisch-naturwissenschaftliches Basiswissen > Fachbereich Informatik, Fakultät Technik > Coblitzallee 1-9 > 68163 Mannheim > > michael.stang at dhbw-mannheim.de > mailto:michael.stang at dhbw-mannheim.de > http://www.mannheim.dhbw.de http://www.dhbw-mannheim.de/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From victoria at vmartinezdelacruz.com Mon Dec 2 17:10:04 2019 From: victoria at vmartinezdelacruz.com (=?UTF-8?Q?Victoria_Mart=C3=ADnez_de_la_Cruz?=) Date: Mon, 2 Dec 2019 14:10:04 -0300 Subject: [stein][manila-ui] error In-Reply-To: References: Message-ID: Hi Ignazio, Are you deploying everything manually? Which is the version of Manila you are using? Have you followed the docs we have for installation or any specific documentation you can point us to? Also, feel free to submit a bug report in https://bugs.launchpad.net/manila-ui. Thanks, Victoria On Fri, Nov 29, 2019 at 11:30 AM Ignazio Cassano wrote: > Hello, > furthter information: in the dashboard admin secion if I push share it > does not list shares . > I the project section it gives the error I sent previously > > Il giorno ven 29 nov 2019 alle ore 15:03 Ignazio Cassano < > ignaziocassano at gmail.com> ha scritto: > >> Hi Carlos, I am installing via yum command >> Thanks >> Ignazio >> >> Il giorno ven 29 nov 2019 alle ore 14:46 Carlos Silva < >> ces.eduardo98 at gmail.com> ha scritto: >> >>> Hi, Ignazio! >>> >>> Could you please provide more information about how have you installed >>> it? It was via package, git, devstack? >>> I'm trying to reproduce the issue in my environment but I'm not able to. >>> >>> Regards, >>> Carlos Silva. >>> >>> Em sex., 29 de nov. de 2019 às 05:42, Ignazio Cassano < >>> ignaziocassano at gmail.com> escreveu: >>> >>>> Hello, >>>> I just installed openstack stein on centos. >>>> Manila works fine my command line but when I click "share" in the >>>> dashboard the following error appears: >>>> >>>> Environment: >>>> >>>> >>>> Request Method: GET >>>> Request URL: http://10.102.184.190/dashboard/project/shares/ >>>> >>>> Django Version: 1.11.20 >>>> Python Version: 2.7.5 >>>> Installed Applications: >>>> ['openstack_dashboard.dashboards.project', >>>> 'neutron_lbaas_dashboard', >>>> 'heat_dashboard', >>>> 'openstack_dashboard.dashboards.admin', >>>> 'openstack_dashboard.dashboards.identity', >>>> 'openstack_dashboard.dashboards.settings', >>>> 'dashboards', >>>> 'openstack_dashboard', >>>> 'django.contrib.contenttypes', >>>> 'django.contrib.auth', >>>> 'django.contrib.sessions', >>>> 'django.contrib.messages', >>>> 'django.contrib.staticfiles', >>>> 'django.contrib.humanize', >>>> 'django_pyscss', >>>> 'debreach', >>>> 'openstack_dashboard.django_pyscss_fix', >>>> 'compressor', >>>> 'horizon', >>>> 'openstack_auth'] >>>> Installed Middleware: >>>> ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', >>>> 'debreach.middleware.RandomCommentMiddleware', >>>> 'django.middleware.common.CommonMiddleware', >>>> 'django.middleware.csrf.CsrfViewMiddleware', >>>> 'django.contrib.sessions.middleware.SessionMiddleware', >>>> 'django.contrib.auth.middleware.AuthenticationMiddleware', >>>> 'horizon.middleware.OperationLogMiddleware', >>>> 'django.contrib.messages.middleware.MessageMiddleware', >>>> 'horizon.middleware.HorizonMiddleware', >>>> 'horizon.themes.ThemeMiddleware', >>>> 'django.middleware.locale.LocaleMiddleware', >>>> 'django.middleware.clickjacking.XFrameOptionsMiddleware', >>>> >>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', >>>> >>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') >>>> >>>> >>>> >>>> Traceback: >>>> >>>> File >>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in >>>> inner >>>> 41. response = get_response(request) >>>> >>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in >>>> _get_response >>>> 187. response = >>>> self.process_exception_by_middleware(e, request) >>>> >>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in >>>> _get_response >>>> 185. response = wrapped_callback(request, >>>> *callback_args, **callback_kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>> 36. return view_func(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>> 52. return view_func(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>> 36. return view_func(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>> 113. return view_func(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>> 84. return view_func(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in >>>> view >>>> 68. return self.dispatch(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in >>>> dispatch >>>> 88. return handler(request, *args, **kwargs) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get >>>> 223. handled = self.construct_tables() >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>> construct_tables >>>> 214. handled = self.handle_table(table) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>> handle_table >>>> 123. data = self._get_data_dict() >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>> _get_data_dict >>>> 43. data.extend(func()) >>>> >>>> File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in >>>> wrapped >>>> 109. value = cache[key] = func(*args, >>>> **kwargs) >>>> >>>> File >>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" >>>> in get_shares_data >>>> 57. share_nets = manila.share_network_list(self.request) >>>> >>>> File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in >>>> share_network_list >>>> 280. return >>>> manilaclient(request).share_networks.list(detailed=detailed, >>>> >>>> Exception Type: AttributeError at /project/shares/ >>>> Exception Value: 'NoneType' object has no attribute 'share_networks' >>>> >>>> >>>> Anyone can help, please ? >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sshnaidm at redhat.com Mon Dec 2 17:13:04 2019 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Mon, 2 Dec 2019 19:13:04 +0200 Subject: "Transible" - convert your current cloud state to Ansible playbooks Message-ID: Hi, all I'd like to present you a new tool I wrote recently, although it's under heavy development, but it can be already used and can be helpful in some of your DevOps/CloudOps tasks. I'd appreciate any feedback on it and any contributions of course :) Transible [1] (TRansformANSIBLE) - converts existing cloud state to ansible playbooks. It's a tool that takes your current cloud resources and represents them as Ansible playbooks. Running these playbooks you *ideally* will get no "changed" tasks and the same cloud configuration. Currently, it works with Openstack servers, security rules, security groups, images, volumes, servers, routers, networks, subnets, users, domains and projects which your cloud includes and defines their config as tasks in Ansible playbooks that are ready for deployment. By running these playbooks you actually can deploy or redeploy your current cloud. What are the possible use cases? 1. Freezing cloud config for further IaaC management If you have a ton of deployed resources and lost your tracking, better to start to manage them according to Infrastructure as Code principles. Creating a configuration of everything you have now allows you to change, delete, add things now via code. 2. Keeping current cloud configuration Sometimes it's required to have a "snapshot" of mandatory resources on the cloud and keep them the same no matter what happens in the cloud, like heavy testing for example. Running these playbooks will ensure you have all you need in the cloud in the required configuration. 3. Moving current infrastructure to another tenant, cloud, provider, whatever If you move to another provider but want to keep all you need and as it's configured right now, you can create playbooks with Transible and use them in the new cloud or provider. 4. In the future, I plan to add translation from one cloud configuration to another and it may help with releasing from vendor lock. Of course, because of various clouds' differences, it'll be limited. Can you come up with more? More details about the tool and how it can be configured for various use cases you can read in my longer articles on Medium [2] or Dev.to [3]. I'd appreciate your feedback here or in issues of Transible project. [4] Thanks [1] https://github.com/sshnaidm/transible [2] https://medium.com/@einarum/https-medium-com-einarum-transible-represents-cloud-configuration-as-ansible-playbooks-a75b067105a7 [3] https://dev.to/sshnaidm/transible-represents-cloud-configuration-as-ansible-playbooks-42dm [4] https://github.com/sshnaidm/transible/issues -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Dec 2 17:44:52 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 18:44:52 +0100 Subject: [stein][manila-ui] error In-Reply-To: References: Message-ID: Hi Victoria, I installed everything manually following documentation for centos stein at docs.openstack.org. The following are the version I am using: python2-manilaclient-1.27.0-1.el7.noarch python2-manila-8.1.0-1.el7.noarch openstack-manila-share-8.1.0-1.el7.noarch openstack-manila-8.1.0-1.el7.noarch openstack-manila-ui-2.18.1-1.el7.noarch I am using netapp share driver: share_driver = manila.share.drivers.netapp.common.NetAppDriver with: driver_handles_share_servers = False Creating manila shares and changing access permissions work fine using command line client. Thanks and regards Ignazio Il giorno lun 2 dic 2019 alle ore 18:10 Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> ha scritto: > Hi Ignazio, > > Are you deploying everything manually? Which is the version of Manila you > are using? Have you followed the docs we have for installation or any > specific documentation you can point us to? > > Also, feel free to submit a bug report in > https://bugs.launchpad.net/manila-ui. > > Thanks, > > Victoria > > On Fri, Nov 29, 2019 at 11:30 AM Ignazio Cassano > wrote: > >> Hello, >> furthter information: in the dashboard admin secion if I push share it >> does not list shares . >> I the project section it gives the error I sent previously >> >> Il giorno ven 29 nov 2019 alle ore 15:03 Ignazio Cassano < >> ignaziocassano at gmail.com> ha scritto: >> >>> Hi Carlos, I am installing via yum command >>> Thanks >>> Ignazio >>> >>> Il giorno ven 29 nov 2019 alle ore 14:46 Carlos Silva < >>> ces.eduardo98 at gmail.com> ha scritto: >>> >>>> Hi, Ignazio! >>>> >>>> Could you please provide more information about how have you installed >>>> it? It was via package, git, devstack? >>>> I'm trying to reproduce the issue in my environment but I'm not able to. >>>> >>>> Regards, >>>> Carlos Silva. >>>> >>>> Em sex., 29 de nov. de 2019 às 05:42, Ignazio Cassano < >>>> ignaziocassano at gmail.com> escreveu: >>>> >>>>> Hello, >>>>> I just installed openstack stein on centos. >>>>> Manila works fine my command line but when I click "share" in the >>>>> dashboard the following error appears: >>>>> >>>>> Environment: >>>>> >>>>> >>>>> Request Method: GET >>>>> Request URL: http://10.102.184.190/dashboard/project/shares/ >>>>> >>>>> Django Version: 1.11.20 >>>>> Python Version: 2.7.5 >>>>> Installed Applications: >>>>> ['openstack_dashboard.dashboards.project', >>>>> 'neutron_lbaas_dashboard', >>>>> 'heat_dashboard', >>>>> 'openstack_dashboard.dashboards.admin', >>>>> 'openstack_dashboard.dashboards.identity', >>>>> 'openstack_dashboard.dashboards.settings', >>>>> 'dashboards', >>>>> 'openstack_dashboard', >>>>> 'django.contrib.contenttypes', >>>>> 'django.contrib.auth', >>>>> 'django.contrib.sessions', >>>>> 'django.contrib.messages', >>>>> 'django.contrib.staticfiles', >>>>> 'django.contrib.humanize', >>>>> 'django_pyscss', >>>>> 'debreach', >>>>> 'openstack_dashboard.django_pyscss_fix', >>>>> 'compressor', >>>>> 'horizon', >>>>> 'openstack_auth'] >>>>> Installed Middleware: >>>>> ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', >>>>> 'debreach.middleware.RandomCommentMiddleware', >>>>> 'django.middleware.common.CommonMiddleware', >>>>> 'django.middleware.csrf.CsrfViewMiddleware', >>>>> 'django.contrib.sessions.middleware.SessionMiddleware', >>>>> 'django.contrib.auth.middleware.AuthenticationMiddleware', >>>>> 'horizon.middleware.OperationLogMiddleware', >>>>> 'django.contrib.messages.middleware.MessageMiddleware', >>>>> 'horizon.middleware.HorizonMiddleware', >>>>> 'horizon.themes.ThemeMiddleware', >>>>> 'django.middleware.locale.LocaleMiddleware', >>>>> 'django.middleware.clickjacking.XFrameOptionsMiddleware', >>>>> >>>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', >>>>> >>>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') >>>>> >>>>> >>>>> >>>>> Traceback: >>>>> >>>>> File >>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in >>>>> inner >>>>> 41. response = get_response(request) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" >>>>> in _get_response >>>>> 187. response = >>>>> self.process_exception_by_middleware(e, request) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" >>>>> in _get_response >>>>> 185. response = wrapped_callback(request, >>>>> *callback_args, **callback_kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 36. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 52. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 36. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 113. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 84. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" >>>>> in view >>>>> 68. return self.dispatch(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" >>>>> in dispatch >>>>> 88. return handler(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get >>>>> 223. handled = self.construct_tables() >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>> construct_tables >>>>> 214. handled = self.handle_table(table) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>> handle_table >>>>> 123. data = self._get_data_dict() >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>> _get_data_dict >>>>> 43. data.extend(func()) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in >>>>> wrapped >>>>> 109. value = cache[key] = func(*args, >>>>> **kwargs) >>>>> >>>>> File >>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" >>>>> in get_shares_data >>>>> 57. share_nets = manila.share_network_list(self.request) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in >>>>> share_network_list >>>>> 280. return >>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>> >>>>> Exception Type: AttributeError at /project/shares/ >>>>> Exception Value: 'NoneType' object has no attribute 'share_networks' >>>>> >>>>> >>>>> Anyone can help, please ? >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmann at ghanshyammann.com Mon Dec 2 17:47:10 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 02 Dec 2019 11:47:10 -0600 Subject: [cinder] Anastasiya accepted for Outreachy In-Reply-To: <72b3d7b5-dfef-a931-c66e-8d0ac7feb5a5@gmail.com> References: <20191126221030.GA107149@sm-workstation> <16eadd0c11e.10a571b60328058.1259845006172962881@ghanshyammann.com> <72b3d7b5-dfef-a931-c66e-8d0ac7feb5a5@gmail.com> Message-ID: <16ec7b928ba.eadc4141462491.5515700181394378936@ghanshyammann.com> ---- On Mon, 02 Dec 2019 07:18:12 -0600 Brian Rosmaita wrote ---- > On 11/27/19 12:02 PM, Ghanshyam Mann wrote: > > ---- On Tue, 26 Nov 2019 16:10:30 -0600 Sean McGinnis wrote ---- > > > On Tue, Nov 26, 2019 at 02:49:04PM -0500, Brian Rosmaita wrote: > > > > On 11/26/19 12:45 PM, Sofia Enriquez wrote: > > > > > Hi Cinder team, > > > > > > > > > > I'd like to announce that Anastasiya will be working with us improving > > > > > the Tempest coverage this round. The internship schedule starts on Dec. > > > > > 3, 2019, to March 3, 2020. Feel free to reach her on IRC /as anastzhyr/ > > > > > if something comes up. > > > > > > > > Congratulations, Anastasiya! Improving tempest coverage is one of our > > > > priorities for Ussuri, so I'm really glad you'll be working on this topic. > > > > > > > > Also, special thanks to you, Sofi, for acting as Anastasiya's mentor. > > > > > > > > > > Totally agree! Welcome Anastasiya and thank you Sofi! > > > > Thanks. One question on the scope of Tempest test coverage. Is it for overall Tempest coverage for all projects or just Cinder Tempest tests? > > Our plan is to improve the tests in the cinder-tempest-plugin so that we > can test some advanced scenarios more thoroughly. We'd certainly > welcome reviews from the QA team to make sure that the code that lands > meets tempest standards and could be incorporated easily into the main > tempest suite if appropriate, but our plan is to selfishly have > Anastasiya focus on Cinder during her internship. Sure, do let me know if you want any help with this. -gmann > > cheers, > brian > > > > > -gmann > > > > > > > > Sean > > > > > > > > > > > From ignaziocassano at gmail.com Mon Dec 2 17:58:20 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 18:58:20 +0100 Subject: [stein][manila-ui] error In-Reply-To: References: Message-ID: Hi again Victoria, The bugs seems te following: 1701381 But it seems for ocata release Ignazio Il giorno lun 2 dic 2019 alle ore 18:10 Victoria Martínez de la Cruz < victoria at vmartinezdelacruz.com> ha scritto: > Hi Ignazio, > > Are you deploying everything manually? Which is the version of Manila you > are using? Have you followed the docs we have for installation or any > specific documentation you can point us to? > > Also, feel free to submit a bug report in > https://bugs.launchpad.net/manila-ui. > > Thanks, > > Victoria > > On Fri, Nov 29, 2019 at 11:30 AM Ignazio Cassano > wrote: > >> Hello, >> furthter information: in the dashboard admin secion if I push share it >> does not list shares . >> I the project section it gives the error I sent previously >> >> Il giorno ven 29 nov 2019 alle ore 15:03 Ignazio Cassano < >> ignaziocassano at gmail.com> ha scritto: >> >>> Hi Carlos, I am installing via yum command >>> Thanks >>> Ignazio >>> >>> Il giorno ven 29 nov 2019 alle ore 14:46 Carlos Silva < >>> ces.eduardo98 at gmail.com> ha scritto: >>> >>>> Hi, Ignazio! >>>> >>>> Could you please provide more information about how have you installed >>>> it? It was via package, git, devstack? >>>> I'm trying to reproduce the issue in my environment but I'm not able to. >>>> >>>> Regards, >>>> Carlos Silva. >>>> >>>> Em sex., 29 de nov. de 2019 às 05:42, Ignazio Cassano < >>>> ignaziocassano at gmail.com> escreveu: >>>> >>>>> Hello, >>>>> I just installed openstack stein on centos. >>>>> Manila works fine my command line but when I click "share" in the >>>>> dashboard the following error appears: >>>>> >>>>> Environment: >>>>> >>>>> >>>>> Request Method: GET >>>>> Request URL: http://10.102.184.190/dashboard/project/shares/ >>>>> >>>>> Django Version: 1.11.20 >>>>> Python Version: 2.7.5 >>>>> Installed Applications: >>>>> ['openstack_dashboard.dashboards.project', >>>>> 'neutron_lbaas_dashboard', >>>>> 'heat_dashboard', >>>>> 'openstack_dashboard.dashboards.admin', >>>>> 'openstack_dashboard.dashboards.identity', >>>>> 'openstack_dashboard.dashboards.settings', >>>>> 'dashboards', >>>>> 'openstack_dashboard', >>>>> 'django.contrib.contenttypes', >>>>> 'django.contrib.auth', >>>>> 'django.contrib.sessions', >>>>> 'django.contrib.messages', >>>>> 'django.contrib.staticfiles', >>>>> 'django.contrib.humanize', >>>>> 'django_pyscss', >>>>> 'debreach', >>>>> 'openstack_dashboard.django_pyscss_fix', >>>>> 'compressor', >>>>> 'horizon', >>>>> 'openstack_auth'] >>>>> Installed Middleware: >>>>> ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', >>>>> 'debreach.middleware.RandomCommentMiddleware', >>>>> 'django.middleware.common.CommonMiddleware', >>>>> 'django.middleware.csrf.CsrfViewMiddleware', >>>>> 'django.contrib.sessions.middleware.SessionMiddleware', >>>>> 'django.contrib.auth.middleware.AuthenticationMiddleware', >>>>> 'horizon.middleware.OperationLogMiddleware', >>>>> 'django.contrib.messages.middleware.MessageMiddleware', >>>>> 'horizon.middleware.HorizonMiddleware', >>>>> 'horizon.themes.ThemeMiddleware', >>>>> 'django.middleware.locale.LocaleMiddleware', >>>>> 'django.middleware.clickjacking.XFrameOptionsMiddleware', >>>>> >>>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', >>>>> >>>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') >>>>> >>>>> >>>>> >>>>> Traceback: >>>>> >>>>> File >>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in >>>>> inner >>>>> 41. response = get_response(request) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" >>>>> in _get_response >>>>> 187. response = >>>>> self.process_exception_by_middleware(e, request) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" >>>>> in _get_response >>>>> 185. response = wrapped_callback(request, >>>>> *callback_args, **callback_kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 36. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 52. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 36. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 113. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>> 84. return view_func(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" >>>>> in view >>>>> 68. return self.dispatch(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" >>>>> in dispatch >>>>> 88. return handler(request, *args, **kwargs) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get >>>>> 223. handled = self.construct_tables() >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>> construct_tables >>>>> 214. handled = self.handle_table(table) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>> handle_table >>>>> 123. data = self._get_data_dict() >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>> _get_data_dict >>>>> 43. data.extend(func()) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in >>>>> wrapped >>>>> 109. value = cache[key] = func(*args, >>>>> **kwargs) >>>>> >>>>> File >>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" >>>>> in get_shares_data >>>>> 57. share_nets = manila.share_network_list(self.request) >>>>> >>>>> File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in >>>>> share_network_list >>>>> 280. return >>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>> >>>>> Exception Type: AttributeError at /project/shares/ >>>>> Exception Value: 'NoneType' object has no attribute 'share_networks' >>>>> >>>>> >>>>> Anyone can help, please ? >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Dec 2 18:02:28 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 19:02:28 +0100 Subject: [stein][manila-ui] error In-Reply-To: References: Message-ID: I am sorry Victoria, I sent you a bug related to magnum ....it is another issues I am facing. Ignazio Il giorno lun 2 dic 2019 alle ore 18:58 Ignazio Cassano < ignaziocassano at gmail.com> ha scritto: > Hi again Victoria, > The bugs seems te following: > 1701381 > But it seems for ocata release > > Ignazio > > Il giorno lun 2 dic 2019 alle ore 18:10 Victoria Martínez de la Cruz < > victoria at vmartinezdelacruz.com> ha scritto: > >> Hi Ignazio, >> >> Are you deploying everything manually? Which is the version of Manila you >> are using? Have you followed the docs we have for installation or any >> specific documentation you can point us to? >> >> Also, feel free to submit a bug report in >> https://bugs.launchpad.net/manila-ui. >> >> Thanks, >> >> Victoria >> >> On Fri, Nov 29, 2019 at 11:30 AM Ignazio Cassano < >> ignaziocassano at gmail.com> wrote: >> >>> Hello, >>> furthter information: in the dashboard admin secion if I push share it >>> does not list shares . >>> I the project section it gives the error I sent previously >>> >>> Il giorno ven 29 nov 2019 alle ore 15:03 Ignazio Cassano < >>> ignaziocassano at gmail.com> ha scritto: >>> >>>> Hi Carlos, I am installing via yum command >>>> Thanks >>>> Ignazio >>>> >>>> Il giorno ven 29 nov 2019 alle ore 14:46 Carlos Silva < >>>> ces.eduardo98 at gmail.com> ha scritto: >>>> >>>>> Hi, Ignazio! >>>>> >>>>> Could you please provide more information about how have you installed >>>>> it? It was via package, git, devstack? >>>>> I'm trying to reproduce the issue in my environment but I'm not able >>>>> to. >>>>> >>>>> Regards, >>>>> Carlos Silva. >>>>> >>>>> Em sex., 29 de nov. de 2019 às 05:42, Ignazio Cassano < >>>>> ignaziocassano at gmail.com> escreveu: >>>>> >>>>>> Hello, >>>>>> I just installed openstack stein on centos. >>>>>> Manila works fine my command line but when I click "share" in the >>>>>> dashboard the following error appears: >>>>>> >>>>>> Environment: >>>>>> >>>>>> >>>>>> Request Method: GET >>>>>> Request URL: http://10.102.184.190/dashboard/project/shares/ >>>>>> >>>>>> Django Version: 1.11.20 >>>>>> Python Version: 2.7.5 >>>>>> Installed Applications: >>>>>> ['openstack_dashboard.dashboards.project', >>>>>> 'neutron_lbaas_dashboard', >>>>>> 'heat_dashboard', >>>>>> 'openstack_dashboard.dashboards.admin', >>>>>> 'openstack_dashboard.dashboards.identity', >>>>>> 'openstack_dashboard.dashboards.settings', >>>>>> 'dashboards', >>>>>> 'openstack_dashboard', >>>>>> 'django.contrib.contenttypes', >>>>>> 'django.contrib.auth', >>>>>> 'django.contrib.sessions', >>>>>> 'django.contrib.messages', >>>>>> 'django.contrib.staticfiles', >>>>>> 'django.contrib.humanize', >>>>>> 'django_pyscss', >>>>>> 'debreach', >>>>>> 'openstack_dashboard.django_pyscss_fix', >>>>>> 'compressor', >>>>>> 'horizon', >>>>>> 'openstack_auth'] >>>>>> Installed Middleware: >>>>>> ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', >>>>>> 'debreach.middleware.RandomCommentMiddleware', >>>>>> 'django.middleware.common.CommonMiddleware', >>>>>> 'django.middleware.csrf.CsrfViewMiddleware', >>>>>> 'django.contrib.sessions.middleware.SessionMiddleware', >>>>>> 'django.contrib.auth.middleware.AuthenticationMiddleware', >>>>>> 'horizon.middleware.OperationLogMiddleware', >>>>>> 'django.contrib.messages.middleware.MessageMiddleware', >>>>>> 'horizon.middleware.HorizonMiddleware', >>>>>> 'horizon.themes.ThemeMiddleware', >>>>>> 'django.middleware.locale.LocaleMiddleware', >>>>>> 'django.middleware.clickjacking.XFrameOptionsMiddleware', >>>>>> >>>>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', >>>>>> >>>>>> 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') >>>>>> >>>>>> >>>>>> >>>>>> Traceback: >>>>>> >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in >>>>>> inner >>>>>> 41. response = get_response(request) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" >>>>>> in _get_response >>>>>> 187. response = >>>>>> self.process_exception_by_middleware(e, request) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" >>>>>> in _get_response >>>>>> 185. response = wrapped_callback(request, >>>>>> *callback_args, **callback_kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>>> 36. return view_func(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>>> 52. return view_func(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>>> 36. return view_func(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>>> 113. return view_func(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec >>>>>> 84. return view_func(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" >>>>>> in view >>>>>> 68. return self.dispatch(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" >>>>>> in dispatch >>>>>> 88. return handler(request, *args, **kwargs) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get >>>>>> 223. handled = self.construct_tables() >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>>> construct_tables >>>>>> 214. handled = self.handle_table(table) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>>> handle_table >>>>>> 123. data = self._get_data_dict() >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in >>>>>> _get_data_dict >>>>>> 43. data.extend(func()) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in >>>>>> wrapped >>>>>> 109. value = cache[key] = func(*args, >>>>>> **kwargs) >>>>>> >>>>>> File >>>>>> "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" >>>>>> in get_shares_data >>>>>> 57. share_nets = manila.share_network_list(self.request) >>>>>> >>>>>> File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in >>>>>> share_network_list >>>>>> 280. return >>>>>> manilaclient(request).share_networks.list(detailed=detailed, >>>>>> >>>>>> Exception Type: AttributeError at /project/shares/ >>>>>> Exception Value: 'NoneType' object has no attribute 'share_networks' >>>>>> >>>>>> >>>>>> Anyone can help, please ? >>>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Dec 2 18:05:16 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 2 Dec 2019 19:05:16 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Message-ID: Hello again, It seems I am facing the bug 1701381 (socket broken pipe) but it is related to ocata :-( Ignazio Il giorno lun 2 dic 2019 alle ore 16:49 Ignazio Cassano < ignaziocassano at gmail.com> ha scritto: > Hello, the stack went fine with paramenter you suggested....many many > thanks. > > I have got further issues : > > 1) /var/log/magnum on controllers in emply. > > Logs are sent on /var/log/messages > > 2) in var log messages I have a lot of errors like the following: > > ec 2 16:47:15 tst2-osctrl01 magnum-api: Traceback (most recent call last): > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib64/python2.7/SocketServer.py", line 568, in process_request > Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.finish_request(request, > client_address) > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request > Dec 2 16:47:15 tst2-osctrl01 magnum-api: > self.RequestHandlerClass(request, client_address, self) > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib64/python2.7/SocketServer.py", line 651, in __init__ > Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.finish() > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib64/python2.7/SocketServer.py", line 710, in finish > Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.wfile.close() > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib64/python2.7/socket.py", line 279, in close > Dec 2 16:47:15 tst2-osctrl01 magnum-api: self.flush() > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib64/python2.7/socket.py", line 303, in flush > Dec 2 16:47:15 tst2-osctrl01 magnum-api: > self._sock.sendall(view[write_offset:write_offset+buffer_size]) > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 401, in > sendall > Dec 2 16:47:15 tst2-osctrl01 magnum-api: tail = self.send(data, flags) > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 395, in > send > Dec 2 16:47:15 tst2-osctrl01 magnum-api: return > self._send_loop(self.fd.send, data, flags) > Dec 2 16:47:15 tst2-osctrl01 magnum-api: File > "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 382, in > _send_loop > Dec 2 16:47:15 tst2-osctrl01 magnum-api: return send_method(data, *args) > Dec 2 16:47:15 tst2-osctrl01 magnum-api: error: [Errno 32] Broken pipe > > I deplpyed magnum on 3 controllers with a pacemaker cluster and haproxy. > I can use dashboard and command line for magnum so I do not understand why > I got the above errors. > > > Regards > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:29 Bharat Kunwar > ha scritto: > >> Yes you can specify comma separated `labels` when you create a cluster >> template/cluster from the dashboard. >> >> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >> >> Also make sure cluster_user_trust=True inside your magnum.conf. >> >> On 2 Dec 2019, at 15:26, Ignazio Cassano >> wrote: >> >> The network plugin is flannel . >> I do not kow what is kube_tag . I read that with it I can specify the >> kubernetes version, right ? >> I am using openstack dashboard for creating my cluster: where I can >> specify heat_container_agent_tag: train-stable in the dashboard ? >> Thanks >> Ignazio >> >> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar < >> bharat at stackhpc.com> ha scritto: >> >>> Hi Ignazio, >>> >>> What version of kube_tag are you trying to run? What network plugin? I >>> suggest using `heat_container_agent_tag: train-stable` as its easier to >>> debug. >>> >>> >>> Best >>> >>> bharat >>> >>> > On 2 Dec 2019, at 15:08, Ignazio Cassano >>> wrote: >>> > >>> > hello, I've just installed stein with magnum on centos 7. I am >>> creating a kubernetes cluster but it is running since 22 minutes and I >>> think it will not work because the heat stack is waiting >>> OSSoftwareDeployment completes >>> > I presume something is not working on heat with magnum. I ran several >>> stack with Software Deployment without magnum, and they worked fine >>> > >>> > Thanks >>> > Ignazio >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From haleyb.dev at gmail.com Mon Dec 2 18:49:42 2019 From: haleyb.dev at gmail.com (Brian Haley) Date: Mon, 2 Dec 2019 13:49:42 -0500 Subject: [neutron] Proposing Jakub Libosvar as Neutron core reviewer In-Reply-To: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> References: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> Message-ID: +1 from me, good to have Kuba back! On 11/28/19 3:59 AM, Slawek Kaplonski wrote: > Hi neutrinos, > > We already started process of migrating networking-ovn driver to be one of in-tree neutron drivers. Blueprint for that is [1]. > As part of this process I today proposed to include networking-ovn-core group into neutron-core group. Mail about it can be found at [2]. > One of persons in networking-ovn-group is Jakub Libosvar who was Neutron core for very long time in the past. He knows very well not only ovn related code but also have great knowledge about all Neutron code base. > So I would like to propose to Jakub as Neutron core reviewer again as he will be back working on neutron again now, after ovn will be in-tree driver. > What do You think about it? > I will wait for Your opinions for 1 week from now. Thx for all Your comments about it. > > [1] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge > [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011240.html > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From kennelson11 at gmail.com Mon Dec 2 21:29:38 2019 From: kennelson11 at gmail.com (Kendall Nelson) Date: Mon, 2 Dec 2019 13:29:38 -0800 Subject: [all] Shanghai PTG Feedback Message-ID: Hello Everyone! Now that we've had a few weeks to catch up after the summit and marinate on everything we accomplished together there, we wanted to take a sec to see if anyone had feedback. Normally we would have held a lunchtime feedback session, but due to venue restrictions, we didn't have a good place to do things like normal so we created the etherpad[1] and are distributing it this way :) Hope to hear everyone's feedback! Thanks! -the Kendalls (diablo_rojo & wendallkaters) [1]https://etherpad.openstack.org/p/PVG-PTG-feedback -------------- next part -------------- An HTML attachment was scrubbed... URL: From bitskrieg at bitskrieg.net Mon Dec 2 21:44:32 2019 From: bitskrieg at bitskrieg.net (Chris Apsey) Date: Mon, 02 Dec 2019 21:44:32 +0000 Subject: Magnum: coe cluster can not be deleted (DELETE_FAILED) In-Reply-To: <157614a8-9f22-2728-4489-70a870dc9302@everyware.ch> References: <157614a8-9f22-2728-4489-70a870dc9302@everyware.ch> Message-ID: Pawel, I'd identify the individual resources created by magnum and delete them using their native process (e.g. openstack server delete). Once I was certain all of the resources were actually gone, I'd drop into the magnum and heat databases and remove the references to them (assuming that manually cleaning up the aforementioned provisioned resources doesn't let you delete the cluster/stack in the normal manner). Chris Sent from ProtonMail mobile -------- Original Message -------- On Dec 2, 2019, 10:53, Pawel Konczalski wrote: > Hello, > > i try to delete a broken magnum (v7.1.1.dev21) cluster but neither the > cluster or the stack entries can be deleted. Any idea how to force / fix > this? > > BR > > Pawel > > openstack coe cluster list | grep DELETE_FAILED > > openstack coe cluster delete ae5c347c-3ba8-4d[39-9125-0](tel:3991250)bdadef253b4 > > openstack coe cluster show ae5c347c-3ba8-4d[39-9125-0](tel:3991250)bdadef253b4 > | status | DELETE_FAILED > ... |--- > | status_reason | Resource DELETE failed: JSONDecodeError: > resources.kube_masters.resources[1].resources.etcd_pool_member: > Expecting value: line 1 column 1 (char 0) > ... > | faults | {'1': 'JSONDecodeError: > resources[1].resources.etcd_pool_member: Expecting value: line 1 column > 1 (char 0)', 'kube_masters': 'JSONDecodeError: > resources.kube_masters.resources[1].resources.etcd_pool_member: > Expecting value: line 1 column 1 (char 0)', '0': 'JSONDecodeError: > resources[0].resources.api_pool_member: Expecting value: line 1 column 1 > (char 0)', 'api_pool_member': 'JSONDecodeError: > resources.api_pool_member: Expecting value: line 1 column 1 (char 0)', > 'etcd_pool_member': 'JSONDecodeError: resources.etcd_pool_member: > Expecting value: line 1 column 1 (char 0)'} | > > openstack stack list | grep DELETE_FAILED > > openstack stack delete 2a907fc4-7b40-43f6-9254-d4084bd056b8 > > t Payload DELETE: ResourceGroup "kube_masters" > [85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec] Stack > "slu-k8s-cluster2-6xiqrpfe7jd6" [2a907fc4-7b40-43f6-9254-d4084bd056b8] > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource Traceback (most > recent call last): > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource File > "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", > line 924, in _action_recorder > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource yield > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource File > "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", > line 2034, in delete > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource *action_args) > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource File > "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/scheduler.py", > line 346, in wrapper > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource step = > next(subtask) > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource File > "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", > line 986, in action_handler_task > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource done = > check(handler_data) > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource File > "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", > line 596, in check_delete_complete > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource return > self._check_status_complete(self.DELETE) > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource File > "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", > line 463, in _check_status_complete > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource action=action) > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource ResourceFailure: > JSONDecodeError: > resources.kube_masters.resources[1].resources.etcd_pool_member: > Expecting value: line 1 column 1 (char 0) > 2019-12-02 12:43:[08.887 22](tel:0888722) ERROR heat.engine.resource > > # openstack stack list --nested | grep 7390a9b1d4be4d75b4bd08ab8107e4ff > | 77848f9e-ff0c-4f7a-9fd7-8ed9e979b998 | > test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd-1-tpqatosey33u > | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | > 2019-05-13T15:28:04Z | 2019-12-02T15:30:53Z | > 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | > | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | > test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd | > 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | > 2019-05-13T15:28:03Z | 2019-12-02T15:30:53Z | > 2a907fc4-7b40-43f6-9254-d4084bd056b8 | > | 4c91c53b-42f4-41ca-a97c-49d[9261069](tel:9261069)d6 | > test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd-0-hm2ucynq6hkc > | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | > 2019-05-13T15:28:03Z | 2019-12-02T15:30:54Z | > 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | > | 2a907fc4-7b40-43f6-9254-d4084bd056b8 | test-k8s-cluster2-6xiqrpfe7jd6 > | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | > 2019-05-13T15:24:45Z | 2019-12-02T15:30:53Z | > None | -------------- next part -------------- An HTML attachment was scrubbed... URL: From naohiro.sameshima at global.ntt Tue Dec 3 03:02:16 2019 From: naohiro.sameshima at global.ntt (=?utf-8?B?TmFvaGlybyBTYW1lc2hpbWHvvIjprqvls7Yg55u05rSL77yJKEdyb3VwKQ==?=) Date: Tue, 3 Dec 2019 03:02:16 +0000 Subject: [glance] [glance_store] requirements-check job fails Message-ID: <63CC5661-FB21-4969-9A24-9C0231625BF2@global.ntt> Hi, I'm Naohiro. I proposed a blueprint [0] (revive s3 driver) in Shanghai PTG and then create a patch [1]. So, I submit a patch in gerrit code review, but unfortunately one of zuul job (requirements-check) failed [2]. I think it seems that doc/requirements.txt is wrong. (Not included in my patch) Error log is below. ~~~~~~~~~~~~~~~~~~~~ Validating doc/requirements.txt WARNING: possible mismatch found for package "sphinx" Attribute "markers" does not match "" does not match "python_version>='3.4'" Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,>=1.6.2', markers='', comment='# BSD', extras=frozenset()) Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,!=2.1.0', markers="python_version>='3.4'", comment='# BSD', extras=frozenset()) WARNING: possible mismatch found for package "sphinx" Attribute "markers" does not match "" does not match "python_version=='2.7'" Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,>=1.6.2', markers='', comment='# BSD', extras=frozenset()) Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,<2.0.0', markers="python_version=='2.7'", comment='# BSD', extras=frozenset()) ERROR: Could not find a global requirements entry to match package sphinx. If the package is already included in the global list, the name or platform markers there may not match the local settings. ~~~~~~~~~~~~~~~~~~~~ Or is above job failing because of my patch? Thank you. Naohiro Sameshima [0]: https://review.opendev.org/#/c/687390/ [1]: https://review.opendev.org/#/c/695844/ [2]: https://zuul.opendev.org/t/openstack/build/4aef8abff3374613ab413aca0ab65381 This email and all contents are subject to the following disclaimer: https://hello.global.ntt/en-us/email-disclaimer -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Tue Dec 3 08:02:40 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 3 Dec 2019 09:02:40 +0100 Subject: [requirements][os-ken][neutron] update of tinyrpc from 1.0.3 to 1.0.4 breaks neutron-grenade In-Reply-To: <20191202151649.zqqzq7vcwlfj5jtv@mthode.org> References: <20191202151649.zqqzq7vcwlfj5jtv@mthode.org> Message-ID: <061AE3DF-5FC4-4E07-BD16-34C1D7F5A203@redhat.com> Hi, I opened story board for this [1] but it seems for me that this isn’t os-ken problem. It seems that version 1.0.4 of tinyrpc is not compatible with python 2.7 and this is failing because of [2]. Neutron-grenade job is old py27 job which I already proposed to remove. See [3] for details. So IMO we should simply wait a bit more, remove this old py27 job and than we should be good with new tinyrpc version. [1] https://storyboard.openstack.org/#!/story/2006975 [2] https://github.com/mbr/tinyrpc/commit/e67c368dd65e03a055d93f38d313d97e69092920 [3] https://review.opendev.org/#/q/owner:%22Slawek+Kaplonski%22+branch:master+topic:drop-py27-support > On 2 Dec 2019, at 16:16, Matthew Thode wrote: > > The only explicit depenency I can find is for os-ken > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > | Repository | Filename | Line | Text | > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > | openstack/os-ken | requirements.txt | 13 | tinyrpc>=0.6 # RPC library, BGP speaker(net_cntl) | > | openstack/requirements | openstack_requirements/tests/files/upper-constraints.txt | 187 | tinyrpc===0.5 | > | openstack/upstream-institute-virtual-environment | elements/upstream-training/static/tmp/requirements.txt | 292 | tinyrpc==1.0.3 | > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > Though it's included in constraints for a bunch of neutron packages though lower-constraints.txt > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > | Repository | Filename | Line | Text | > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > | openstack/dragonflow | lower-constraints.txt | 151 | tinyrpc==0.8 | > | openstack/networking-bagpipe | lower-constraints.txt | 143 | tinyrpc==0.6 | > | openstack/networking-baremetal | lower-constraints.txt | 139 | tinyrpc==0.8 | > | openstack/networking-bgpvpn | lower-constraints.txt | 165 | tinyrpc==0.6 | > | openstack/networking-generic-switch | lower-constraints.txt | 134 | tinyrpc==0.6 | > | openstack/networking-hyperv | lower-constraints.txt | 131 | tinyrpc==0.6 | > | openstack/networking-l2gw | lower-constraints.txt | 137 | tinyrpc==0.6 | > | openstack/networking-odl | lower-constraints.txt | 192 | tinyrpc==0.8 | > | openstack/networking-ovn | lower-constraints.txt | 145 | tinyrpc==0.6 | > | openstack/networking-powervm | lower-constraints.txt | 138 | tinyrpc==0.6 | > | openstack/networking-sfc | lower-constraints.txt | 146 | tinyrpc==0.8 | > | openstack/neutron | lower-constraints.txt | 147 | tinyrpc==0.6 | > | openstack/neutron-dynamic-routing | lower-constraints.txt | 141 | tinyrpc==0.6 | > | openstack/neutron-fwaas | lower-constraints.txt | 141 | tinyrpc==0.6 | > | openstack/neutron-vpnaas | lower-constraints.txt | 144 | tinyrpc==0.6 | > | openstack/os-ken | lower-constraints.txt | 144 | tinyrpc==0.6 | > | openstack/os-ken | requirements.txt | 13 | tinyrpc>=0.6 # RPC library, BGP speaker(net_cntl) | > | openstack/requirements | openstack_requirements/tests/files/upper-constraints.txt | 187 | tinyrpc===0.5 | > | openstack/requirements | upper-constraints.txt | 270 | tinyrpc===1.0.3 | > | openstack/tricircle | lower-constraints.txt | 149 | tinyrpc==0.6 | > | openstack/upstream-institute-virtual-environment | elements/upstream-training/static/tmp/requirements.txt | 292 | tinyrpc==1.0.3 | > | x/flame | upper-constraints.txt | 208 | tinyrpc===0.9.3 | > | x/networking-arista | lower-constraints.txt | 146 | tinyrpc==0.6 | > | x/networking-fujitsu | lower-constraints.txt | 142 | tinyrpc==0.8 | > | x/networking-omnipath | lower-constraints.txt | 144 | tinyrpc==0.6 | > | x/networking-opencontrail | lower-constraints.txt | 149 | tinyrpc==0.6 | > | x/networking-vsphere | lower-constraints.txt | 142 | tinyrpc==0.6 | > | x/neutron-interconnection | lower-constraints.txt | 145 | tinyrpc==0.6 | > | x/tap-as-a-service | lower-constraints.txt | 139 | tinyrpc==0.8 | > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > https://github.com/mbr/tinyrpc/compare/1.0.3...1.0.4 isn't too big but does seem to have a refactor in it. > > If you want to depend on a requirements patch for testing, I've created https://review.opendev.org/692915 > for testing the update. > > Thanks, > > -- > Matthew Thode — Slawek Kaplonski Senior software engineer Red Hat From mthode at mthode.org Tue Dec 3 08:06:25 2019 From: mthode at mthode.org (Matthew Thode) Date: Tue, 3 Dec 2019 02:06:25 -0600 Subject: [requirements][os-ken][neutron] update of tinyrpc from 1.0.3 to 1.0.4 breaks neutron-grenade In-Reply-To: <061AE3DF-5FC4-4E07-BD16-34C1D7F5A203@redhat.com> References: <20191202151649.zqqzq7vcwlfj5jtv@mthode.org> <061AE3DF-5FC4-4E07-BD16-34C1D7F5A203@redhat.com> Message-ID: <20191203080625.of3frhoimaodnjf7@mthode.org> On 19-12-03 09:02:40, Slawek Kaplonski wrote: > Hi, > > I opened story board for this [1] but it seems for me that this isn’t os-ken problem. It seems that version 1.0.4 of tinyrpc is not compatible with python 2.7 and this is failing because of [2]. > Neutron-grenade job is old py27 job which I already proposed to remove. See [3] for details. So IMO we should simply wait a bit more, remove this old py27 job and than we should be good with new tinyrpc version. > > [1] https://storyboard.openstack.org/#!/story/2006975 > [2] https://github.com/mbr/tinyrpc/commit/e67c368dd65e03a055d93f38d313d97e69092920 > [3] https://review.opendev.org/#/q/owner:%22Slawek+Kaplonski%22+branch:master+topic:drop-py27-support > > > On 2 Dec 2019, at 16:16, Matthew Thode wrote: > > > > The only explicit depenency I can find is for os-ken > > > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > | Repository | Filename | Line | Text | > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > | openstack/os-ken | requirements.txt | 13 | tinyrpc>=0.6 # RPC library, BGP speaker(net_cntl) | > > | openstack/requirements | openstack_requirements/tests/files/upper-constraints.txt | 187 | tinyrpc===0.5 | > > | openstack/upstream-institute-virtual-environment | elements/upstream-training/static/tmp/requirements.txt | 292 | tinyrpc==1.0.3 | > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > > > Though it's included in constraints for a bunch of neutron packages though lower-constraints.txt > > > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > | Repository | Filename | Line | Text | > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > | openstack/dragonflow | lower-constraints.txt | 151 | tinyrpc==0.8 | > > | openstack/networking-bagpipe | lower-constraints.txt | 143 | tinyrpc==0.6 | > > | openstack/networking-baremetal | lower-constraints.txt | 139 | tinyrpc==0.8 | > > | openstack/networking-bgpvpn | lower-constraints.txt | 165 | tinyrpc==0.6 | > > | openstack/networking-generic-switch | lower-constraints.txt | 134 | tinyrpc==0.6 | > > | openstack/networking-hyperv | lower-constraints.txt | 131 | tinyrpc==0.6 | > > | openstack/networking-l2gw | lower-constraints.txt | 137 | tinyrpc==0.6 | > > | openstack/networking-odl | lower-constraints.txt | 192 | tinyrpc==0.8 | > > | openstack/networking-ovn | lower-constraints.txt | 145 | tinyrpc==0.6 | > > | openstack/networking-powervm | lower-constraints.txt | 138 | tinyrpc==0.6 | > > | openstack/networking-sfc | lower-constraints.txt | 146 | tinyrpc==0.8 | > > | openstack/neutron | lower-constraints.txt | 147 | tinyrpc==0.6 | > > | openstack/neutron-dynamic-routing | lower-constraints.txt | 141 | tinyrpc==0.6 | > > | openstack/neutron-fwaas | lower-constraints.txt | 141 | tinyrpc==0.6 | > > | openstack/neutron-vpnaas | lower-constraints.txt | 144 | tinyrpc==0.6 | > > | openstack/os-ken | lower-constraints.txt | 144 | tinyrpc==0.6 | > > | openstack/os-ken | requirements.txt | 13 | tinyrpc>=0.6 # RPC library, BGP speaker(net_cntl) | > > | openstack/requirements | openstack_requirements/tests/files/upper-constraints.txt | 187 | tinyrpc===0.5 | > > | openstack/requirements | upper-constraints.txt | 270 | tinyrpc===1.0.3 | > > | openstack/tricircle | lower-constraints.txt | 149 | tinyrpc==0.6 | > > | openstack/upstream-institute-virtual-environment | elements/upstream-training/static/tmp/requirements.txt | 292 | tinyrpc==1.0.3 | > > | x/flame | upper-constraints.txt | 208 | tinyrpc===0.9.3 | > > | x/networking-arista | lower-constraints.txt | 146 | tinyrpc==0.6 | > > | x/networking-fujitsu | lower-constraints.txt | 142 | tinyrpc==0.8 | > > | x/networking-omnipath | lower-constraints.txt | 144 | tinyrpc==0.6 | > > | x/networking-opencontrail | lower-constraints.txt | 149 | tinyrpc==0.6 | > > | x/networking-vsphere | lower-constraints.txt | 142 | tinyrpc==0.6 | > > | x/neutron-interconnection | lower-constraints.txt | 145 | tinyrpc==0.6 | > > | x/tap-as-a-service | lower-constraints.txt | 139 | tinyrpc==0.8 | > > +--------------------------------------------------+----------------------------------------------------------+------+----------------------------------------------------+ > > > > https://github.com/mbr/tinyrpc/compare/1.0.3...1.0.4 isn't too big but does seem to have a refactor in it. > > > > If you want to depend on a requirements patch for testing, I've created https://review.opendev.org/692915 > > for testing the update. > > > > Thanks, > > > > -- > > Matthew Thode > > — > Slawek Kaplonski > Senior software engineer > Red Hat > Your right, I forgot about that, thanks for the reminder. -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From radoslaw.piliszek at gmail.com Tue Dec 3 08:14:35 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 3 Dec 2019 09:14:35 +0100 Subject: [gnocchi][telemetry][ceilometer][cloudkitty] Gnocchi unmaintained In-Reply-To: References: <54efd463-6af5-3f4e-23db-90a6208d253c@binero.se> <64cdeb4aa46955c22711d7880a1848ba@objectif-libre.com> <9e3c2bd1-69d7-0492-5744-b802a4584551@matthias-runge.de> <0b6bd1f3-acd6-8b9c-7e0e-1f7d241a6aa4@debian.org> <20191121152901.xzyv6o6q6jstc36u@yuggoth.org> <20191121205356.fdwb6oaeoppmk6up@firewall> <8184e8c7-b5aa-9801-c4a6-b1e9f8db11b2@suse.com> Message-ID: +1, it would surely be nice to get to know some final stance on this. -yoctozepto czw., 28 lis 2019 o 06:49 Mohammed Naser napisał(a): > > On Fri, Nov 22, 2019 at 3:50 AM Witek Bedyk wrote: > > > > On 11/21/19 9:53 PM, Nate Johnston wrote: > > > > > I know several very large sites (ingesting billions of records per day) > > > that run community InfluxDB and they get HA by putting influx-proxy [1] > > > in front of it. I've evaluated it for large scale uses before as well, > > > and with influx-proxy I found no need for the clustering option. > > > > Similar architecture is followed by Monasca. InfluxDB instances can be > > assigned to different Kafka consumer groups and consume messages > > independently from the message queue. In case one of the instances is > > down all the measurements are still buffered and get persisted as soon > > as the instance is available again. > > I'm curious on what the final decision of the Ceilometer team regarding > this discussion? > > > Best greetings > > Witek > > > From skaplons at redhat.com Tue Dec 3 08:13:05 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 3 Dec 2019 09:13:05 +0100 Subject: State of the Gate In-Reply-To: <262BDACC-9139-4B66-9E5B-F95CF84BD324@redhat.com> References: <20191031211535.vk7rtiq3pvsb6j2t@skaplons-mac> <262BDACC-9139-4B66-9E5B-F95CF84BD324@redhat.com> Message-ID: Hi, > On 1 Dec 2019, at 11:44, Slawek Kaplonski wrote: > > Hi, > >> On 31 Oct 2019, at 23:03, Matt Riedemann wrote: >> >> On 10/31/2019 4:15 PM, Slawek Kaplonski wrote: >>>> 3. CirrOS guest SSH issues >>>> >>>> There are several (some might be duplicates): >>>> >>>> http://status.openstack.org/elastic-recheck/index.html#1848078 >>> This one is I think the same as we have reported in >>> https://bugs.launchpad.net/neutron/+bug/1850557 >>> Basically we noticed issues with dhcp after resize/migration/shelve of instance >>> but I didn't have time to investigate it yet. >> >> Hmm, https://review.opendev.org/#/c/670591/ is new to devstack in Train and was backported to stable/stein. I wonder if that is too aggressive and is causing issues with operations where the guest is stopped and started, though for resize/migrate/shelve/unshelve the guest is destroyed on one host and re-spawned on another, so I would think that having a graceful shutdown for the guest wouldn't matter in those cases, unless it has to do with leaving the guest "dirty" somehow before transferring the root disk / creating a snapshot (in the case of shelve). Maybe we should bump that back up to 10 seconds? > > I finally spent some time on investigating this issue. I was able to reproduce it locally and I found that the problem is in openvswitch firewall. All is described in [1]. I just pushed patch which should fix this. It’s in [2]. My patch is now merged so I hope we will not see this issue with SSH in resize/shelve tests anymore. > > [1] https://bugs.launchpad.net/neutron/+bug/1850557 > [2] https://review.opendev.org/696794 > >> >> -- >> >> Thanks, >> >> Matt >> > > — > Slawek Kaplonski > Senior software engineer > Red Hat — Slawek Kaplonski Senior software engineer Red Hat From amotoki at gmail.com Tue Dec 3 08:17:58 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Tue, 3 Dec 2019 17:17:58 +0900 Subject: [neutron] Proposing Jakub Libosvar as Neutron core reviewer In-Reply-To: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> References: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> Message-ID: +1 from me. It would be nice to have him again. On Thu, Nov 28, 2019 at 6:01 PM Slawek Kaplonski wrote: > > Hi neutrinos, > > We already started process of migrating networking-ovn driver to be one of in-tree neutron drivers. Blueprint for that is [1]. > As part of this process I today proposed to include networking-ovn-core group into neutron-core group. Mail about it can be found at [2]. > One of persons in networking-ovn-group is Jakub Libosvar who was Neutron core for very long time in the past. He knows very well not only ovn related code but also have great knowledge about all Neutron code base. > So I would like to propose to Jakub as Neutron core reviewer again as he will be back working on neutron again now, after ovn will be in-tree driver. > What do You think about it? > I will wait for Your opinions for 1 week from now. Thx for all Your comments about it. > > [1] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge > [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011240.html > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From alfredo.deluca at gmail.com Tue Dec 3 08:57:56 2019 From: alfredo.deluca at gmail.com (Alfredo De Luca) Date: Tue, 3 Dec 2019 09:57:56 +0100 Subject: VMs auto start Message-ID: Hi all. Just a quick one. I can't find the settings for VMs auto start after a computer node reboots. Can you point me to the right docs? Which one is it? And you need to add it on nova scheduler right? Cheers -- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: From missile0407 at gmail.com Tue Dec 3 09:27:56 2019 From: missile0407 at gmail.com (Eddie Yen) Date: Tue, 3 Dec 2019 17:27:56 +0800 Subject: VMs auto start In-Reply-To: References: Message-ID: Hi Alfredo, You have to add below config into nova.conf on every compute node. resume_guests_state_on_host_boot = True Then restart nova-compute service. - Eddie Alfredo De Luca 於 2019年12月3日 週二 下午5:03寫道： > Hi all. > Just a quick one. I can't find the settings for VMs auto start after a > computer node reboots. > Can you point me to the right docs? > > Which one is it? And you need to add it on nova scheduler right? > > > Cheers > > -- > *Alfredo* > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Tue Dec 3 09:39:58 2019 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 3 Dec 2019 10:39:58 +0100 Subject: [neutron][nova][large scale SIG] Rootwrap daemon and privsep In-Reply-To: References: <20191129155335.GC27522@sync> <20191129155633.GD27522@sync> <43282856-ccd3-fb30-01a2-47e6a1814a06@openstack.org> <20191202100624.GE27522@sync> Message-ID: <56565ce0-3c9c-469e-75c3-42decb023675@openstack.org> Matt Riedemann wrote: > [...] > I want to say mikal converted everything native to nova from rootwrap to > privsep and that was completed in Train: > > https://docs.openstack.org/releasenotes/nova/train.html#security-issues > > "The transition from rootwrap (or sudo) to privsep has been completed > for nova. The only case where rootwrap is still used is to start privsep > helpers. All other rootwrap configurations for nova may now be removed." > > Looking at what's in the compute.filters file looks like it's all stuff > for os-brick, but I though os-brick was fully using privsep natively as > well? Maybe it's just a matter of someone working on this TODO: > > https://opendev.org/openstack/nova/src/branch/master/etc/nova/rootwrap.d/compute.filters#L16 That's great news! I'll have a deeper look and propose changes if appropriate. Cheers, -- Thierry Carrez (ttx) From ltoscano at redhat.com Tue Dec 3 10:30:56 2019 From: ltoscano at redhat.com (Luigi Toscano) Date: Tue, 03 Dec 2019 11:30:56 +0100 Subject: [glance] [glance_store] requirements-check job fails In-Reply-To: <63CC5661-FB21-4969-9A24-9C0231625BF2@global.ntt> References: <63CC5661-FB21-4969-9A24-9C0231625BF2@global.ntt> Message-ID: <49386241.WZNmq7ITLL@whitebase.usersys.redhat.com> On Tuesday, 3 December 2019 04:02:16 CET Naohiro Sameshima（鮫島 直洋）(Group) wrote: > Hi, > > I'm Naohiro. > > I proposed a blueprint [0] (revive s3 driver) in Shanghai PTG and then > create a patch [1]. > So, I submit a patch in gerrit code review, but unfortunately one of zuul > job (requirements-check) failed [2]. > I think it seems that doc/requirements.txt is wrong. (Not included in my > patch) I commented on the review, and you are right; this is the missing bit: https://review.opendev.org/697050 Can you please rebase your patch on top of it^? -- Luigi From alfredo.deluca at gmail.com Tue Dec 3 10:59:42 2019 From: alfredo.deluca at gmail.com (Alfredo De Luca) Date: Tue, 3 Dec 2019 11:59:42 +0100 Subject: VMs auto start In-Reply-To: References: Message-ID: thanks Eddie. Appreciated. Alfredo On Tue, Dec 3, 2019 at 10:28 AM Eddie Yen wrote: > Hi Alfredo, > > You have to add below config into nova.conf on every compute node. > > resume_guests_state_on_host_boot = True > > Then restart nova-compute service. > > - Eddie > > Alfredo De Luca 於 2019年12月3日 週二 下午5:03寫道： > >> Hi all. >> Just a quick one. I can't find the settings for VMs auto start after a >> computer node reboots. >> Can you point me to the right docs? >> >> Which one is it? And you need to add it on nova scheduler right? >> >> >> Cheers >> >> -- >> *Alfredo* >> >> -- *Alfredo* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Tue Dec 3 13:51:07 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 3 Dec 2019 14:51:07 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Message-ID: Hello , now the cluster creation completes but : oot at test-HP-ProBook-450-G3:~# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-698dc979db-9vc2q 0/1 Pending 0 16m calico-node-mszlr 2/2 Running 0 16m calico-node-rxn2r 2/2 Running 0 16m calico-node-s2l5j 2/2 Running 0 16m coredns-865bd969f-kdvpf 0/1 Pending 0 16m heapster-7bf5794cc7-hb9rm 0/1 Pending 0 16m kube-dns-autoscaler-57bd7f54d5-zztvc 0/1 Pending 0 16m kubernetes-dashboard-d48c76949-dr9f4 0/1 Pending 0 16m Using flannel some pods are pending as well. Regards Ignazio Il giorno lun 2 dic 2019 alle ore 16:29 Bharat Kunwar ha scritto: > Yes you can specify comma separated `labels` when you create a cluster > template/cluster from the dashboard. > > e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. > > Also make sure cluster_user_trust=True inside your magnum.conf. > > On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: > > The network plugin is flannel . > I do not kow what is kube_tag . I read that with it I can specify the > kubernetes version, right ? > I am using openstack dashboard for creating my cluster: where I can > specify heat_container_agent_tag: train-stable in the dashboard ? > Thanks > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar > ha scritto: > >> Hi Ignazio, >> >> What version of kube_tag are you trying to run? What network plugin? I >> suggest using `heat_container_agent_tag: train-stable` as its easier to >> debug. >> >> >> Best >> >> bharat >> >> > On 2 Dec 2019, at 15:08, Ignazio Cassano >> wrote: >> > >> > hello, I've just installed stein with magnum on centos 7. I am creating >> a kubernetes cluster but it is running since 22 minutes and I think it will >> not work because the heat stack is waiting OSSoftwareDeployment completes >> > I presume something is not working on heat with magnum. I ran several >> stack with Software Deployment without magnum, and they worked fine >> > >> > Thanks >> > Ignazio >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Tue Dec 3 13:53:04 2019 From: smooney at redhat.com (Sean Mooney) Date: Tue, 3 Dec 2019 13:53:04 +0000 Subject: State of the Gate In-Reply-To: References: Message-ID: On Thu, Oct 31, 2019 at 3:29 PM Matt Riedemann wrote: > > Things are great! Surprise! I just wanted to let everyone know. Later! > . > . > . > . > . > Now that you've been tricked, on Halloween no less, I'm here to tell you > that things suck right now. This is your periodic digest of issues. Grab > some fun-sized candy bars and read on. > > I think right now we have three major issues. > > 1. http://status.openstack.org/elastic-recheck/index.html#1763070 > > This has resurfaced and I'm not sure why, nor do I think we ever had a > great handle on what is causing this or how to work around it so if > anyone has new ideas please chip in. the only theory i have on that is in the absense of any other indication we could be running out of entorpy in the vm which can lead to https connections failing. we might want to consider enabling the virtio random number generator in the gate vms. https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L54-L59 this need to be enabled in the flavours too but low entroy can cass ssl connection to fail https://major.io/2007/07/01/check-available-entropy-in-linux/ > > 2. http://status.openstack.org/elastic-recheck/index.html#1844929 > > I've done some digging into this one and my notes are in the bug report. > It mostly affects grenade jobs but is not entirely restricted to grenade > jobs. It's also mostly on OVH and FortNebula nodes but not totally. > > From looking at the mysql logs in the grenade jobs, mysqld is > (re)started three times. I think (1) for initial package install, (2) > for stacking devstack on the old side, and (3) for stacking devstack on > the new side. After the last restart, there are a lot of aborted > connection messages in the msyql error log. It's around then that > grenade is running post-upgrade smoke tests to create a server and the > nova-scheduler times out communicating with the nova_cell1 database. > > I have a few patches up to grenade/devstack [1] to try some things and > get more msyql logs but so far they aren't really helpful. We need > someone with some more mysql debugging experience to help here, maybe > zzzeek or mordred? > > 3. CirrOS guest SSH issues > > There are several (some might be duplicates): > > http://status.openstack.org/elastic-recheck/index.html#1848078 > http://status.openstack.org/elastic-recheck/index.html#1808010 (most hits) > http://status.openstack.org/elastic-recheck/index.html#1463631 > http://status.openstack.org/elastic-recheck/index.html#1849857 > http://status.openstack.org/elastic-recheck/index.html#1737039 > http://status.openstack.org/elastic-recheck/index.html#1840355 > http://status.openstack.org/elastic-recheck/index.html#1843610 > > A few notes here. > > a) We're still using CirrOS 0.4.0 since Stein: > > https://review.opendev.org/#/c/521825/ > > And that image was published nearly 2 years ago and there are no newer > versions on the CirrOS download site so we can't try a newer image to > see if that fixes things. > > b) Some of the issues above are related to running out of disk in the > guest. I'm not sure what is causing that, but I have posted a devstack > patch that is related: > > https://review.opendev.org/#/c/690991 > > tl;dr before Stein the tempest flavors we used had disk=0 so nova would > fit the root disk to the size of the image. Since Stein the tempest > flavors specify root disk size (1GiB for the CirrOS images). My patch > pads an extra 1GiB to the root disk on the tempest flavors. One side > effect of that is the volumes tempest creates will go from 1GiB to 2GiB > which could be a problem if a lot of tempest volume tests run at the > same time, though we do have a volume group size of 24GB in gate runs so > I think we're OK for now. I'm not sure my patch would help, but it's an > idea. > > As for the other key generation and dhcp lease failures, I don't know > what to do about those.We need more eyes on these issues to generate so the ssh key generation issue may also be down to entorpoy. i have not looked at those specific failueres but i did note in some failed test in the past the we printed the kernel entropy in the guest and it was like 36 so some other very low number(it should be in the hundreds). if we have low entropy key generation will take a long time. https://wiki.debian.org/BoottimeEntropyStarvation > some ideas or see if we're doing something wrong in our tests, e.g. > generating too much data for the config drive? Not using config drive in > some cases? Metadata API server is too slow (note we cache the metadata > since [2])? Other ideas on injecting logs for debug? > > [1] https://review.opendev.org/#/q/topic:bug/1844929+status:open > [2] https://review.opendev.org/#/q/I9082be077b59acd3a39910fa64e29147cb5c2dd7 > > -- > > Thanks, > > Matt > From mriedemos at gmail.com Tue Dec 3 14:32:21 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 3 Dec 2019 08:32:21 -0600 Subject: State of the Gate In-Reply-To: References: <20191031211535.vk7rtiq3pvsb6j2t@skaplons-mac> <262BDACC-9139-4B66-9E5B-F95CF84BD324@redhat.com> Message-ID: <7bc93b94-3c49-96a0-1382-e5a6663d7ed4@gmail.com> On 12/3/2019 2:13 AM, Slawek Kaplonski wrote: >> I finally spent some time on investigating this issue. I was able to reproduce it locally and I found that the problem is in openvswitch firewall. All is described in [1]. I just pushed patch which should fix this. It’s in [2]. > My patch is now merged so I hope we will not see this issue with SSH in resize/shelve tests anymore. > Awesome. Thanks as always for digging into gate issues Slawek! -- Thanks, Matt From fungi at yuggoth.org Tue Dec 3 14:42:48 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 3 Dec 2019 14:42:48 +0000 Subject: State of the Gate In-Reply-To: References: Message-ID: <20191203144247.ec7gvw25mqcmoy6h@yuggoth.org> On 2019-12-03 13:53:04 +0000 (+0000), Sean Mooney wrote: [...] > we might want to consider enabling the virtio random number > generator in the gate vms. [...] That should be safe to do regardless; we already pre-install haveged in them so they have a limitless supply of entropy to provide to the guests Tempest boots. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From elod.illes at est.tech Tue Dec 3 15:00:18 2019 From: elod.illes at est.tech (=?utf-8?B?RWzDtWQgSWxsw6lz?=) Date: Tue, 3 Dec 2019 15:00:18 +0000 Subject: [cinder][ops][extended-maintenance-sig][public-cloud-sig][enterprise-wg] Cinder to EOL some branches In-Reply-To: <70f9cd7b-535d-8f6b-dbe2-3578bd21c346@gmail.com> References: <3c5fd6e6-8ae4-d300-71a7-97b22431cb3b@gmail.com> <7e8b4158-89b2-6207-6f06-215782e0b126@est.tech> <70f9cd7b-535d-8f6b-dbe2-3578bd21c346@gmail.com> Message-ID: <6ca22c6a-c0c8-711a-91b5-111702cc2d06@est.tech> Hi Brian, Thanks in advance for proposing in Cinder meeting to wait to EOL Pike and Ocata. So far I have been looking after stable/* branches. I had a quick check and it seems that the CI on driverfixes/* branches are not in a good shape for some time. Those branches: - don't have periodic jobs, so we can't see how long they are in this faulty state - the test jobs are very limited - as I see there aren't really arriving patches lately (actually one arrived in newton ~2 months ago) I can try to search for solutions there to fix the gate, but maybe the activity there is less than ocata and pike and if there aren't others who are interested in driverfixes/* branches then maybe those reached the state to move them to EOL. Thanks again, Előd On 2019. 12. 02. 16:25, Brian Rosmaita wrote: > On 11/27/19 12:02 PM, Elõd Illés wrote: >> Hi, >> >> First of all, sorry for the late response. >> >> About EOLing Ocata and Pike: Extended Maintenance was formed just to >> have a common place for interested parties, vendors, operators, to push >> bugfixes to. Currently these branches are in a good shape, check / gate >> jobs work and as far as I see there are a couple of backports, too (not >> too many, though). So hopefully it's not a waste of efforts. Why don't >> we keep them open as long as the CI works and there are patches? Of >> course, whenever e.g. Ocata branch / CI becomes unmaintainable (zuul v3 >> incompatibilities) or there aren't anyone who fixes the issues there, we >> can put it to EOL then. > > Quick question: the driverfixes/mitaka and driverfixes/newton gates > are broken now.  Do you have any interest in getting these working, or > are you only interested in stable/ocata and stable/pike? > >> I write this, because my employer supports Extended Maintenance, and I >> usually try to fix CI issues on stable branches and reviewing patches >> there. > > Thanks for following this strategy, it's good for the community, and > we appreciate it. > >> So maybe I can be some help here, too. Please consider leaving >> branches in Extended Maintenance open as long as they are in a good >> shape and there are bugfixes coming. > > This is good feedback.  As I said in the original email, we don't mind > keeping these open as long as people are actually using them.  I'll > put a proposal on this week's Cinder meeting agenda to wait to EOL > ocata and pike until their gates break and no one steps up to fix > them, instead of proactively EOL'ing them now. > > Do let me know your position on the driverfixes branches right away, > though. > > > thanks, > brian > >> >> Thanks, >> >> Előd >> >> >> On 2019. 11. 25. 20:21, Brian Rosmaita wrote: >>> This is a courtesy notice that having received no responses to my >>> email of 28 October [0] proposing to EOL some currently open Cinder >>> branches, and following the policy articulated in [1], at today's >>> Virtual PTG meeting the Cinder project team has decided to put the >>> following stable branches into the End of Life state: >>>    driverfixes/mitaka >>>    driverfixes/newton >>>    stable/ocata >>>    stable/pike >>> >>> I will submit the paperwork to get this process moving one week from >>> today (2 December 2019). >>> >>> >>> cheers, >>> brian >>> >>> [0] >>> http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010385.html >>> >>> [1] >>> https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-life >>> >>> >> > > From skaplons at redhat.com Tue Dec 3 15:12:02 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 3 Dec 2019 16:12:02 +0100 Subject: State of the Gate In-Reply-To: References: Message-ID: <20191203151202.jdhqh6z4irpepz4l@skaplons-mac> Hi, On Tue, Dec 03, 2019 at 01:53:04PM +0000, Sean Mooney wrote: > On Thu, Oct 31, 2019 at 3:29 PM Matt Riedemann wrote: > > > > Things are great! Surprise! I just wanted to let everyone know. Later! > > . > > . > > . > > . > > . > > Now that you've been tricked, on Halloween no less, I'm here to tell you > > that things suck right now. This is your periodic digest of issues. Grab > > some fun-sized candy bars and read on. > > > > I think right now we have three major issues. > > > > 1. http://status.openstack.org/elastic-recheck/index.html#1763070 > > > > This has resurfaced and I'm not sure why, nor do I think we ever had a > > great handle on what is causing this or how to work around it so if > > anyone has new ideas please chip in. > the only theory i have on that is in the absense of any other indication > we could be running out of entorpy in the vm which can lead to https > connections failing. we might want to consider enabling the virtio > random number generator in > the gate vms. > https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L54-L59 > this need to be enabled in the flavours too but low entroy can cass > ssl connection to fail > https://major.io/2007/07/01/check-available-entropy-in-linux/ You may be right with lack of entropy is culprit in some issues. E.g. here: https://zuul.opendev.org/t/openstack/build/edcf837457a741abb752693723319b15/log/controller/logs/tempest_log.txt.gz#14652 it took 120 seconds to Initialize random number generator on guest VM and because of that network wasn't configured properly and test failed. > > > > 2. http://status.openstack.org/elastic-recheck/index.html#1844929 > > > > I've done some digging into this one and my notes are in the bug report. > > It mostly affects grenade jobs but is not entirely restricted to grenade > > jobs. It's also mostly on OVH and FortNebula nodes but not totally. > > > > From looking at the mysql logs in the grenade jobs, mysqld is > > (re)started three times. I think (1) for initial package install, (2) > > for stacking devstack on the old side, and (3) for stacking devstack on > > the new side. After the last restart, there are a lot of aborted > > connection messages in the msyql error log. It's around then that > > grenade is running post-upgrade smoke tests to create a server and the > > nova-scheduler times out communicating with the nova_cell1 database. > > > > I have a few patches up to grenade/devstack [1] to try some things and > > get more msyql logs but so far they aren't really helpful. We need > > someone with some more mysql debugging experience to help here, maybe > > zzzeek or mordred? > > > > 3. CirrOS guest SSH issues > > > > There are several (some might be duplicates): > > > > http://status.openstack.org/elastic-recheck/index.html#1848078 > > http://status.openstack.org/elastic-recheck/index.html#1808010 (most hits) > > http://status.openstack.org/elastic-recheck/index.html#1463631 > > http://status.openstack.org/elastic-recheck/index.html#1849857 > > http://status.openstack.org/elastic-recheck/index.html#1737039 > > http://status.openstack.org/elastic-recheck/index.html#1840355 > > http://status.openstack.org/elastic-recheck/index.html#1843610 > > > > A few notes here. > > > > a) We're still using CirrOS 0.4.0 since Stein: > > > > https://review.opendev.org/#/c/521825/ > > > > And that image was published nearly 2 years ago and there are no newer > > versions on the CirrOS download site so we can't try a newer image to > > see if that fixes things. > > > > b) Some of the issues above are related to running out of disk in the > > guest. I'm not sure what is causing that, but I have posted a devstack > > patch that is related: > > > > https://review.opendev.org/#/c/690991 > > > > tl;dr before Stein the tempest flavors we used had disk=0 so nova would > > fit the root disk to the size of the image. Since Stein the tempest > > flavors specify root disk size (1GiB for the CirrOS images). My patch > > pads an extra 1GiB to the root disk on the tempest flavors. One side > > effect of that is the volumes tempest creates will go from 1GiB to 2GiB > > which could be a problem if a lot of tempest volume tests run at the > > same time, though we do have a volume group size of 24GB in gate runs so > > I think we're OK for now. I'm not sure my patch would help, but it's an > > idea. > > > > As for the other key generation and dhcp lease failures, I don't know > > what to do about those.We need more eyes on these issues to generate > > so the ssh key generation issue may also be down to entorpoy. > i have not looked at those specific failueres but i did note in some failed > test in the past the we printed the kernel entropy in the guest and it > was like 36 > so some other very low number(it should be in the hundreds). if we > have low entropy key generation will take > a long time. https://wiki.debian.org/BoottimeEntropyStarvation > > > > some ideas or see if we're doing something wrong in our tests, e.g. > > generating too much data for the config drive? Not using config drive in > > some cases? Metadata API server is too slow (note we cache the metadata > > since [2])? Other ideas on injecting logs for debug? > > > > [1] https://review.opendev.org/#/q/topic:bug/1844929+status:open > > [2] https://review.opendev.org/#/q/I9082be077b59acd3a39910fa64e29147cb5c2dd7 > > > > -- > > > > Thanks, > > > > Matt > > > > -- Slawek Kaplonski Senior software engineer Red Hat From amy at demarco.com Tue Dec 3 15:21:53 2019 From: amy at demarco.com (Amy Marrich) Date: Tue, 3 Dec 2019 09:21:53 -0600 Subject: [openstack-community] Openstack FW rules In-Reply-To: References: Message-ID: Hi Lior, Forwarding this to the OpenStack discuss list where someone might be able to help you. Thanks, Amy (spotz) On Tue, Dec 3, 2019 at 7:41 AM Lior Fellus wrote: > Hi, > i am using OpenStack "Mitaka" and we have 3 controllers and 3 computes on > our environment and all managed by Fuel server. > recently i implemented a Veeam backup Server in our environment and in > order to backup the computes and the controllers i need to open ssh to them > directly because Veeam not support ssh tunnel via Fuel server. > can anyone explain me how ? > > Regards, > _______________________________________________ > Community mailing list > Community at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/community > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nate.johnston at redhat.com Tue Dec 3 15:23:35 2019 From: nate.johnston at redhat.com (Nate Johnston) Date: Tue, 3 Dec 2019 10:23:35 -0500 Subject: [neutron] Bug Deputy report 2019-11-25 - 2019-12-02 Message-ID: <20191203152335.b2sgwv3k7zlim2r7@firewall> Here are my notes as Neutron bug deputy for last week. It was a pretty slow week. Critical: - "Old conjunction left after sg update" * URL: https://bugs.launchpad.net/neutron/+bug/1854131 * Status: Fix merged (yang-li) - "py36 unit test cases fails" * URL: https://bugs.launchpad.net/bugs/1854051 * Gate-Failure * Status: Unassigned, may be a typing issue, will bring it up in Neutron CI meeting. High: - "Neutron fails to create bandwidth providers if CONF.host is set" * URL: https://bugs.launchpad.net/neutron/+bug/1853840 * Status: In progress, assigned to rubasov - "[Functional tests] Timeout exception in list_namespace_pids" * URL: https://bugs.launchpad.net/bugs/1854462 * Gate-Failure * Status: Unassigned, will bring it up in Neutron CI meeting. Medium: - "The /v2.0/ports/{port_id}/bindings APIs are not documented" * URL: https://bugs.launchpad.net/bugs/1853873 * Status: Assigned to mlavalle Low: - "Avoid raising NetworkInterfaceNotFound exception in DHCP agent logs" * URL: https://bugs.launchpad.net/bugs/1854723 * Status: In progress, assigned to ralonsoh - "DhcpLocalProcess "_enable" method should not call 'restart' --> 'enable'" * URL: https://bugs.launchpad.net/neutron/+bug/1854940 * Status: Assigned to ralonsoh Invalid: - "minor versions 14.0.2 & 14.0.3 are not compatible in dvr-ha" * URL: https://bugs.launchpad.net/bugs/1854050 * Status: agent running higher version than neutron server not supported Thanks, Nate From mihalis68 at gmail.com Tue Dec 3 15:32:53 2019 From: mihalis68 at gmail.com (Chris Morgan) Date: Tue, 3 Dec 2019 10:32:53 -0500 Subject: [ops] ops meetups team meeting 2019-12-3 Message-ID: Minutes from todays meeting on #openstack-operators Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_team.2019-12-03-15.03.html Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_team.2019-12-03-15.03.txt Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_team.2019-12-03-15.03.log.html London meetup links: https://go.bloomberg.com/attend/invite/openstack-operators-meetup-london2020/ https://etherpad.openstack.org/p/LON-2020-OPS-TOPICS South Korea meetup proposal: https://etherpad.openstack.org/p/ops-meetup-2nd-korea-2020 Event notification twitter account: https://twitter.com/osopsmeetup Chris Morgan - on behalf of the OpenStack Ops Meetups team -- Chris Morgan -------------- next part -------------- An HTML attachment was scrubbed... URL: From amy at demarco.com Tue Dec 3 16:09:48 2019 From: amy at demarco.com (Amy Marrich) Date: Tue, 3 Dec 2019 10:09:48 -0600 Subject: [External] Re: [openstack-community] how do I revert a volume snapshot? In-Reply-To: <542c8678c84f4b01b3544b10f74c1d51@IN-CCI-D1S14.ads.iu.edu> References: <51e6da9e5c674cf6bca53627879d466c@IN-CCI-D1S14.ads.iu.edu> <542c8678c84f4b01b3544b10f74c1d51@IN-CCI-D1S14.ads.iu.edu> Message-ID: John, Sorry I misread what you were trying to do. I thought you wanted to create a new instance with the snapshot not revert to it. I've included the link for how to revert volumes. https://docs.openstack.org/cinder/pike/admin/blockstorage-volume-backups.html I'm also forwarding this email to the OpenStack Discuss list which is the appropriate list for dev and ops issues. Thanks, Amy (spotz) On Tue, Dec 3, 2019 at 9:29 AM Ratliff, John wrote: > Thanks, but that feels like a really complicated process to rollback. It > seems I would need to delete my old machine and volume, make a volume from > the snapshot, create a new one with the same port, a new volume and specify > the cloud-init script. > > > > Is that really the best way to revert to a snapshot? I don’t have a lot of > experience with openstack, and the documentation and scope is quite large, > but I’ve used vmware, libvirt with qemu, hyper-v, and virtualbox, and they > all had a fairly simple one-step process to revert a snapshot. I feel like > I’m missing something easy. > > > > Thanks. > > > > --John > > > > *From:* Amy Marrich > *Sent:* Monday, December 2, 2019 1:49 PM > *To:* Ratliff, John > *Cc:* community at lists.openstack.org > *Subject:* [External] Re: [openstack-community] how do I revert a volume > snapshot? > > > > This message was sent from a non-IU address. Please exercise caution when > clicking links or opening attachments from external sources. > > > > You should be able to create a new instance using that snapshot as the > image. > > > > Amy (spotz) > > > > On Mon, Dec 2, 2019 at 11:27 AM Ratliff, John wrote: > > I am trying to create an instance that I can revert to a snapshot. I > created the snapshot with openstack volume snapshot create, but I don’t see > a way to revert the instance to that image. > > > > What can I do to make use of the snapshot? > > > > Thanks. > > > > --John Ratliff > > > > > > > > _______________________________________________ > Community mailing list > Community at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/community > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From km.giuseppesannino at gmail.com Tue Dec 3 16:24:10 2019 From: km.giuseppesannino at gmail.com (Giuseppe Sannino) Date: Tue, 3 Dec 2019 17:24:10 +0100 Subject: [kolla][cinder][glance][ceph] Corrupt image while downloading from Ceph Message-ID: Hi community, need your help. *>>> Background <<<* I'm using kolla-ansible 8.0.0 to deploy a 1+3 "Stein" cluster. Ceph is used as backend. The configuration is a bit peculiar. The control runs on a VM hosted in a separate network compared to the one where the baremetal servers hosting the OS Compute services are. On the Compute Hosts, we have the following services: glance_api neutron_metadata_agent neutron_l3_agent neutron_dhcp_agent neutron_openvswitch_agent openvswitch_vswitchd openvswitch_db nova_compute nova_libvirt nova_ssh cinder_backup cinder_volume chrony cron kolla_toolbox fluentd Services APIs and Authentication run on the controller. In a standard "lab configuration" everything works fine. *>>> Fault Scenario <<<* We are trying to verify possible issues (and the way to work around them) in case latency between Controller and Compuite increases. And we have found one quite fast. Basically, if you try to create a volume from a RAW image (stored in Ceph) it will fail. >From glance-api.log on the controller: 2019-12-03 16:00:11.840 27 INFO eventlet.wsgi.server [req-225aae45-ad93-40f5-835d-027f93e3307d 615252134b844dbeb7acc34219e431e6 0049baebd0f742de915b11ec18509803 - default default] Traceback (most recent call last): File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/wsgi.py", line 572, in handle_one_response write(b''.join(towrite)) File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/wsgi.py", line 518, in write wfile.writelines(towrite) File "/usr/lib64/python2.7/socket.py", line 334, in writelines self.flush() File "/usr/lib64/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/greenio/base.py", line 401, in sendall tail = self.send(data, flags) File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/greenio/base.py", line 395, in send return self._send_loop(self.fd.send, data, flags) File "/var/lib/kolla/venv/lib/python2.7/site-packages/eventlet/greenio/base.py", line 382, in _send_loop return send_method(data, *args) error: [Errno 104] Connection reset by peer >From the cinder-volume.log on the computes: : 019-12-03 16:00:15.932 34 ERROR oslo_messaging.rpc.server None, None) 2019-12-03 16:00:15.932 34 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python2.7/site-packages/cinder/image/image_utils.py", line 410, in fetch 2019-12-03 16:00:15.932 34 ERROR oslo_messaging.rpc.server reason=reason) 2019-12-03 16:00:15.932 34 ERROR oslo_messaging.rpc.server ImageDownloadFailed: Failed to download image 6e7bb902-917e-4c9e-ba9f-3ee811a2502a, reason: IOError: 32 Corrupt image download. Hash was 88b062103e34c9824d7172afaa9a80befd00e1bef86d16a362572f01bd887a0551c188e98526eecdeedca262d3364175d384352c10d203bdb6a5b87b0593f231 expected adc29d5ce6129337e1e9bf00cc3f0798682c021c6f1a0aab46213438a6de8c6b027180389aa21196e7f708214815221a9a0c6029a96badafefca624bf58e4bff *>>> Troubleshooting <<<* At a first glance it seems a problem related to the size of the image. We have tried with: Cirros Raw (39MB) => It works Ubuntu18 QCOW2 (328MB) => It works Ubuntu18 Raw (2.2GB) => IT FAILS !!!! Any suggestion about where to address our effort? Many thanks in advance BR /Giuseppe -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Tue Dec 3 17:01:49 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Tue, 3 Dec 2019 12:01:49 -0500 Subject: [cinder] weekly meeting change reminder Message-ID: <57396165-cd45-81b8-5ffe-7801f40da514@gmail.com> This is a quick reminder that this week's Cinder meeting will be held as usual on Wednesday, but: new time: 1400 UTC new channel: #openstack-meeting-4 See you there! brian From Rajini.Karthik at Dell.com Tue Dec 3 17:13:00 2019 From: Rajini.Karthik at Dell.com (Rajini.Karthik at Dell.com) Date: Tue, 3 Dec 2019 17:13:00 +0000 Subject: [Thirdparty CI] Thirdparty CI Site is down Message-ID: <690a4fed37d94f55b6cb7f1e46982f10@AUSX13MPS308.AMER.DELL.COM> The thirdparty CI site is down http://ciwatch.mmedvede.net/project?project=ironic&time=7+days, no results are displayed. Anyone has any insights? Thanks Rajini -------------- next part -------------- An HTML attachment was scrubbed... URL: From eandersson at blizzard.com Tue Dec 3 17:24:54 2019 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Tue, 3 Dec 2019 17:24:54 +0000 Subject: [neutron] api performance at scale Message-ID: Is there a SIG or similar discussion on neutron performance at scale? For us nova used to be the biggest concern, but a lot of work has been done and nova now performers great. Instead we are having issues to get Neutron to perform at scale. Obvious calls like security groups are performing really poorly, and nova-compute defaults for refreshing the network cache on computes causes massive issues with Neutron. Best Regards, Erik Olof Gunnar Andersson -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaser at vexxhost.com Tue Dec 3 17:40:30 2019 From: mnaser at vexxhost.com (Mohammed Naser) Date: Tue, 3 Dec 2019 12:40:30 -0500 Subject: [Thirdparty CI] Thirdparty CI Site is down In-Reply-To: <690a4fed37d94f55b6cb7f1e46982f10@AUSX13MPS308.AMER.DELL.COM> References: <690a4fed37d94f55b6cb7f1e46982f10@AUSX13MPS308.AMER.DELL.COM> Message-ID: On Tue, Dec 3, 2019 at 12:17 PM wrote: > > > > The thirdparty CI site is down http://ciwatch.mmedvede.net/project?project=ironic&time=7+days, no results are displayed. > This service deployment is not maintained by OpenStack Infra team. If the service needs attention, please let me know: contact at mmedvede.net > > Anyone has any insights? > > Thanks > > Rajini > > -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From mriedemos at gmail.com Tue Dec 3 17:44:18 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 3 Dec 2019 11:44:18 -0600 Subject: [neutron] api performance at scale In-Reply-To: References: Message-ID: On 12/3/2019 11:24 AM, Erik Olof Gunnar Andersson wrote: > For us nova used to be the biggest concern, but a lot of work has been > done and nova now performers great. Instead we are having issues to get > Neutron to perform at scale. Obvious calls like security groups are > performing really poorly, and nova-compute defaults for refreshing the > network cache on computes causes massive issues with Neutron. I wonder how much of the performance hit is due to rootwrap usage in neutron (nova's conversion to privsep was completed in Train). Nova might be the bees knees, but I know there are things in nova we could do to be smarter about not hammering the neutron API as much, e.g.: https://review.opendev.org/#/c/465792/ - make bulk queries to neutron when refreshing the instance network info cache https://review.opendev.org/#/q/I7de14456d04370c842b4c35597dca3a628a826a2 - be smarter about filtering to avoid expensive joins https://bugs.launchpad.net/nova/+bug/1567655 - nova's internal network info cache only stores information about ports and their related networks/subnets/ips but the security group information related to the ports attached to a server is fetched directly anytime it's needed, including when listing servers with details. So if you're an admin listing all servers across all tenants, that could get pretty slow. I've long thought we should cache the security group information like we do for ports for read-only operations like GET /servers/detail but it's a non-trivial amount of work to make that happen and we'd definitely want benchmarks and such to justify the change. Note ttx has started a large ops SIG or whatever so this is probably something to discuss there: https://wiki.openstack.org/wiki/Large_Scale_SIG -- Thanks, Matt From mriedemos at gmail.com Tue Dec 3 17:44:52 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 3 Dec 2019 11:44:52 -0600 Subject: [neutron] api performance at scale In-Reply-To: References: Message-ID: <74a4d72a-8a16-7f0f-5851-eef22e988e8b@gmail.com> On 12/3/2019 11:24 AM, Erik Olof Gunnar Andersson wrote: > For us nova used to be the biggest concern, but a lot of work has been > done and nova now performers great. Instead we are having issues to get > Neutron to perform at scale. Obvious calls like security groups are > performing really poorly, and nova-compute defaults for refreshing the > network cache on computes causes massive issues with Neutron. I wonder how much of the performance hit is due to rootwrap usage in neutron (nova's conversion to privsep was completed in Train). Nova might be the bees knees, but I know there are things in nova we could do to be smarter about not hammering the neutron API as much, e.g.: https://review.opendev.org/#/c/465792/ - make bulk queries to neutron when refreshing the instance network info cache https://review.opendev.org/#/q/I7de14456d04370c842b4c35597dca3a628a826a2 - be smarter about filtering to avoid expensive joins https://bugs.launchpad.net/nova/+bug/1567655 - nova's internal network info cache only stores information about ports and their related networks/subnets/ips but the security group information related to the ports attached to a server is fetched directly anytime it's needed, including when listing servers with details. So if you're an admin listing all servers across all tenants, that could get pretty slow. I've long thought we should cache the security group information like we do for ports for read-only operations like GET /servers/detail but it's a non-trivial amount of work to make that happen and we'd definitely want benchmarks and such to justify the change. Note ttx has started a large ops SIG or whatever so this is probably something to discuss there: https://wiki.openstack.org/wiki/Large_Scale_SIG -- Thanks, Matt From arnaud.morin at gmail.com Tue Dec 3 17:49:57 2019 From: arnaud.morin at gmail.com (Arnaud MORIN) Date: Tue, 3 Dec 2019 18:49:57 +0100 Subject: [neutron][nova][large scale SIG] Rootwrap daemon and privsep In-Reply-To: <56565ce0-3c9c-469e-75c3-42decb023675@openstack.org> References: <20191129155335.GC27522@sync> <20191129155633.GD27522@sync> <43282856-ccd3-fb30-01a2-47e6a1814a06@openstack.org> <20191202100624.GE27522@sync> <56565ce0-3c9c-469e-75c3-42decb023675@openstack.org> Message-ID: Hey, Thanks for the explanations! Le mar. 3 déc. 2019 à 10:43, Thierry Carrez a écrit : > Matt Riedemann wrote: > > [...] > > I want to say mikal converted everything native to nova from rootwrap to > > privsep and that was completed in Train: > > > > https://docs.openstack.org/releasenotes/nova/train.html#security-issues > > > > "The transition from rootwrap (or sudo) to privsep has been completed > > for nova. The only case where rootwrap is still used is to start privsep > > helpers. All other rootwrap configurations for nova may now be removed." > > > > Looking at what's in the compute.filters file looks like it's all stuff > > for os-brick, but I though os-brick was fully using privsep natively as > > well? Maybe it's just a matter of someone working on this TODO: > > > > > https://opendev.org/openstack/nova/src/branch/master/etc/nova/rootwrap.d/compute.filters#L16 > > That's great news! I'll have a deeper look and propose changes if > appropriate. > > Cheers, > > -- > Thierry Carrez (ttx) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sshnaidm at redhat.com Tue Dec 3 18:18:49 2019 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Tue, 3 Dec 2019 20:18:49 +0200 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: References: Message-ID: Hi, all In the meeting today we agreed to meet every Thursday starting *this week* at 4.00 PM UTC on #openstack-sdks channel on Freenode. We'll discuss everything related to Openstack Ansible modules. Agenda and topics are in the etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules (I've created a new one, because we don't limit to Ironic modules only, it's about all of them in general) Short minutes from meeting today: Organizational: 1. We meet every Thursday from this week at 4.00 PM UTC on #openstack-sdks 2. Interested parties for now are: Ironic, Tripleo, Openstack-Ansible, Kolla-ansible, OpenstackSDK teams. Feel free to join and add yourself in the etherpad. [1] 3. We'll track our work in Storyboard for ansible-collections-openstack (in progress) 4. Openstack Ansible modules will live as collections under Ansible SIG in repo openstack/ansible-collections-openstack [2] because there are issues with different licensing: GPLv3 for Ansible in upstream and Openstack license (Apache2). 5. Ansible upstream Openstack modules will be merge-frozen when we'll have our collections fully working and will be deprecated from Ansible at some point in the future. 6. Openstack Ansible collections will be published to Galaxy. 7. There is a list of people that can be pinged for reviews in ansible-collections-openstack project, feel free to join there [1] Technical: 1. We use openstacksdk instead of [project]client modules. 2. We will rename modules to be more like os_[service_type] named, examples are in Ironic modules etherpad [3] Logs from meeting today you can find here: http://eavesdrop.openstack.org/meetings/ansible_sig/2019/ansible_sig.2019-12-03-15.01.log.html Please feel free to participate and add topics to agenda. [1] [1] https://etherpad.openstack.org/p/openstack-ansible-modules [2] https://review.opendev.org/#/c/684740/ [3] https://etherpad.openstack.org/p/ironic-ansible-modules Thanks On Wed, Nov 27, 2019 at 7:57 PM Sagi Shnaidman wrote: > Hi, all > > in the light of finding the new home place for openstack related ansible > modules [1] I'd like to discuss the best strategy to create Ironic ansible > modules. Existing Ironic modules in Ansible repo don't cover even half of > Ironic functionality, don't fit current needs and definitely require an > additional work. There are a few topics that require attention and better > be solved before modules are written to save additional work. We prepared > an etherpad [2] with all these questions and if you have ideas or > suggestions on how it should look you're welcome to update it. > We'd like to decide the final place for them, name conventions (the most > complex one!), what they should look like and how better to implement. > Anybody interested in Ansible and baremetal management in Openstack, > you're more than welcome to contribute. > > Thanks > > [1] https://review.opendev.org/#/c/684740/ > [2] https://etherpad.openstack.org/p/ironic-ansible-modules > > -- > Best regards > Sagi Shnaidman > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From zbitter at redhat.com Tue Dec 3 19:50:55 2019 From: zbitter at redhat.com (Zane Bitter) Date: Tue, 3 Dec 2019 14:50:55 -0500 Subject: [heat] Addressing the large patch backlog. In-Reply-To: References: Message-ID: <2940d9b7-d979-539d-79af-b54e2ee83a9f@redhat.com> On 12/11/19 12:21 pm, David Peacock wrote: > Hi all, > > For those interested in the workings of the Heat project, I'd like to > kick off a call to action. > > At the time of writing there are approximately 200 open patches against > the core Heat project repo alone, not counting the other Heat repos. > Recently I started going through and triaging the patches I'd consider > "historical" with an arbitrary cut off for this definition of August 1st > of this year. > > There are 148 patches which meet this definition, dating all the way > back to 2015.  I have gone through them all and placed them into a > spreadsheet [0] which I'd invite you all to check.  Provided is a link > to the patch in question, initial upload date, last meaningful update > date, primary author, and a high level summary of the patch. Thanks for doing this David! I added a column for "Core Reviewer comments" as well. > Additionally I've broken the patches down into three recommended states > based on a high level first pass. > > *_Abandon_* > > 34 patches are candidates to be abandoned; they usually are of > debatable utility, have significant outstanding concerns, or have no > followup from the original developer in a very long time.  In many > cases, all of these conditions. *Without good reason or explanation from > the original developer, these patches may ultimately be cleared out.* I went through and abandoned 26 of these patches. There were a handful of patches recommended for abandoning that I think still have value to be salvaged from. I added comments in the spreadsheet, prefixed with (ZB). > *_Rebase + Merge_* > > 38 patches are with a high level look in reasonably good shape, perform > a stated goal, and may be trivial to core review and ultimately rebase > and merge. *If you're the original developer or otherwise interested in > these patches and wish to see them through the merge process, please > rebase the patch.* Most of the original developers here are long gone, and those that aren't have probably long since given up on these patches. Would you have time to start going through these and rebase or recheck them (as required), then provide a list somewhere of priority reviews (i.e. those that rebased easily and passed tests) for cores to go through? It should be pretty easy to merge a good number of these quite quickly. (Maybe wait for the gate to be unbroken first though.) > *_Research_* > > 76 patches are sufficiently complex that they'll need a much closer > look.  Some of these patches are in a seemingly "finished" state, some > are a way off.  Some have unanswered concerns from core review and have > been left dangling. *If you're the original developer or otherwise > interested in working these patches through to completion, please do get > involved.* As a first step it'd be great if we could extract from this a list of stuff that looks promising but hasn't been reviewed by a core. Those would be the next highest priority to review I'd expect. > When I started this little mission I wasn't quite sure what to expect. > What I have found is that as much as there was anticipated cruft to > clear out, there is a great deal of very good work lurking here, waiting > to see the light of day, and it would be so good to see this work > realised.  :-) +1 > If you have anything to say, feel free to write back on list, and if > you'd like to coordinate with me any efforts with these patches I can be > found by email or on Freenode in the #heat channel; I'm dpeacock. > > Based on feedback of this idea, and indeed on each individual patch, I > hope we can get this backlog under control, and harvest some of this > excellent code! > > Thank you, > David Peacock > > [0] https://ethercalc.openstack.org/b3qtqyhkg9g1 Please be mindful of > accidental edits. > > From cboylan at sapwetik.org Tue Dec 3 20:13:41 2019 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 03 Dec 2019 12:13:41 -0800 Subject: Thread on OpenDev Independence and Governance Message-ID: <705bf3ff-b25c-4969-98c8-e708ce77b343@www.fastmail.com> Hello, I wanted everyone to be aware of this thread [0] at openstack-infra at lists.openstack.org that I've started to formalize OpenDev's future independence and governance. Currently the infra team is formally an OpenStack project, but we think that we'll be better able to serve our non OpenStack users if the OpenDev aspects of this team become independent with its own governance. We are more than happy to hear feedback so please read this thread and respond if the topic interests you. Note, I'm using this pointer thread in an effort to avoid cross posting and losing feedback. It would be great if people can respond to the original on openstack-infra at lists. [0] http://lists.openstack.org/pipermail/openstack-infra/2019-December/006537.html Thank you, Clark From anmar.salih1 at gmail.com Wed Dec 4 00:20:27 2019 From: anmar.salih1 at gmail.com (Anmar Salih) Date: Tue, 3 Dec 2019 19:20:27 -0500 Subject: Add functionality to Swift Message-ID: Hey, I have four containers in Openstack Swift. One of the containers called "Cont1", and I am using the web browser to upload and download objects to/from Cont1. I need to add a script that runs each time someone downloads an object from the container Cont1. Let's say I want Swift to execute a small script every time someone clicks on the Download button. Wondering if that possible? Where I should add the code. I cloned Swift Stein git on my local machine but not sure where to add my script. Take a look at the attached image, to the very right there is a download button for object1. So when someone clicks on that button, I want to execute a small script, then after executing the script, Swift will start to download object1. https://imgur.com/cCZIJqR Note: I need to add functionality to Swift, so when someone requests an object download, swift will take an action based on the container name. Best Regards. Anmar Salih -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Wed Dec 4 00:26:34 2019 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 03 Dec 2019 16:26:34 -0800 Subject: State of the Gate In-Reply-To: <20191203151202.jdhqh6z4irpepz4l@skaplons-mac> References: <20191203151202.jdhqh6z4irpepz4l@skaplons-mac> Message-ID: On Tue, Dec 3, 2019, at 7:12 AM, Slawek Kaplonski wrote: > Hi, > > On Tue, Dec 03, 2019 at 01:53:04PM +0000, Sean Mooney wrote: > > On Thu, Oct 31, 2019 at 3:29 PM Matt Riedemann wrote: > > > > > > Things are great! Surprise! I just wanted to let everyone know. Later! > > > . > > > . > > > . > > > . > > > . > > > Now that you've been tricked, on Halloween no less, I'm here to tell you > > > that things suck right now. This is your periodic digest of issues. Grab > > > some fun-sized candy bars and read on. > > > > > > I think right now we have three major issues. > > > > > > 1. http://status.openstack.org/elastic-recheck/index.html#1763070 > > > > > > This has resurfaced and I'm not sure why, nor do I think we ever had a > > > great handle on what is causing this or how to work around it so if > > > anyone has new ideas please chip in. > > the only theory i have on that is in the absense of any other indication > > we could be running out of entorpy in the vm which can lead to https > > connections failing. we might want to consider enabling the virtio > > random number generator in > > the gate vms. > > https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L54-L59 > > this need to be enabled in the flavours too but low entroy can cass > > ssl connection to fail > > https://major.io/2007/07/01/check-available-entropy-in-linux/ > > You may be right with lack of entropy is culprit in some issues. E.g. > here: > https://zuul.opendev.org/t/openstack/build/edcf837457a741abb752693723319b15/log/controller/logs/tempest_log.txt.gz#14652 > it took 120 seconds to Initialize random number generator on guest VM > and > because of that network wasn't configured properly and test failed. > https://review.opendev.org/#/c/697194/ is a first pass attempt at implementing the virtio rng device. How do we verify it is actually working? Clark From Albert.Braden at synopsys.com Wed Dec 4 00:51:31 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Wed, 4 Dec 2019 00:51:31 +0000 Subject: Rebooted VM disappears when HV lacks memory Message-ID: We are running Rocky and we have the allocation bug because we set cpu_allocation_ratio and ram_allocation_ratio to 1 on the controllers but not on the hypervisors, so the hypervisors are oversubscribed. When we try to reboot a VM when the HV is short memory, the VM is deleted from the HV. It still exists in Openstack but not on the HV. Is it possible to recover a VM from this state? Logs: http://dpaste.com/1E8VQ4V root at us01-p02-hv128:~# virsh list --all --uuid|grep 19ad13a4-f974-43ec-88a5-a6dbc13667a2 root at us01-p02-hv128:~# root at us01odc-p02-ctrl1:~# oss 19ad13a4-f974-43ec-88a5-a6dbc13667a2 +-------------------------------------+-----------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+-----------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | us01-p02-hv128 | | OS-EXT-SRV-ATTR:hypervisor_hostname | us01-p02-hv128.internal.synopsys.com | | OS-EXT-SRV-ATTR:instance_name | instance-00000c85 | | OS-EXT-STS:power_state | NOSTATE | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | stopped | | OS-SRV-USG:launched_at | 2019-11-11T20:11:30.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | vg-network=10.195.72.215 | | config_drive | | | created | 2019-11-11T20:10:40Z | | flavor | s1.16cx120g (3cfd8fbc-55da-4c1d-a717-ed86d3a0219f) | | hostId | fa4acc9f16b333592492f385f41bf15d593a269e72e03d24f752a611 | | id | 19ad13a4-f974-43ec-88a5-a6dbc13667a2 | | image | QSC-P-CentOS6.6-19P1-v4 (91497018-7b42-4448-aa2a-4c08fc9621ac) | | key_name | None | | name | vctrain2-qscp-cs66-849-104 | | project_id | 0b591e0864d04e6b8b6afa18a5a4a638 | | properties | BU='VC-Product', Farm='odc_vctrain', Owner='jvrp', Workspace='vctrain2-qscp-cs66' | | security_groups | name='default' | | status | SHUTOFF | | updated | 2019-12-03T23:57:49Z | | user_id | 0df75a295783f2f7c3843642f3a170f500bbeb4c69806bfd53e166b772617bf4 | | volumes_attached | | +-------------------------------------+-----------------------------------------------------------------------------------+ -------------- next part -------------- An HTML attachment was scrubbed... URL: From licanwei_cn at 163.com Wed Dec 4 02:43:46 2019 From: licanwei_cn at 163.com (licanwei) Date: Wed, 4 Dec 2019 10:43:46 +0800 (GMT+08:00) Subject: [Watcher] No IRC meeting today Message-ID: thanks, licanwei | | licanwei_cn | | 邮箱：licanwei_cn at 163.com | 签名由 网易邮箱大师 定制 -------------- next part -------------- An HTML attachment was scrubbed... URL: From eandersson at blizzard.com Wed Dec 4 05:42:26 2019 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Wed, 4 Dec 2019 05:42:26 +0000 Subject: [neutron] api performance at scale In-Reply-To: References: , Message-ID: Yea - I think those patches would help a lot. Especially the security group related change. Security groups for some reason are the most expensive call in Neutron for us. In our larger deployments the simplest security group list commands takes 60 seconds to perform. We had very similar issues with neutron-lbaas, but those calls has since been fixed. The large ops SIG is unfortunately 1AM (Pacific Time) over here. I can try to attend it, but wouldn't be easy. ________________________________ From: Matt Riedemann Sent: Tuesday, December 3, 2019 9:44 AM To: openstack-discuss at lists.openstack.org Subject: Re: [neutron] api performance at scale On 12/3/2019 11:24 AM, Erik Olof Gunnar Andersson wrote: > For us nova used to be the biggest concern, but a lot of work has been > done and nova now performers great. Instead we are having issues to get > Neutron to perform at scale. Obvious calls like security groups are > performing really poorly, and nova-compute defaults for refreshing the > network cache on computes causes massive issues with Neutron. I wonder how much of the performance hit is due to rootwrap usage in neutron (nova's conversion to privsep was completed in Train). Nova might be the bees knees, but I know there are things in nova we could do to be smarter about not hammering the neutron API as much, e.g.: https://urldefense.com/v3/__https://review.opendev.org/*/c/465792/__;Iw!2E0gRdhhnqPNNL0!0A97ZwnFJg3RpxtEwi5sVytDBtU_R8YdJ5P9h8OAUX7ciGEtHKVExLKEUIssCQ7svA$ - make bulk queries to neutron when refreshing the instance network info cache https://urldefense.com/v3/__https://review.opendev.org/*/q/I7de14456d04370c842b4c35597dca3a628a826a2__;Iw!2E0gRdhhnqPNNL0!0A97ZwnFJg3RpxtEwi5sVytDBtU_R8YdJ5P9h8OAUX7ciGEtHKVExLKEUIsamDve8Q$ - be smarter about filtering to avoid expensive joins https://urldefense.com/v3/__https://bugs.launchpad.net/nova/*bug/1567655__;Kw!2E0gRdhhnqPNNL0!0A97ZwnFJg3RpxtEwi5sVytDBtU_R8YdJ5P9h8OAUX7ciGEtHKVExLKEUIubIZxDAg$ - nova's internal network info cache only stores information about ports and their related networks/subnets/ips but the security group information related to the ports attached to a server is fetched directly anytime it's needed, including when listing servers with details. So if you're an admin listing all servers across all tenants, that could get pretty slow. I've long thought we should cache the security group information like we do for ports for read-only operations like GET /servers/detail but it's a non-trivial amount of work to make that happen and we'd definitely want benchmarks and such to justify the change. Note ttx has started a large ops SIG or whatever so this is probably something to discuss there: https://urldefense.com/v3/__https://wiki.openstack.org/wiki/Large_Scale_SIG__;!2E0gRdhhnqPNNL0!0A97ZwnFJg3RpxtEwi5sVytDBtU_R8YdJ5P9h8OAUX7ciGEtHKVExLKEUItHA86njA$ -- Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Wed Dec 4 10:25:16 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 4 Dec 2019 11:25:16 +0100 Subject: [neutron] api performance at scale In-Reply-To: References: Message-ID: <20191204102516.r7emk77mgyckbdfu@skaplons-mac> Hi, In the past we had biweekly meeting related to performance of Neutron. Now we included this as one of the points on Monday's Neutron team meeting. Please sync with Miguel Lavalle about that. He is leader of this performance subteam in Neutron and he is working on some profiling and identifying things which are slowing Neutron most. Speaking about security groups, is Your problem on API level or backend level? If it's on backend, what firewall driver are You using? Openvswitch or iptables_hybrid (or maybe some other)? Also, I know we have big performance issue if You are using security group with remote_security_group set in it (it's added by default to default SG). In such case if You have many ports using same SG, every time when You add new port to this SG, all other ports are updated by L2 agent and that is very slow if there is many ports there. So removing remote_security_group from rules and create rules based on remote CIDRs would help a lot with this. We were discussing this in Denver PTG but I don't think any bug on launchpad was reported for this. On Tue, Dec 03, 2019 at 05:24:54PM +0000, Erik Olof Gunnar Andersson wrote: > Is there a SIG or similar discussion on neutron performance at scale? > > For us nova used to be the biggest concern, but a lot of work has been done and nova now performers great. Instead we are having issues to get Neutron to perform at scale. Obvious calls like security groups are performing really poorly, and nova-compute defaults for refreshing the network cache on computes causes massive issues with Neutron. > > Best Regards, Erik Olof Gunnar Andersson > -- Slawek Kaplonski Senior software engineer Red Hat From stig.openstack at telfer.org Wed Dec 4 10:45:24 2019 From: stig.openstack at telfer.org (Stig Telfer) Date: Wed, 4 Dec 2019 10:45:24 +0000 Subject: [scientific-sig] IRC meeting today - unified limits for mixed baremetal/virt Message-ID: <69A5AE94-AAA0-4AA0-B871-732C39FE23DC@telfer.org> Hi All - We have a Scientific SIG IRC meeting at 1100 UTC in channel #openstack-meeting. Everyone is welcome. Today’s agenda is available here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_December_4th_2019 We’d like to discuss an update on unified limits, and how that will enable useful capabilities such as enforcing quotas across mixed environments - virtual, baremetal, GPUs, etc. Cheers, Stig (oneswig) From jean-philippe at evrard.me Wed Dec 4 11:18:43 2019 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Wed, 04 Dec 2019 12:18:43 +0100 Subject: [tc] Meetings: Agenda for december 2019 and new date for January 2020 Message-ID: <65e2c0548883582319e7ee374805b789b7ad7f48.camel@evrard.me> Hello everyone, Our next meeting is happening tomorrow 5 december, and the agenda is... still on the wiki. If you want it's also a little below [1]. I want to also highlight that the January 2020 meeting is happening on Thursday 16 instead of the first Thursday as usual. Regards, Jean-Philippe Evrard (evrardjp) [1]: Agenda for this meeting: Follow up on past action items: ricolin: (refresh) Follow up with SIG chairs about guidelines https://etherpad.openstack.org/p/SIGs-guideline ttx: (refresh) contact interested parties in a new 'large scale' sig (help with mnaser, jroll reaching out to verizon media) New initiatives and/or report on previous initiatives: gmann: report on the community goals for U and V, py2 drop, and goal select process schedule. anyone to help gmann? mnaser: sync up with swift team on python3 migration ricolin: report on multi-arch sig and other sigs ttx: Technical vision reflection update mnaser: Summary of the maintain issue with Telemetry mnaser: report oslo metrics project jungleboyj: update on the blog post about the analysis of the Foundation user survey mugsie: release naming progress mnaser: report on the infra liaison -- what's happening with the static hosting ttx: merge tc/uc update evrardjp: concepts repo addendums: gmann: define goal select process schedule -- is the current state clear? evrardjp: Patches pending mnaser: Summary of the stable branch policy discussion since the summit From naohiro.sameshima at global.ntt Wed Dec 4 11:37:35 2019 From: naohiro.sameshima at global.ntt (=?utf-8?B?TmFvaGlybyBTYW1lc2hpbWHvvIjprqvls7Yg55u05rSL77yJKEdyb3VwKQ==?=) Date: Wed, 4 Dec 2019 11:37:35 +0000 Subject: [glance] [glance_store] requirements-check job fails In-Reply-To: <49386241.WZNmq7ITLL@whitebase.usersys.redhat.com> References: <63CC5661-FB21-4969-9A24-9C0231625BF2@global.ntt> <49386241.WZNmq7ITLL@whitebase.usersys.redhat.com> Message-ID: Hi Luigi, > I commented on the review, and you are right; this is the missing bit: > Can you please rebase your patch on top of it^? Fortunately, all zuul jobs success! Thanks to your help. > Luigi This email and all contents are subject to the following disclaimer: https://hello.global.ntt/en-us/email-disclaimer -------------- next part -------------- An HTML attachment was scrubbed... URL: From smooney at redhat.com Wed Dec 4 12:29:43 2019 From: smooney at redhat.com (Sean Mooney) Date: Wed, 04 Dec 2019 12:29:43 +0000 Subject: Rebooted VM disappears when HV lacks memory In-Reply-To: References: Message-ID: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> On Wed, 2019-12-04 at 00:51 +0000, Albert Braden wrote: > We are running Rocky and we have the allocation bug because we set cpu_allocation_ratio and ram_allocation_ratio to 1 > on the controllers but not on the hypervisors, so the hypervisors are oversubscribed. When we try to reboot a VM when > the HV is short memory, the VM is deleted from the HV. It still exists in Openstack but not on the HV. Is it possible > to recover a VM from this state? so the vm is likely not deleted the harddisk and the libvirt domain will still be defiend. my guess is starting the vm cause the oom reaper to kill the qemu process when it was relaunched. if you are deploying with memory over subsription enabled you should configre enough swap space equal to total memory* ram_allocation_ratio. if you do that you you should be able to start the vm. now the other option is while the vm is stopped you can cold migrate it to another host and then start it there. > > Logs: http://dpaste.com/1E8VQ4V > > root at us01-p02-hv128:~# virsh list --all --uuid|grep 19ad13a4-f974-43ec-88a5-a6dbc13667a2 > root at us01-p02-hv128:~# > > root at us01odc-p02-ctrl1:~# oss 19ad13a4-f974-43ec-88a5-a6dbc13667a2 > +-------------------------------------+------------------------------------------------------------------------------- > ----+ > > Field | > > Value | > > +-------------------------------------+------------------------------------------------------------------------------- > ----+ > > OS-DCF:diskConfig | > > MANUAL | > > OS-EXT-AZ:availability_zone | > > nova | > > OS-EXT-SRV-ATTR:host | us01-p02- > > hv128 | > > OS-EXT-SRV-ATTR:hypervisor_hostname | us01-p02- > > hv128.internal.synopsys.com | > > OS-EXT-SRV-ATTR:instance_name | instance- > > 00000c85 | > > OS-EXT-STS:power_state | > > NOSTATE | > > OS-EXT-STS:task_state | > > None | > > OS-EXT-STS:vm_state | > > stopped | > > OS-SRV-USG:launched_at | 2019-11- > > 11T20:11:30.000000 | > > OS-SRV-USG:terminated_at | > > None | > > accessIPv4 | > > | > > accessIPv6 | > > | > > addresses | vg- > > network=10.195.72.215 | > > config_drive | > > | > > created | 2019-11- > > 11T20:10:40Z | > > flavor | s1.16cx120g (3cfd8fbc-55da-4c1d-a717- > > ed86d3a0219f) | > > hostId | > > fa4acc9f16b333592492f385f41bf15d593a269e72e03d24f752a611 | > > id | 19ad13a4-f974-43ec-88a5- > > a6dbc13667a2 | > > image | QSC-P-CentOS6.6-19P1-v4 (91497018-7b42-4448-aa2a- > > 4c08fc9621ac) | > > key_name | > > None | > > name | vctrain2-qscp-cs66-849- > > 104 | > > project_id | > > 0b591e0864d04e6b8b6afa18a5a4a638 | > > properties | BU='VC-Product', Farm='odc_vctrain', Owner='jvrp', Workspace='vctrain2-qscp- > > cs66' | > > security_groups | > > name='default' | > > status | > > SHUTOFF | > > updated | 2019-12- > > 03T23:57:49Z | > > user_id | > > 0df75a295783f2f7c3843642f3a170f500bbeb4c69806bfd53e166b772617bf4 | > > volumes_attached | > > | > > +-------------------------------------+------------------------------------------------------------------------------- > ----+ From jesse at odyssey4.me Wed Dec 4 12:43:04 2019 From: jesse at odyssey4.me (Jesse Pretorius) Date: Wed, 4 Dec 2019 12:43:04 +0000 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: References: Message-ID: <572abd10691af9fdf51b6da74bc65f12c638a24b.camel@odyssey4.me> On Tue, 2019-12-03 at 20:18 +0200, Sagi Shnaidman wrote: In the meeting today we agreed to meet every Thursday starting *this week* at 4.00 PM UTC on #openstack-sdks channel on Freenode. Hi everyone, I've pushed up https://review.opendev.org/697278 to reflect the meetin as discussed. I do wonder why the #openstack-sdks channel was chosen versus #openstack-ansible-sig. This may be a misunderstanding on my part though, so please let me know in review whether I have my wires crossed. I'm also not entirely certain about the frequency of the meetings, so please comment in review about that too. I noticed that https://governance.openstack.org/sigs/ points the Ansible SIG web page to <https://etherpad.openstack.org/p/ansible-sig%20instead> https://etherpad.openstack.org/p/ansible-sig instead of https://wiki.openstack.org/wiki/Ansible_SIG (which is outdated). I don't know if the wiki is the correct place for the SIG overview, but we should probably get that sorted out to ensure there's one reference location. I can't seem to find the source for https://governance.openstack.org/sigs/ - if someone can let me know where that is then I'll push up a patch to point it at the wiki. I'd be happy to update the wiki page too with the content from the etherpad, assuming that the content is deemed appropriate. Cheers, Jesse IRC: odyssey4me -------------- next part -------------- An HTML attachment was scrubbed... URL: From jean-philippe at evrard.me Wed Dec 4 14:05:29 2019 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Wed, 04 Dec 2019 15:05:29 +0100 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: <572abd10691af9fdf51b6da74bc65f12c638a24b.camel@odyssey4.me> References: <572abd10691af9fdf51b6da74bc65f12c638a24b.camel@odyssey4.me> Message-ID: <570129054cb9ff40a7ac151bc49553176423c9f5.camel@evrard.me> On Wed, 2019-12-04 at 12:43 +0000, Jesse Pretorius wrote: > > I can't seem to find the source for > https://governance.openstack.org/sigs/ - if someone can let me know > where that is then I'll push up a patch to point it at the wiki. I'd > be happy to update the wiki page too with the content from the > etherpad, assuming that the content is deemed appropriate. https://opendev.org/openstack/governance-sigs . Though if possible the information should be in governance. Thanks for updating things! JP -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Wed Dec 4 15:40:43 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 4 Dec 2019 10:40:43 -0500 Subject: [cinder][ops][extended-maintenance-sig][public-cloud-sig][enterprise-wg] Cinder to EOL some branches In-Reply-To: <6ca22c6a-c0c8-711a-91b5-111702cc2d06@est.tech> References: <3c5fd6e6-8ae4-d300-71a7-97b22431cb3b@gmail.com> <7e8b4158-89b2-6207-6f06-215782e0b126@est.tech> <70f9cd7b-535d-8f6b-dbe2-3578bd21c346@gmail.com> <6ca22c6a-c0c8-711a-91b5-111702cc2d06@est.tech> Message-ID: The tl;dr is that we won't EOL anything until Monday 9 December at the earliest; see below for details. On 12/3/19 10:00 AM, Elõd Illés wrote: > Hi Brian, > > Thanks in advance for proposing in Cinder meeting to wait to EOL Pike > and Ocata. > > So far I have been looking after stable/* branches. There seemed to be consensus at today's meeting that we could keep stable/ocata and stable/pike in EM mode, which would be consistent with other openstack projects. > > I had a quick check and it seems that the CI on driverfixes/* branches > are not in a good shape for some time. Those branches: > > - don't have periodic jobs, so we can't see how long they are in this > faulty state > - the test jobs are very limited > - as I see there aren't really arriving patches lately (actually one > arrived in newton ~2 months ago) > > I can try to search for solutions there to fix the gate, but maybe the > activity there is less than ocata and pike and if there aren't others > who are interested in driverfixes/* branches then maybe those reached > the state to move them to EOL. I agree with your analysis of the state of the driverfixes branches. Would you be interested in getting driverfixes/newton in working order, or are you only interested in us keeping stable/o and /p open? To be clear, the current idea is: - EOL driverfixes/mitaka and driverfixes/newton - leave stable/ocata and stable/pike in EM mode > > Thanks again, > > Előd > > > On 2019. 12. 02. 16:25, Brian Rosmaita wrote: >> On 11/27/19 12:02 PM, Elõd Illés wrote: >>> Hi, >>> >>> First of all, sorry for the late response. >>> >>> About EOLing Ocata and Pike: Extended Maintenance was formed just to >>> have a common place for interested parties, vendors, operators, to push >>> bugfixes to. Currently these branches are in a good shape, check / gate >>> jobs work and as far as I see there are a couple of backports, too (not >>> too many, though). So hopefully it's not a waste of efforts. Why don't >>> we keep them open as long as the CI works and there are patches? Of >>> course, whenever e.g. Ocata branch / CI becomes unmaintainable (zuul v3 >>> incompatibilities) or there aren't anyone who fixes the issues there, we >>> can put it to EOL then. >> >> Quick question: the driverfixes/mitaka and driverfixes/newton gates >> are broken now.  Do you have any interest in getting these working, or >> are you only interested in stable/ocata and stable/pike? >> >>> I write this, because my employer supports Extended Maintenance, and I >>> usually try to fix CI issues on stable branches and reviewing patches >>> there. >> >> Thanks for following this strategy, it's good for the community, and >> we appreciate it. >> >>> So maybe I can be some help here, too. Please consider leaving >>> branches in Extended Maintenance open as long as they are in a good >>> shape and there are bugfixes coming. >> >> This is good feedback.  As I said in the original email, we don't mind >> keeping these open as long as people are actually using them.  I'll >> put a proposal on this week's Cinder meeting agenda to wait to EOL >> ocata and pike until their gates break and no one steps up to fix >> them, instead of proactively EOL'ing them now. >> >> Do let me know your position on the driverfixes branches right away, >> though. >> >> >> thanks, >> brian >> >>> >>> Thanks, >>> >>> Előd >>> >>> >>> On 2019. 11. 25. 20:21, Brian Rosmaita wrote: >>>> This is a courtesy notice that having received no responses to my >>>> email of 28 October [0] proposing to EOL some currently open Cinder >>>> branches, and following the policy articulated in [1], at today's >>>> Virtual PTG meeting the Cinder project team has decided to put the >>>> following stable branches into the End of Life state: >>>>    driverfixes/mitaka >>>>    driverfixes/newton >>>>    stable/ocata >>>>    stable/pike >>>> >>>> I will submit the paperwork to get this process moving one week from >>>> today (2 December 2019). >>>> >>>> >>>> cheers, >>>> brian >>>> >>>> [0] >>>> http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010385.html >>>> >>>> [1] >>>> https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-life >>>> >>>> >>> >> >> > From skaplons at redhat.com Wed Dec 4 16:01:31 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Wed, 4 Dec 2019 17:01:31 +0100 Subject: [neutron] Proposing Jakub Libosvar as Neutron core reviewer In-Reply-To: References: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> Message-ID: Hi, There was no any votes against during last week. So welcome back in the Neutron core team Jakub :) > On 3 Dec 2019, at 09:17, Akihiro Motoki wrote: > > +1 from me. It would be nice to have him again. > > On Thu, Nov 28, 2019 at 6:01 PM Slawek Kaplonski wrote: >> >> Hi neutrinos, >> >> We already started process of migrating networking-ovn driver to be one of in-tree neutron drivers. Blueprint for that is [1]. >> As part of this process I today proposed to include networking-ovn-core group into neutron-core group. Mail about it can be found at [2]. >> One of persons in networking-ovn-group is Jakub Libosvar who was Neutron core for very long time in the past. He knows very well not only ovn related code but also have great knowledge about all Neutron code base. >> So I would like to propose to Jakub as Neutron core reviewer again as he will be back working on neutron again now, after ovn will be in-tree driver. >> What do You think about it? >> I will wait for Your opinions for 1 week from now. Thx for all Your comments about it. >> >> [1] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge >> [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011240.html >> >> — >> Slawek Kaplonski >> Senior software engineer >> Red Hat >> >> > — Slawek Kaplonski Senior software engineer Red Hat From manchandavishal143 at gmail.com Wed Dec 4 16:13:04 2019 From: manchandavishal143 at gmail.com (vishal manchanda) Date: Wed, 4 Dec 2019 21:43:04 +0530 Subject: [horizon] [plugins] Supported Django version for horizon in Ussuri Message-ID: Hello Everyone, As discussed at this weekly meeting[1] and Shanghai PTG[2], Horizon team is going to bump default version of Django to the next LTS release which is 2.2. Django 2.2 support is already added in Horizon and all its plugins [3]. Thanks to all for taking this at the priority and quick approvals on the patches. After Switching the default version of Django to 2.2(latest LTS), We will drop Django 1.11 in the next step(as its extended supports ending soon [4]). Also, Horizon usually supports LTS version of Django[5]. Please find below plan for coming weeks: Step 1: Switch the default version to Django 2.2 in upper-constraints till milestone-1. Additionally, Due to these changes, there might be a possibility of potential risks. Horizon and horizon plugins already support Django 2.2 as unit tests, but they are not tested in the real deployments with Django 2.2. So After switching the default version, if anything breaks we will keep working on fixing those issues. Step 2: Drop Django 1.11 during milestone-2. In case of any issue, please feel free to reach-out to Horizon channel #openstack-horizon. [1] http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-alt/%23openstack-meeting-alt.2019-11-27.log.html#t2019-11-27T15:27:23 [2] https://etherpad.openstack.org/p/horizon-u-ptg#45 [3] https://review.opendev.org/#/q/topic:django22+(status:open+OR+status:merged) [4] https://www.djangoproject.com/weblog/2015/jun/25/roadmap/ [5] https://docs.openstack.org/horizon/latest/contributor/policy.html#django-support [6] https://docs.djangoproject.com/en/2.2/releases/2.2/ cheers, Vishal Manchanda -------------- next part -------------- An HTML attachment was scrubbed... URL: From grant at civo.com Wed Dec 4 17:00:27 2019 From: grant at civo.com (Grant Morley) Date: Wed, 4 Dec 2019 17:00:27 +0000 Subject: DHCP timeout when creating instances for specific tenants Message-ID: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> Hi all, I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch. We are seeing the following on the instance logs: https://pastebin.com/hDstsd8G Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected. Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light? I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment. Many thanks, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From pabelanger at redhat.com Wed Dec 4 18:22:09 2019 From: pabelanger at redhat.com (Paul Belanger) Date: Wed, 4 Dec 2019 13:22:09 -0500 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: <570129054cb9ff40a7ac151bc49553176423c9f5.camel@evrard.me> References: <572abd10691af9fdf51b6da74bc65f12c638a24b.camel@odyssey4.me> <570129054cb9ff40a7ac151bc49553176423c9f5.camel@evrard.me> Message-ID: <20191204182209.GA590405@localhost.localdomain> On Wed, Dec 04, 2019 at 03:05:29PM +0100, Jean-Philippe Evrard wrote: > On Wed, 2019-12-04 at 12:43 +0000, Jesse Pretorius wrote: > > > > I can't seem to find the source for > > https://governance.openstack.org/sigs/ - if someone can let me know > > where that is then I'll push up a patch to point it at the wiki. I'd > > be happy to update the wiki page too with the content from the > > etherpad, assuming that the content is deemed appropriate. > > https://opendev.org/openstack/governance-sigs . Though if possible the > information should be in governance. > Thanks for updating things! > Could we also update http://eavesdrop.openstack.org/ to reference the ansible-sig meeting time? I'd like to add this info into calander. - Paul From eandersson at blizzard.com Wed Dec 4 18:46:43 2019 From: eandersson at blizzard.com (Erik Olof Gunnar Andersson) Date: Wed, 4 Dec 2019 18:46:43 +0000 Subject: [neutron] api performance at scale In-Reply-To: <20191204102516.r7emk77mgyckbdfu@skaplons-mac> References: <20191204102516.r7emk77mgyckbdfu@skaplons-mac> Message-ID: The problem is primarily on the API level. Just to give you an idea, we have ansible playbooks we use to manage security groups in one region that needs to be re-designed, but before we upgraded from Mitaka it took maybe 10 minutes to run, with Rocky it takes about 7 hours. I'll reach out to Miguel. Other issues has been - The DHCP agent performing very poorly many ports, but I think all of those issues has been fixed and backported to Rocky now. - Extreme memory usage. Each Neutron-server process uses up to 8.2GB memory during high load. - Lots of database locking causing issues, especially after an upgrade. One of the side effects has been that agent heartbeat queue to grow exponentially. It has recently been disused heavily in the neutron channel. Very similar to this bug https://bugs.launchpad.net/neutron/+bug/1853071 Thanks! Best Regards, Erik Olof Gunnar Andersson -----Original Message----- From: Slawek Kaplonski Sent: Wednesday, December 4, 2019 2:25 AM To: Erik Olof Gunnar Andersson Cc: openstack-discuss at lists.openstack.org Subject: Re: [neutron] api performance at scale Hi, In the past we had biweekly meeting related to performance of Neutron. Now we included this as one of the points on Monday's Neutron team meeting. Please sync with Miguel Lavalle about that. He is leader of this performance subteam in Neutron and he is working on some profiling and identifying things which are slowing Neutron most. Speaking about security groups, is Your problem on API level or backend level? If it's on backend, what firewall driver are You using? Openvswitch or iptables_hybrid (or maybe some other)? Also, I know we have big performance issue if You are using security group with remote_security_group set in it (it's added by default to default SG). In such case if You have many ports using same SG, every time when You add new port to this SG, all other ports are updated by L2 agent and that is very slow if there is many ports there. So removing remote_security_group from rules and create rules based on remote CIDRs would help a lot with this. We were discussing this in Denver PTG but I don't think any bug on launchpad was reported for this. On Tue, Dec 03, 2019 at 05:24:54PM +0000, Erik Olof Gunnar Andersson wrote: > Is there a SIG or similar discussion on neutron performance at scale? > > For us nova used to be the biggest concern, but a lot of work has been done and nova now performers great. Instead we are having issues to get Neutron to perform at scale. Obvious calls like security groups are performing really poorly, and nova-compute defaults for refreshing the network cache on computes causes massive issues with Neutron. > > Best Regards, Erik Olof Gunnar Andersson > -- Slawek Kaplonski Senior software engineer Red Hat From grant at civo.com Wed Dec 4 18:50:27 2019 From: grant at civo.com (Grant Morley) Date: Wed, 4 Dec 2019 18:50:27 +0000 Subject: DHCP timeout when creating instances for specific tenants Message-ID: <2036f345-1a47-9529-43ca-22d960a13d5a@civo.com> Hi all, I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch. We are seeing the following on the instance logs: https://pastebin.com/hDstsd8G Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected. Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light? I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment. Many thanks, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Wed Dec 4 18:56:20 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Wed, 4 Dec 2019 12:56:20 -0600 Subject: [nova][ironic] Changing an owner of a provisioned node Message-ID: <5049a2b0-4ab8-261f-b17b-e99288646f97@gmail.com> The 1.50 microversion [1] in the ironic API added the "owner" field to the node and I'm trying to use that to add some scheduler filtering in nova [2]. It's my understanding that the owner field on a provisioned node (instance_uuid on the node is set) can be changed, but I'm surprised that is allowed. Was that an oversight in developing that feature? The use case for the scheduler filter is baremetal nodes are owned by different (non-admin) projects in a deployment. When a non-admin project creates a baremetal server via nova, nova will filter out nodes that are not owned by the project (based on the node.owner field). If a node isn't owned by any project, only admins can use it. Admins also have access to all nodes regardless of owner. Given that, let's say user 1 from project A creates a server on nova X that is owned by project A (node.owner=A). Then the node.owner is changed to project B. What should happen? Should nova detect that ownership change and stop the node or something? Note that with other resources that can transfer ownership, like volumes, that can only be done when they aren't in use. So why don't we have the same rules for nodes? Assuming we do want to enforce this in the API (a 409 response when trying to change the owner on a provisioned node), how would that be done given this is a problem since 1.50 which was added in Stein? Would a policy rule be added to ironic to determine if someone can change the owner on a provisioned node and if so, what would be the default rule? The same as "baremetal:node:update" (rule:is_admin)? [1] https://docs.openstack.org/ironic/latest/contributor/webapi-version-history.html#id7 [2] https://blueprints.launchpad.net/nova/+spec/ironic-tenant-filter -- Thanks, Matt From rosmaita.fossdev at gmail.com Wed Dec 4 19:07:20 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 4 Dec 2019 14:07:20 -0500 Subject: [cinder] Ussuri Shanghai and Virtual PTG update Message-ID: The PTG summary document has been updated with a quick synopsis of what we discussed at the Virtual PTG last week: https://wiki.openstack.org/wiki/CinderUssuriPTGSummary All sessions (both Shanghai and Virtual) were recorded. The above page contains links to the recordings, which you can also find on the Cinder YouTube channel. Please look through the summary document to remind yourself about any actions you volunteered for. Additionally, there's an etherpad of unclaimed action items if you're looking for something to do: https://etherpad.openstack.org/p/cinder-ussuri-ptg-actions Thanks to everyone who participated in the PTG events. cheers, brian From gouthampravi at gmail.com Wed Dec 4 19:07:11 2019 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Wed, 4 Dec 2019 11:07:11 -0800 Subject: [manila] Ussuri Virtual PTG schedule and plans In-Reply-To: References: Message-ID: Greetings! Huge thanks to those of you that participated in our online poll [1] to decide a date/time for our virtual meeting. After confirming availability, conflicts, accessibility between the interested parties, we've decided to host the meeting on BlueJeans on 11th December 2019 between 13:00 UTC and 20:00 UTC. We promised to be flexible with the schedule due to the vast time zones we cover within our contributors. So please let me know via irc or via the etherpad at [2] if you have any scheduling conflicts we must consider. Hope to see you all soon! Regards, Goutham (gouthamr) [1] https://xoyondo.com/dp/qQbsoHtNhi4DCki Manila Virtual PTG Poll [2] https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual [3] https://bit.ly/33NuIBl -- Add event to your Google Calendar On Wed, Oct 23, 2019 at 7:08 PM Goutham Pacha Ravi wrote: > > Hey Zorillas* and interested Stackers, > > (* manila contributors in the wild) > > A part of our community will be attending the Summit+Forum+PTG in Shanghai in a couple of weeks. However, we haven't figured out a way to enable remote participation during our PTG (Wednesday, 6th Nov 2019 between 9:00 AM and 4:30 PM) there. So, as discussed during our weekly meeting, I'd like to take a poll of when you would like to gather for a remote PTG that would supplement the gathering in Shanghai. > > Please take this poll and indicate your availability: https://xoyondo.com/dp/qQbsoHtNhi4DCki > > The prospective date for this meeting is between 2nd Dec and 13th Dec (Ussuri-1 Milestone is on 12th Dec). The meeting time will be 15:00 UTC to 22:00 UTC. This time may not work for our contributors in the Far East (APAC), but I'm hoping we'll address them primarily at the Shanghai get together, and relay their thoughts here. We will also be recording the entire meeting and will share notes on these lists. > > Thanks, > Goutham > > From dtantsur at redhat.com Wed Dec 4 19:49:11 2019 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Wed, 4 Dec 2019 20:49:11 +0100 Subject: [nova][ironic] Changing an owner of a provisioned node In-Reply-To: <5049a2b0-4ab8-261f-b17b-e99288646f97@gmail.com> References: <5049a2b0-4ab8-261f-b17b-e99288646f97@gmail.com> Message-ID: Hi, On Wed, Dec 4, 2019 at 7:58 PM Matt Riedemann wrote: > The 1.50 microversion [1] in the ironic API added the "owner" field to > the node and I'm trying to use that to add some scheduler filtering in > nova [2]. It's my understanding that the owner field on a provisioned > node (instance_uuid on the node is set) can be changed, but I'm > surprised that is allowed. Was that an oversight in developing that > feature? > I think so.. we have also uncovered it while discussing https://review.opendev.org/#/c/696707/ which can make this issue worse. > > The use case for the scheduler filter is baremetal nodes are owned by > different (non-admin) projects in a deployment. When a non-admin project > creates a baremetal server via nova, nova will filter out nodes that are > not owned by the project (based on the node.owner field). If a node > isn't owned by any project, only admins can use it. Admins also have > access to all nodes regardless of owner. > > Given that, let's say user 1 from project A creates a server on nova X > that is owned by project A (node.owner=A). Then the node.owner is > changed to project B. What should happen? Should nova detect that > ownership change and stop the node or something? > > Note that with other resources that can transfer ownership, like > volumes, that can only be done when they aren't in use. So why don't we > have the same rules for nodes? > > Assuming we do want to enforce this in the API (a 409 response when > trying to change the owner on a provisioned node), how would that be > done given this is a problem since 1.50 which was added in Stein? Would > a policy rule be added to ironic to determine if someone can change the > owner on a provisioned node and if so, what would be the default rule? > The same as "baremetal:node:update" (rule:is_admin)? > I like the idea of something like baremetal:node:update_owner defaulting to rule:is_admin (NOT to baremetal:node:update). I can work on a patch tomorrow if nobody beats me to it. Dmitry > > [1] > > https://docs.openstack.org/ironic/latest/contributor/webapi-version-history.html#id7 > [2] https://blueprints.launchpad.net/nova/+spec/ironic-tenant-filter > > -- > > Thanks, > > Matt > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From baha at linux.vnet.ibm.com Wed Dec 4 19:56:30 2019 From: baha at linux.vnet.ibm.com (Adam Kimball) Date: Wed, 4 Dec 2019 14:56:30 -0500 Subject: [tripleo][ppc64le] Unexpected Return Code from TripleO Container Build Playbooks on ppc64le Message-ID: <51aeb4cc-5490-a2b2-c4e2-58ef35c92d35@linux.vnet.ibm.com> Hello all, Mike Turek and I have been working to get a ppc64le job to build tripleo containers for quite some time now. A few weeks back, we stumbled into a strange issue where the containers build job was issuing a return code 1, despite all containers successfully building. This has largely blocked our ability to push containers up to docker hub or report success through the delorean API. We've been pushing a number of patches to expand the logging of both our job, and the tripleo playbooks themselves, to get a better idea of what's going on. The most recent was a patch to show which containers have python futures that have been throwing exceptions, and why. [1] The end result of this seems to be that a number of jobs are reporting as incomplete. An example of this can be seen at timestamp 20:16:07 of an example build log. [2] However, upon checking the list of successfully built containers [3], or the RDO registry itself [4], one can see that the containers producing job not complete errors have actually built, and are being uploaded. The error log generated by the tripleo playbooks is also empty. [5] At this point, we're wondering what the path forward is. It seems like the issue stems from some unintended behavior in the tripleo playbooks themselves, not anything from our job. We're trying to figure out if this behavior is something that should be preventing us from reporting successful builds, and if so, how it can be fixed. Thanks, Adam Kimball [1] - https://review.opendev.org/#/c/695723/ [2] - https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/1803/logs/logs/build.log [3] - https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/1803/logs/logs/containers-successfully-built.log [4] - https://console.registry.rdoproject.org/registry [5] - https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/1803/logs/logs/containers-build-errors.log From tzumainn at redhat.com Wed Dec 4 19:56:37 2019 From: tzumainn at redhat.com (Tzu-Mainn Chen) Date: Wed, 4 Dec 2019 14:56:37 -0500 Subject: [nova][ironic] Changing an owner of a provisioned node In-Reply-To: References: <5049a2b0-4ab8-261f-b17b-e99288646f97@gmail.com> Message-ID: On Wed, Dec 4, 2019 at 2:55 PM Dmitry Tantsur wrote: > Hi, > > On Wed, Dec 4, 2019 at 7:58 PM Matt Riedemann wrote: > >> The 1.50 microversion [1] in the ironic API added the "owner" field to >> the node and I'm trying to use that to add some scheduler filtering in >> nova [2]. It's my understanding that the owner field on a provisioned >> node (instance_uuid on the node is set) can be changed, but I'm >> surprised that is allowed. Was that an oversight in developing that >> feature? >> > > I think so.. we have also uncovered it while discussing > https://review.opendev.org/#/c/696707/ which can make this issue worse. > > >> >> The use case for the scheduler filter is baremetal nodes are owned by >> different (non-admin) projects in a deployment. When a non-admin project >> creates a baremetal server via nova, nova will filter out nodes that are >> not owned by the project (based on the node.owner field). If a node >> isn't owned by any project, only admins can use it. Admins also have >> access to all nodes regardless of owner. >> >> Given that, let's say user 1 from project A creates a server on nova X >> that is owned by project A (node.owner=A). Then the node.owner is >> changed to project B. What should happen? Should nova detect that >> ownership change and stop the node or something? >> >> Note that with other resources that can transfer ownership, like >> volumes, that can only be done when they aren't in use. So why don't we >> have the same rules for nodes? >> >> Assuming we do want to enforce this in the API (a 409 response when >> trying to change the owner on a provisioned node), how would that be >> done given this is a problem since 1.50 which was added in Stein? Would >> a policy rule be added to ironic to determine if someone can change the >> owner on a provisioned node and if so, what would be the default rule? >> The same as "baremetal:node:update" (rule:is_admin)? >> > > I like the idea of something like baremetal:node:update_owner defaulting > to rule:is_admin (NOT to baremetal:node:update). I can work on a patch > tomorrow if nobody beats me to it. > I'm happy to take this on. Thanks! Mainn > Dmitry > > >> >> [1] >> >> https://docs.openstack.org/ironic/latest/contributor/webapi-version-history.html#id7 >> [2] https://blueprints.launchpad.net/nova/+spec/ironic-tenant-filter >> >> -- >> >> Thanks, >> >> Matt >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz at redhat.com Wed Dec 4 20:13:47 2019 From: aschultz at redhat.com (Alex Schultz) Date: Wed, 4 Dec 2019 13:13:47 -0700 Subject: [tripleo][ppc64le] Unexpected Return Code from TripleO Container Build Playbooks on ppc64le In-Reply-To: <51aeb4cc-5490-a2b2-c4e2-58ef35c92d35@linux.vnet.ibm.com> References: <51aeb4cc-5490-a2b2-c4e2-58ef35c92d35@linux.vnet.ibm.com> Message-ID: On Wed, Dec 4, 2019 at 1:02 PM Adam Kimball wrote: > Hello all, > > Mike Turek and I have been working to get a ppc64le job to build tripleo > containers for quite some time now. A few weeks back, we stumbled into a > strange issue where the containers build job was issuing a return code > 1, despite all containers successfully building. This has largely > blocked our ability to push containers up to docker hub or report > success through the delorean API. > > We've been pushing a number of patches to expand the logging of both our > job, and the tripleo playbooks themselves, to get a better idea of > what's going on. The most recent was a patch to show which containers > have python futures that have been throwing exceptions, and why. [1] > > The end result of this seems to be that a number of jobs are reporting > as incomplete. An example of this can be seen at timestamp 20:16:07 of > an example build log. [2] > > However, upon checking the list of successfully built containers [3], or > the RDO registry itself [4], one can see that the containers producing > job not complete errors have actually built, and are being uploaded. The > error log generated by the tripleo playbooks is also empty. [5] > > At this point, we're wondering what the path forward is. It seems like > the issue stems from some unintended behavior in the tripleo playbooks > themselves, not anything from our job. We're trying to figure out if > this behavior is something that should be preventing us from reporting > successful builds, and if so, how it can be fixed. > > Thanks, > Adam Kimball > > > [1] - https://review.opendev.org/#/c/695723/ > [2] - > > https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/1803/logs/logs/build.log In reviewing this, it looks like it's failing on the push to the rdo registry. Would it be possible to keep a node and manually see what a push command is doing? For example, sudo buildah push --tls-verify=False trunk.registry.rdoproject.org/tripleomaster/centos-binary-mistral-executor:36a84820e51bad57c6bbb92429f3afb3d9da29c2_6e3b098e_ppc64le docker://trunk.registry.rdoproject.org/tripleomaster/centos-binary-mistral-executor:36a84820e51bad57c6bbb92429f3afb3d9da29c2_6e3b098e_ppc64le We output the commands that are being run so I'm wondering if there's some form of output that's getting eaten. Thanks, -Alex > [3] - > > https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/1803/logs/logs/containers-successfully-built.log > [4] - https://console.registry.rdoproject.org/registry > [5] - > > https://centos.logs.rdoproject.org/tripleo-upstream-containers-build-master-ppc64le/1803/logs/logs/containers-build-errors.log > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mthode at mthode.org Wed Dec 4 22:14:36 2019 From: mthode at mthode.org (Matthew Thode) Date: Wed, 4 Dec 2019 16:14:36 -0600 Subject: [all][requirements][drop-py27-support] Requirements is dropping cross tests with projects that will be dropping support for python2.7 this cycle Message-ID: <20191204221436.onvqxqx2mt2g32qs@mthode.org> Hi all, The requirements project is dropping python2.7 cross tests we do for those projects that are also dropping python2.7 support this cycle. We are doing this as wel do not know when projects will drop support (neutron seems to have done so, cinder is soon as well). This was originally scheduled for later but need to be pushed up as we missed the dependency requirements has on those projects for cross jobs. Here's the review if you are curious :D https://review.opendev.org/697377 -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From sean.mcginnis at gmx.com Wed Dec 4 22:24:24 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Wed, 4 Dec 2019 16:24:24 -0600 Subject: [all][requirements][drop-py27-support] Requirements is dropping cross tests with projects that will be dropping support for python2.7 this cycle In-Reply-To: <20191204221436.onvqxqx2mt2g32qs@mthode.org> References: <20191204221436.onvqxqx2mt2g32qs@mthode.org> Message-ID: <20191204222424.GA527721@sm-workstation> > > The requirements project is dropping python2.7 cross tests we do for > those projects that are also dropping python2.7 support this cycle. We > are doing this as wel do not know when projects will drop support > (neutron seems to have done so, cinder is soon as well). This was > originally scheduled for later but need to be pushed up as we missed the > dependency requirements has on those projects for cross jobs. > Just to expand on this a little from the conversation we had in the requirements channel... The schedule currently has libraries keeping py2 support until milestone 2, which isn't until mid-February. So we're in a tough spot here as far as requirements gating. We would like to continue testing lib updates with py27 to make sure nothing breaks. But at the same time, services are no longer gating on py27, so incompatible changes are creeping in and therefore we are not able to run the full jobs to exercise the updated lib code under py27. Hopefully this is a non-issue and most libs have their own testing under py27 that can give a reasonable assurance that things aren't completely broken. It's a temporary problem, and I'm sure February will be here before we know it and the issue will completely go away. In the meantime, please pay attention to lib changes to watch for issues. We may end up getting to a point where we need to accelerate the timeline (and probably deal with some broken gates), but I think we should be able to work through most issues fairly quickly. Thanks, Sean From Albert.Braden at synopsys.com Wed Dec 4 22:25:23 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Wed, 4 Dec 2019 22:25:23 +0000 Subject: Rebooted VM disappears when HV lacks memory In-Reply-To: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> References: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> Message-ID: Thank you Sean. We've scheduled a change to fix this in our prod cluster, but we have a mess to clean up in the meantime; this is very helpful! -----Original Message----- From: Sean Mooney Sent: Wednesday, December 4, 2019 4:30 AM To: Albert Braden ; openstack-discuss at lists.openstack.org Subject: Re: Rebooted VM disappears when HV lacks memory On Wed, 2019-12-04 at 00:51 +0000, Albert Braden wrote: > We are running Rocky and we have the allocation bug because we set cpu_allocation_ratio and ram_allocation_ratio to 1 > on the controllers but not on the hypervisors, so the hypervisors are oversubscribed. When we try to reboot a VM when > the HV is short memory, the VM is deleted from the HV. It still exists in Openstack but not on the HV. Is it possible > to recover a VM from this state? so the vm is likely not deleted the harddisk and the libvirt domain will still be defiend. my guess is starting the vm cause the oom reaper to kill the qemu process when it was relaunched. if you are deploying with memory over subsription enabled you should configre enough swap space equal to total memory* ram_allocation_ratio. if you do that you you should be able to start the vm. now the other option is while the vm is stopped you can cold migrate it to another host and then start it there. > > Logs: https://urldefense.proofpoint.com/v2/url?u=http-3A__dpaste.com_1E8VQ4V&d=DwICaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=7fgYsjA7VtYl3Zo7ZOo08R0HaR3R7RhHMwgCUqpKA0I&s=Frt7z6L65bkZegYdEev8ZM6gYH1UblhS0dziKFbB1zY&e= > > root at us01-p02-hv128:~# virsh list --all --uuid|grep 19ad13a4-f974-43ec-88a5-a6dbc13667a2 > root at us01-p02-hv128:~# > > root at us01odc-p02-ctrl1:~# oss 19ad13a4-f974-43ec-88a5-a6dbc13667a2 > +-------------------------------------+------------------------------------------------------------------------------- > ----+ > > Field | > > Value | > > +-------------------------------------+------------------------------------------------------------------------------- > ----+ > > OS-DCF:diskConfig | > > MANUAL | > > OS-EXT-AZ:availability_zone | > > nova | > > OS-EXT-SRV-ATTR:host | us01-p02- > > hv128 | > > OS-EXT-SRV-ATTR:hypervisor_hostname | us01-p02- > > hv128.internal.synopsys.com | > > OS-EXT-SRV-ATTR:instance_name | instance- > > 00000c85 | > > OS-EXT-STS:power_state | > > NOSTATE | > > OS-EXT-STS:task_state | > > None | > > OS-EXT-STS:vm_state | > > stopped | > > OS-SRV-USG:launched_at | 2019-11- > > 11T20:11:30.000000 | > > OS-SRV-USG:terminated_at | > > None | > > accessIPv4 | > > | > > accessIPv6 | > > | > > addresses | vg- > > network=10.195.72.215 | > > config_drive | > > | > > created | 2019-11- > > 11T20:10:40Z | > > flavor | s1.16cx120g (3cfd8fbc-55da-4c1d-a717- > > ed86d3a0219f) | > > hostId | > > fa4acc9f16b333592492f385f41bf15d593a269e72e03d24f752a611 | > > id | 19ad13a4-f974-43ec-88a5- > > a6dbc13667a2 | > > image | QSC-P-CentOS6.6-19P1-v4 (91497018-7b42-4448-aa2a- > > 4c08fc9621ac) | > > key_name | > > None | > > name | vctrain2-qscp-cs66-849- > > 104 | > > project_id | > > 0b591e0864d04e6b8b6afa18a5a4a638 | > > properties | BU='VC-Product', Farm='odc_vctrain', Owner='jvrp', Workspace='vctrain2-qscp- > > cs66' | > > security_groups | > > name='default' | > > status | > > SHUTOFF | > > updated | 2019-12- > > 03T23:57:49Z | > > user_id | > > 0df75a295783f2f7c3843642f3a170f500bbeb4c69806bfd53e166b772617bf4 | > > volumes_attached | > > | > > +-------------------------------------+------------------------------------------------------------------------------- > ----+ From emiller at genesishosting.com Wed Dec 4 22:47:54 2019 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 4 Dec 2019 16:47:54 -0600 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> Hi Grant, Are you sure this is a DHCP timeout and not a DNS resolution issue? I ask because we have seen a strange DNS issue occur that can cause something similar. Are the VMs being assigned an IP after they finally boot? Eric K. Miller Genesis Hosting Solutions, LLC Try our Genesis Public Cloud - powered by OpenStack! https://genesishosting.com/ From: Grant Morley [mailto:grant at civo.com] Sent: Wednesday, December 04, 2019 11:00 AM To: openstack-operators at lists.openstack.org Cc: Ian Banks Subject: DHCP timeout when creating instances for specific tenants Hi all, I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch. We are seeing the following on the instance logs: https://pastebin.com/hDstsd8G Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected. Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light? I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment. Many thanks, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Wed Dec 4 23:01:00 2019 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 4 Dec 2019 15:01:00 -0800 Subject: [TC] 'V' Technical Elections Message-ID: Hello! /me takes TC hat off and puts election official hat on Having run our handy dandy tool to project dates[1] for the elections, we have some decisions to make again. At this point we have threeish options: 1. Run the elections with a week in between with the TC election happening BEFORE the PTL election like the script generated. The TC election would run March 10 to March 31 and the PTL Election would run April 7 to April 21. The primary concern here is that it might lead to some election burnout. 2. Run the elections back to back with the PTL election first (closer to the historic norm). This would be tweaking the generated dates by pulling the PTL election three weeks up and pushing back the TC dates by two weeks. The PTL election would run March 3 to March 17 and the PTL Election would run March 17 to April 7. Both elections would be concluded by milestone 3. Similar concerns with this option to the first with election burnout. 3. Run the elections in parallel as a combined election, similar to how we did for the Train elections. This is much easier for election officials because we only need to generate one electorate. The two elections would start March 10 and be done by March 31 following the TC election model with a nominations period, a campaigning period (optional for PTL elections), and then a polling period (assuming we need a runoff). Other info to keep in mind when formulating your opinions: - Extra ATC deadline currently is March 23-27 which will likely need to get bumped earlier for any of these options - RC1 is the week of April 20-24 - Milestone 3 is April 9 - Release schedule for reference[2] - For TC member terms, those closing in on their ~1 year terms would be closer to 13 months since the train elections concluded March 5th of last year -Kendall (diablo_rojo) & the Election Officials [1]http://paste.openstack.org/show/787114/ [2]https://releases.openstack.org/ussuri/schedule.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Wed Dec 4 23:39:32 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 4 Dec 2019 23:39:32 +0000 Subject: [TC] Election terms (was: V' Technical Elections) In-Reply-To: References: Message-ID: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> On 2019-12-04 15:01:00 -0800 (-0800), Kendall Nelson wrote: [...] > For TC member terms, those closing in on their ~1 year terms would > be closer to 13 months since the train elections concluded March > 5th of last year [...] Just to clarify, this is what our (recently updated) Technical Committee Member Policy appendix to the OSF Bylaws says about TC term durations: 2.a. [...] Each Technical Committee member shall hold the seat for a term not to exceed sixteen months, but may be re-elected to the Technical Committee. After January 1, 2019, the term for the members of the Technical Committee shall be approved by a majority of the Technical Committee (“Term”) and shall be published publicly before each Technical Committee election; if no such Term is published the Term will be twelve calendar months. [...] https://www.openstack.org/legal/technical-committee-member-policy/ So those members elected in March can't actually serve past the anniversary of their elections *unless* prior to the election in which they were elected the seated TC had declared they would serve a longer term. The bylaws allow the TC to set a longer term length, but they must do so before the members who will serve that term are elected. So terms for half the TC members will be expiring at the start of March, quite possibly weeks before another TC election is held to fill their seats. This is what the TC charter seems to say: If the vacancy opens less than four weeks before the candidacy period for the next scheduled TC election begins, and the seat vacated would have been contested in the upcoming election anyway, then the seat will remain open until the election and filled by the normal election process. https://governance.openstack.org/tc/reference/charter.html So with a strict reading of the bylaws and charter wording, the TC will spend a few weeks with only 6 members until the suggested election date. There are a few options: 1. Push the TC election back further. This probably similarly implies backing the extra-ATC deadline up even more. 2. Amend the charter to allow the TC to reappoint the holders of the expired seats until the next election (I can't quite tell if this would violate the term limit wording in the bylaws, but it needs discussing). 3. Probably only a solution for 2021 and later elections, but start declaring now that TC seat terms are something like "14 months or until the date of the next election" so that the TC doesn't need to predict election timing more than a year in advance (OSF bylaws authorize us to make it up to 16 months for that matter). -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From grant at civo.com Wed Dec 4 23:40:28 2019 From: grant at civo.com (Grant Morley) Date: Wed, 4 Dec 2019 23:40:28 +0000 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> Message-ID: <7fe77627-25c8-2b33-b577-757058763826@civo.com> Hi Eric, Thanks for getting back to me. I am fairly sure it is a DHCP error. The instances are getting an IP when they eventually boot, it is just taking a long time for them to bring up networking. The strange thing is, it only seems to be new tenants. All existing tenants are absolutely fine. I can check DNS as well just to be on the safe side, however I wasn't seeing any errors in the Nova or Neutron logs when the instance(s) were being created. Regards, On 04/12/2019 22:47, Eric K. Miller wrote: > > Hi Grant, > > Are you sure this is a DHCP timeout and not a DNS resolution issue?  I > ask because we have seen a strange DNS issue occur that can cause > something similar. > > Are the VMs being assigned an IP after they finally boot? > > Eric K. Miller > > Genesis Hosting Solutions, LLC > > Try our Genesis Public Cloud - powered by OpenStack!eut > > https://genesishosting.com/ > > *From:*Grant Morley [mailto:grant at civo.com] > *Sent:* Wednesday, December 04, 2019 11:00 AM > *To:* openstack-operators at lists.openstack.org > *Cc:* Ian Banks > *Subject:* DHCP timeout when creating instances for specific tenants > > Hi all, > > I wonder if anyone can help shed any light on an odd issue we are > seeing with only a couple of specific tenants. Basically if they > launch an instance they are taking about 5 minutes to launch rather > than our usual 30 second or so launch. > > We are seeing the following on the instance logs: > > https://pastebin.com/hDstsd8G > > Weirdly it only seems to be happening for 1 or 2 new tenants. I have > tested this on our personal account and a few other customers have > tested and their instances launch really quickly as expected. > > Is there anything specific during the tenant creation that can cause > this issue? Or are there any logs in nova / neutron I should be > looking out for that might shed some light? > > I haven't seen anything that is obvious. Any help would be much > appreciated as we are a little stumped at the moment. > > Many thanks, > > -- > > Grant Morley > > Cloud Lead, Civo Ltd > > www.civo.com | Signup for an account! > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emiller at genesishosting.com Wed Dec 4 23:46:07 2019 From: emiller at genesishosting.com (Eric K. Miller) Date: Wed, 4 Dec 2019 17:46:07 -0600 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: <7fe77627-25c8-2b33-b577-757058763826@civo.com> References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> <7fe77627-25c8-2b33-b577-757058763826@civo.com> Message-ID: <046E9C0290DD9149B106B72FC9156BEA04771513@gmsxchsvr01.thecreation.com> Hi Grant, I didn't see any DNS errors either. The solution was to explicitly configure the dns servers in the subnet. Are you doing this already? Or are you relying on the dnsmasq processes created for the router to respond to DNS queries (and forward them respectively)? Eric From: Grant Morley [mailto:grant at civo.com] Sent: Wednesday, December 04, 2019 5:40 PM To: Eric K. Miller; openstack-operators at lists.openstack.org Cc: Ian Banks Subject: Re: DHCP timeout when creating instances for specific tenants Hi Eric, Thanks for getting back to me. I am fairly sure it is a DHCP error. The instances are getting an IP when they eventually boot, it is just taking a long time for them to bring up networking. The strange thing is, it only seems to be new tenants. All existing tenants are absolutely fine. I can check DNS as well just to be on the safe side, however I wasn't seeing any errors in the Nova or Neutron logs when the instance(s) were being created. Regards, On 04/12/2019 22:47, Eric K. Miller wrote: Hi Grant, Are you sure this is a DHCP timeout and not a DNS resolution issue? I ask because we have seen a strange DNS issue occur that can cause something similar. Are the VMs being assigned an IP after they finally boot? Eric K. Miller Genesis Hosting Solutions, LLC Try our Genesis Public Cloud - powered by OpenStack!eut https://genesishosting.com/ From: Grant Morley [mailto:grant at civo.com] Sent: Wednesday, December 04, 2019 11:00 AM To: openstack-operators at lists.openstack.org Cc: Ian Banks Subject: DHCP timeout when creating instances for specific tenants Hi all, I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch. We are seeing the following on the instance logs: https://pastebin.com/hDstsd8G Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected. Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light? I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment. Many thanks, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ~WRD374.jpg Type: image/jpeg Size: 823 bytes Desc: ~WRD374.jpg URL: From Cory at hawkless.id.au Wed Dec 4 23:49:35 2019 From: Cory at hawkless.id.au (Cory Hawkless) Date: Wed, 4 Dec 2019 23:49:35 +0000 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: <7fe77627-25c8-2b33-b577-757058763826@civo.com> References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> <7fe77627-25c8-2b33-b577-757058763826@civo.com> Message-ID: <6899b9085b784618bc49cdeda0bc3103@hawkless.id.au> Are they failing to contact the metadata service and hanging during the boot process while they try and receive metadata? From the VM can you hit http://169.254.169.254 – That’s the default IP of the metadata server, it should respond with a basic page showing some date based subdirectories If it doesn’t respond you can start following the metadata service path instead of DHCP Given that the machines come up with an IP eventually leads me to think the DHCP service is actually working ok. From: Grant Morley [mailto:grant at civo.com] Sent: Thursday, 5 December 2019 10:10 AM To: Eric K. Miller ; openstack-operators at lists.openstack.org Cc: Ian Banks Subject: Re: DHCP timeout when creating instances for specific tenants Hi Eric, Thanks for getting back to me. I am fairly sure it is a DHCP error. The instances are getting an IP when they eventually boot, it is just taking a long time for them to bring up networking. The strange thing is, it only seems to be new tenants. All existing tenants are absolutely fine. I can check DNS as well just to be on the safe side, however I wasn't seeing any errors in the Nova or Neutron logs when the instance(s) were being created. Regards, On 04/12/2019 22:47, Eric K. Miller wrote: Hi Grant, Are you sure this is a DHCP timeout and not a DNS resolution issue? I ask because we have seen a strange DNS issue occur that can cause something similar. Are the VMs being assigned an IP after they finally boot? Eric K. Miller Genesis Hosting Solutions, LLC Try our Genesis Public Cloud - powered by OpenStack!eut https://genesishosting.com/ From: Grant Morley [mailto:grant at civo.com] Sent: Wednesday, December 04, 2019 11:00 AM To: openstack-operators at lists.openstack.org Cc: Ian Banks Subject: DHCP timeout when creating instances for specific tenants Hi all, I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch. We are seeing the following on the instance logs: https://pastebin.com/hDstsd8G Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected. Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light? I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment. Many thanks, -- [https://www.civo.com/images/email-logo.jpg] Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Wed Dec 4 23:50:45 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 4 Dec 2019 23:50:45 +0000 Subject: [TC] Right-sizing the Technical Committee (was: 'V' Technical Elections) In-Reply-To: References: Message-ID: <20191204235045.dx3the2pjz6c26jw@yuggoth.org> Tangential to election scheduling but still on the topic of election planning, last cycle a bunch of folks jumped on the "let's shrink the TC!" bandwagon *while* the election process was already underway. That was of course not an appropriate time to talk about changes to election parameters. But now(ish) *is* the right time. So to reopen that discussion we previously put a pin in, how many TC seats should we fill in the coming election, and how many should we delete? There were a few different suggestions, some following a less aggressive timeline than others. We would normally have 7 seats up for grabs in the coming round... do we reduce it to 6 (and work with an even-number-sized TC), or just 5 (targeting a TC of 11 for Ussuri into "V")? Or something even more drastic like just letting them all expire and filling none, immediately down-sizing to a TC of 6 members? Thoughts? -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From grant at civo.com Wed Dec 4 23:53:17 2019 From: grant at civo.com (Grant Morley) Date: Wed, 4 Dec 2019 23:53:17 +0000 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: <046E9C0290DD9149B106B72FC9156BEA04771513@gmsxchsvr01.thecreation.com> References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> <7fe77627-25c8-2b33-b577-757058763826@civo.com> <046E9C0290DD9149B106B72FC9156BEA04771513@gmsxchsvr01.thecreation.com> Message-ID: <1417a1ca-7367-ee24-0896-5405cc77db71@civo.com> HI Eric, We are indeed already setting the DNS servers explicitly in the subnet so I don't think that is the issue ( from what I can tell ) I was wondering if it could be an issue with Neutron not responding in time for the DHCP request from the instance, however I haven't yet found any evidence of this. The only other thought I had was that it could be an issue with RabbitMQ somehow and potentially increasing the "rpc timeout" on neutron to see if that helps as I have seen some errors stating that RabbitMQ didn't respond to a message request in time. However I think it could be a red herring as I would assume if RabbitMQ was to blame, existing tenants and instances would also be suffering. Grant On 04/12/2019 23:46, Eric K. Miller wrote: > > Hi Grant, > > I didn't see any DNS errors either.  The solution was to explicitly > configure the dns servers in the subnet.  Are you doing this already?  > Or are you relying on the dnsmasq processes created for the router to > respond to DNS queries (and forward them respectively)? > > > Eric > > *From:*Grant Morley [mailto:grant at civo.com] > *Sent:* Wednesday, December 04, 2019 5:40 PM > *To:* Eric K. Miller; openstack-operators at lists.openstack.org > *Cc:* Ian Banks > *Subject:* Re: DHCP timeout when creating instances for specific tenants > > Hi Eric, > > Thanks for getting back to me. I am fairly sure it is a DHCP error. > The instances are getting an IP when they eventually boot, it is just > taking a long time for them to bring up networking. The strange thing > is, it only seems to be new tenants. All existing tenants are > absolutely fine. > > I can check DNS as well just to be on the safe side, however I wasn't > seeing any errors in the Nova or Neutron logs when the instance(s) > were being created. > > Regards, > > On 04/12/2019 22:47, Eric K. Miller wrote: > > Hi Grant, > > Are you sure this is a DHCP timeout and not a DNS resolution > issue?  I ask because we have seen a strange DNS issue occur that > can cause something similar. > > Are the VMs being assigned an IP after they finally boot? > > Eric K. Miller > > Genesis Hosting Solutions, LLC > > Try our Genesis Public Cloud - powered by OpenStack!eut > > https://genesishosting.com/ > > *From:*Grant Morley [mailto:grant at civo.com] > *Sent:* Wednesday, December 04, 2019 11:00 AM > *To:* openstack-operators at lists.openstack.org > > *Cc:* Ian Banks > *Subject:* DHCP timeout when creating instances for specific tenants > > Hi all, > > I wonder if anyone can help shed any light on an odd issue we are > seeing with only a couple of specific tenants. Basically if they > launch an instance they are taking about 5 minutes to launch > rather than our usual 30 second or so launch. > > We are seeing the following on the instance logs: > > https://pastebin.com/hDstsd8G > > Weirdly it only seems to be happening for 1 or 2 new tenants. I > have tested this on our personal account and a few other customers > have tested and their instances launch really quickly as expected. > > Is there anything specific during the tenant creation that can > cause this issue? Or are there any logs in nova / neutron I should > be looking out for that might shed some light? > > I haven't seen anything that is obvious. Any help would be much > appreciated as we are a little stumped at the moment. > > Many thanks, > > -- > > Image removed by sender. > > Grant Morley > > Cloud Lead, Civo Ltd > > www.civo.com | Signup for an account! > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ~WRD374.jpg Type: image/jpeg Size: 823 bytes Desc: not available URL: From grant at civo.com Thu Dec 5 00:00:18 2019 From: grant at civo.com (Grant Morley) Date: Thu, 5 Dec 2019 00:00:18 +0000 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: <6899b9085b784618bc49cdeda0bc3103@hawkless.id.au> References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> <7fe77627-25c8-2b33-b577-757058763826@civo.com> <6899b9085b784618bc49cdeda0bc3103@hawkless.id.au> Message-ID: Hi Cory, Thanks for the response. I'll take a look at the metadata service from the instance and from OpenStack itself tomorrow now. It's midnight here in the UK and I need to get some rest. Thanks for the tip, hopefully I'll find something useful to go on from there. Grant, On 04/12/2019 23:49, Cory Hawkless wrote: > > Are they failing to contact the metadata service and hanging during > the boot process while they try and receive metadata? > > From the VM can you hit http://169.254.169.254 – That’s the default IP > of the metadata server, it should respond with a basic page showing > some date based subdirectories > > If it doesn’t respond you can start following the metadata service > path instead of DHCP > > Given that the machines come up with an IP eventually leads me to > think the DHCP service is actually working ok. > > *From:*Grant Morley [mailto:grant at civo.com] > *Sent:* Thursday, 5 December 2019 10:10 AM > *To:* Eric K. Miller ; > openstack-operators at lists.openstack.org > *Cc:* Ian Banks > *Subject:* Re: DHCP timeout when creating instances for specific tenants > > Hi Eric, > > Thanks for getting back to me. I am fairly sure it is a DHCP error. > The instances are getting an IP when they eventually boot, it is just > taking a long time for them to bring up networking. The strange thing > is, it only seems to be new tenants. All existing tenants are > absolutely fine. > > I can check DNS as well just to be on the safe side, however I wasn't > seeing any errors in the Nova or Neutron logs when the instance(s) > were being created. > > Regards, > > On 04/12/2019 22:47, Eric K. Miller wrote: > > Hi Grant, > > Are you sure this is a DHCP timeout and not a DNS resolution > issue?  I ask because we have seen a strange DNS issue occur that > can cause something similar. > > Are the VMs being assigned an IP after they finally boot? > > Eric K. Miller > > Genesis Hosting Solutions, LLC > > Try our Genesis Public Cloud - powered by OpenStack!eut > > https://genesishosting.com/ > > *From:*Grant Morley [mailto:grant at civo.com] > *Sent:* Wednesday, December 04, 2019 11:00 AM > *To:* openstack-operators at lists.openstack.org > > *Cc:* Ian Banks > *Subject:* DHCP timeout when creating instances for specific tenants > > Hi all, > > I wonder if anyone can help shed any light on an odd issue we are > seeing with only a couple of specific tenants. Basically if they > launch an instance they are taking about 5 minutes to launch > rather than our usual 30 second or so launch. > > We are seeing the following on the instance logs: > > https://pastebin.com/hDstsd8G > > Weirdly it only seems to be happening for 1 or 2 new tenants. I > have tested this on our personal account and a few other customers > have tested and their instances launch really quickly as expected. > > Is there anything specific during the tenant creation that can > cause this issue? Or are there any logs in nova / neutron I should > be looking out for that might shed some light? > > I haven't seen anything that is obvious. Any help would be much > appreciated as we are a little stumped at the moment. > > Many thanks, > > -- > > Grant Morley > > Cloud Lead, Civo Ltd > > www.civo.com | Signup for an account! > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From laurentfdumont at gmail.com Thu Dec 5 00:09:57 2019 From: laurentfdumont at gmail.com (Laurent Dumont) Date: Wed, 4 Dec 2019 19:09:57 -0500 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> <7fe77627-25c8-2b33-b577-757058763826@civo.com> <6899b9085b784618bc49cdeda0bc3103@hawkless.id.au> Message-ID: Maybe the following could provide a bit more data : - Launch a test instance in the tenant project experiencing the issue. - tcpdump directly on the instance TAP interface - confirm if you are seeing DHCP DISCOVER/REQUEST/OFFER - Would also allow you to see the Cloudinit traffic. On Wed, Dec 4, 2019 at 7:06 PM Grant Morley wrote: > Hi Cory, > > Thanks for the response. I'll take a look at the metadata service from the > instance and from OpenStack itself tomorrow now. It's midnight here in the > UK and I need to get some rest. Thanks for the tip, hopefully I'll find > something useful to go on from there. > > Grant, > On 04/12/2019 23:49, Cory Hawkless wrote: > > Are they failing to contact the metadata service and hanging during the > boot process while they try and receive metadata? > > From the VM can you hit http://169.254.169.254 – That’s the default IP of > the metadata server, it should respond with a basic page showing some date > based subdirectories > > If it doesn’t respond you can start following the metadata service path > instead of DHCP > > Given that the machines come up with an IP eventually leads me to think > the DHCP service is actually working ok. > > > > *From:* Grant Morley [mailto:grant at civo.com ] > *Sent:* Thursday, 5 December 2019 10:10 AM > *To:* Eric K. Miller > ; openstack-operators at lists.openstack.org > *Cc:* Ian Banks > *Subject:* Re: DHCP timeout when creating instances for specific tenants > > > > Hi Eric, > > Thanks for getting back to me. I am fairly sure it is a DHCP error. The > instances are getting an IP when they eventually boot, it is just taking a > long time for them to bring up networking. The strange thing is, it only > seems to be new tenants. All existing tenants are absolutely fine. > > I can check DNS as well just to be on the safe side, however I wasn't > seeing any errors in the Nova or Neutron logs when the instance(s) were > being created. > > Regards, > > On 04/12/2019 22:47, Eric K. Miller wrote: > > Hi Grant, > > > > Are you sure this is a DHCP timeout and not a DNS resolution issue? I ask > because we have seen a strange DNS issue occur that can cause something > similar. > > > > Are the VMs being assigned an IP after they finally boot? > > > > Eric K. Miller > > Genesis Hosting Solutions, LLC > > Try our Genesis Public Cloud - powered by OpenStack!eut > > https://genesishosting.com/ > > > > > > > > *From:* Grant Morley [mailto:grant at civo.com ] > *Sent:* Wednesday, December 04, 2019 11:00 AM > *To:* openstack-operators at lists.openstack.org > *Cc:* Ian Banks > *Subject:* DHCP timeout when creating instances for specific tenants > > > > Hi all, > > I wonder if anyone can help shed any light on an odd issue we are seeing > with only a couple of specific tenants. Basically if they launch an > instance they are taking about 5 minutes to launch rather than our usual 30 > second or so launch. > > We are seeing the following on the instance logs: > > https://pastebin.com/hDstsd8G > > Weirdly it only seems to be happening for 1 or 2 new tenants. I have > tested this on our personal account and a few other customers have tested > and their instances launch really quickly as expected. > > Is there anything specific during the tenant creation that can cause this > issue? Or are there any logs in nova / neutron I should be looking out for > that might shed some light? > > I haven't seen anything that is obvious. Any help would be much > appreciated as we are a little stumped at the moment. > > Many thanks, > > -- > > > > Grant Morley > > Cloud Lead, Civo Ltd > > www.civo.com | Signup for an account! > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ileixe at gmail.com Thu Dec 5 02:03:50 2019 From: ileixe at gmail.com (=?UTF-8?B?7JaR7Jyg7ISd?=) Date: Thu, 5 Dec 2019 11:03:50 +0900 Subject: What's the best recommendation of cache solution for Openstack services? Message-ID: Hi, stackers. I wonder how other system deal with the problem we encountered. Any advice would be appreciated. He had a over 1000 hypervisors and hit some connection bottleneck in a DB. The DB instance handle all Openstack services (nova, neutron, cinder, glance ...) Also, we have some additional systems (kubernetes, consul...) which periodically request openstack API. Since most of request from additional systems is READ, I am trying to find cache approach for Openstack service. Although some services like keystone has their own caching layer, I could not find general solutions. Is there any project to solve this problem (reduce DB overhead in a lot of read overhead) I found oslo.cache only provide decorator for a function, so wonder there is any service level approach. Thanks From zbitter at redhat.com Thu Dec 5 03:18:06 2019 From: zbitter at redhat.com (Zane Bitter) Date: Wed, 4 Dec 2019 22:18:06 -0500 Subject: [TC] Right-sizing the Technical Committee In-Reply-To: <20191204235045.dx3the2pjz6c26jw@yuggoth.org> References: <20191204235045.dx3the2pjz6c26jw@yuggoth.org> Message-ID: <05c6d862-b769-47dd-2c0a-7b5591ae6082@redhat.com> On 4/12/19 6:50 pm, Jeremy Stanley wrote: > Tangential to election scheduling but still on the topic of election > planning, last cycle a bunch of folks jumped on the "let's shrink > the TC!" bandwagon *while* the election process was already > underway. That was of course not an appropriate time to talk about > changes to election parameters. But now(ish) *is* the right time. > > So to reopen that discussion we previously put a pin in, how many TC > seats should we fill in the coming election, and how many should we > delete? There were a few different suggestions, some following a > less aggressive timeline than others. We would normally have 7 seats > up for grabs in the coming round... do we reduce it to 6 (and work > with an even-number-sized TC), or just 5 (targeting a TC of 11 for > Ussuri into "V")? Or something even more drastic like just letting > them all expire and filling none, immediately down-sizing to a TC of > 6 members? Thoughts? This is pretty well-settled: https://review.opendev.org/681266 (At least assuming we ignore the fact that JP merged it when it had only 8 of the 9 required votes, which I only just noticed. Naughty JP.) From kennelson11 at gmail.com Thu Dec 5 03:18:49 2019 From: kennelson11 at gmail.com (Kendall Nelson) Date: Wed, 4 Dec 2019 19:18:49 -0800 Subject: Swedish University Engagement Program Message-ID: Hello Everyone! During the PTG, I talked to Kim Hindhart (along with Ildiko Vancsa, Sean McGinnis, and Victoria Martinez de la Cruz) about a program he was working on with several universities in Sweden (Blekinge Institute of Technology, Uppsala University, and KTH Royal Institute of Technology) to get students involved in open source, specifically involved in OpenStack. While this program is still getting off the ground, with its first round to start in this coming February with students working on Horizon (led by Kim), we are hoping to collect work for the second round that will be more student driven. The only disclaimer is that items suggested might not get picked up until the fall semester in September. I am collecting low-hanging-fruit here on this etherpad[1].Hopefully you have a lot of things tagged as such on Launchpad/StoryBoard already and can easily add those links to the etherpad. Can't wait to see what we can come up with! -Kendall (diablo_rojo) [1] https://etherpad.openstack.org/p/swedish-university-program-work -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Thu Dec 5 03:44:47 2019 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Thu, 5 Dec 2019 11:44:47 +0800 Subject: [heat][heat-translator] add Hiroyuki Jo to heat-translator core reviewer Message-ID: Hi all Hiroyuki Jo has been active in the past few months. Also, he raises his willingness to help maintain patches in the last PTG [1]. It's my pleasure to add Hiroyuki Jo to heat-translator-core team with condition that me and other core members should help to review his patch (so no self-approval) and provide review guideline if needed. Free feel to suggest otherwise. [1] https://etherpad.openstack.org/p/PVG-heat -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungleboyj at gmail.com Thu Dec 5 05:22:33 2019 From: jungleboyj at gmail.com (Jay Bryant) Date: Wed, 4 Dec 2019 23:22:33 -0600 Subject: [TC] Election terms In-Reply-To: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> References: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> Message-ID: <6fa1e07f-2147-9b1f-0326-371f81edc119@gmail.com> On 12/4/2019 5:39 PM, Jeremy Stanley wrote: > On 2019-12-04 15:01:00 -0800 (-0800), Kendall Nelson wrote: > [...] >> For TC member terms, those closing in on their ~1 year terms would >> be closer to 13 months since the train elections concluded March >> 5th of last year > [...] > > Just to clarify, this is what our (recently updated) Technical > Committee Member Policy appendix to the OSF Bylaws says about TC > term durations: > > 2.a. [...] Each Technical Committee member shall hold the seat > for a term not to exceed sixteen months, but may be re-elected > to the Technical Committee. After January 1, 2019, the term for > the members of the Technical Committee shall be approved by a > majority of the Technical Committee (“Term”) and shall be > published publicly before each Technical Committee election; if > no such Term is published the Term will be twelve calendar > months. [...] > > https://www.openstack.org/legal/technical-committee-member-policy/ > > So those members elected in March can't actually serve past the > anniversary of their elections *unless* prior to the election in > which they were elected the seated TC had declared they would serve > a longer term. The bylaws allow the TC to set a longer term length, > but they must do so before the members who will serve that term are > elected. > > So terms for half the TC members will be expiring at the start of > March, quite possibly weeks before another TC election is held to > fill their seats. This is what the TC charter seems to say: > > If the vacancy opens less than four weeks before the candidacy > period for the next scheduled TC election begins, and the seat > vacated would have been contested in the upcoming election > anyway, then the seat will remain open until the election and > filled by the normal election process. > > https://governance.openstack.org/tc/reference/charter.html > > So with a strict reading of the bylaws and charter wording, the TC > will spend a few weeks with only 6 members until the suggested > election date. There are a few options: > > 1. Push the TC election back further. This probably similarly > implies backing the extra-ATC deadline up even more. I am guessing that this is hard for the election committee and seems to avoid the fact that we seem to have more widely varying release schedules. > > 2. Amend the charter to allow the TC to reappoint the holders of the > expired seats until the next election (I can't quite tell if this > would violate the term limit wording in the bylaws, but it needs > discussing). I think for this particular case it is worth discussing and is something we maybe want to formalize for the future in case other unexpected circumstances arise. > 3. Probably only a solution for 2021 and later elections, but start > declaring now that TC seat terms are something like "14 months or > until the date of the next election" so that the TC doesn't need to > predict election timing more than a year in advance (OSF bylaws > authorize us to make it up to 16 months for that matter). I would support this change in the future or we could combine 2 and 3 such that they have a 14 month term or until the date of the next election.  If the next election is beyond 14 months the TC can reappoint the holders until the election can be held.  That would be a catch all. Jay From jean-philippe at evrard.me Thu Dec 5 11:17:02 2019 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Thu, 05 Dec 2019 12:17:02 +0100 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: <20191204182209.GA590405@localhost.localdomain> References: <572abd10691af9fdf51b6da74bc65f12c638a24b.camel@odyssey4.me> <570129054cb9ff40a7ac151bc49553176423c9f5.camel@evrard.me> <20191204182209.GA590405@localhost.localdomain> Message-ID: <13280cc10859171f77d5bcd4079670dcf7e32e8e.camel@evrard.me> On Wed, 2019-12-04 at 13:22 -0500, Paul Belanger wrote: > Could we also update http://eavesdrop.openstack.org/ to reference the > ansible-sig meeting time? I'd like to add this info into calander. > It is discussed here [1]. Regards, JP [1]: https://review.opendev.org/#/c/697278/ From jean-philippe at evrard.me Thu Dec 5 11:26:58 2019 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Thu, 05 Dec 2019 12:26:58 +0100 Subject: [TC] Right-sizing the Technical Committee In-Reply-To: <05c6d862-b769-47dd-2c0a-7b5591ae6082@redhat.com> References: <20191204235045.dx3the2pjz6c26jw@yuggoth.org> <05c6d862-b769-47dd-2c0a-7b5591ae6082@redhat.com> Message-ID: <4f280085f2b11505bfca42da6b36c30396c4e20c.camel@evrard.me> On Wed, 2019-12-04 at 22:18 -0500, Zane Bitter wrote: > On 4/12/19 6:50 pm, Jeremy Stanley wrote: > > Tangential to election scheduling but still on the topic of > > election > > planning, last cycle a bunch of folks jumped on the "let's shrink > > the TC!" bandwagon *while* the election process was already > > underway. That was of course not an appropriate time to talk about > > changes to election parameters. But now(ish) *is* the right time. > > > > So to reopen that discussion we previously put a pin in, how many > > TC > > seats should we fill in the coming election, and how many should we > > delete? There were a few different suggestions, some following a > > less aggressive timeline than others. We would normally have 7 > > seats > > up for grabs in the coming round... do we reduce it to 6 (and work > > with an even-number-sized TC), or just 5 (targeting a TC of 11 for > > Ussuri into "V")? Or something even more drastic like just letting > > them all expire and filling none, immediately down-sizing to a TC > > of > > 6 members? Thoughts? > > This is pretty well-settled: > > https://review.opendev.org/681266 > > (At least assuming we ignore the fact that JP merged it when it had > only > 8 of the 9 required votes, which I only just noticed. Naughty JP.) > > You know I like being naughty! However, I don't think I was it this time: For a TC of 13 members, 7 is the simple majority. We had consensus too :) I wrote this email to (shameless plug) remind that only 5 votes are technically required for motions when we have 13 members [1], but I generally try to reach the simple majority if possible. Regards, JP [1]: https://governance.openstack.org/tc/reference/charter.html#motions From grant at civo.com Thu Dec 5 11:38:36 2019 From: grant at civo.com (Grant Morley) Date: Thu, 5 Dec 2019 11:38:36 +0000 Subject: DHCP timeout when creating instances for specific tenants In-Reply-To: References: <4f73ebf9-b912-74ef-c899-d35bd7727422@civo.com> <046E9C0290DD9149B106B72FC9156BEA0477150F@gmsxchsvr01.thecreation.com> <7fe77627-25c8-2b33-b577-757058763826@civo.com> <6899b9085b784618bc49cdeda0bc3103@hawkless.id.au> Message-ID: Hi all, It looks like the issue was actually with the Ubuntu image for both 16.04 and 18.04. We changed the dhcp timeout in "/etc/dhcp/dhclient.conf" from 300 seconds down to 2 seconds and the instances then worked absolutely fine. Not sure why it was only happening for some tenants and not others but that has resolved it. I am still going to look into the metadata service as the fix doesn't feel right to me still. Thanks again for all your help. Grant On 05/12/2019 00:09, Laurent Dumont wrote: > Maybe the following could provide a bit more data : > > * Launch a test instance in the tenant project experiencing the issue. > * tcpdump directly on the instance TAP interface - confirm if you > are seeing DHCP DISCOVER/REQUEST/OFFER > * Would also allow you to see the Cloudinit traffic. > > > On Wed, Dec 4, 2019 at 7:06 PM Grant Morley > wrote: > > Hi Cory, > > Thanks for the response. I'll take a look at the metadata service > from the instance and from OpenStack itself tomorrow now. It's > midnight here in the UK and I need to get some rest. Thanks for > the tip, hopefully I'll find something useful to go on from there. > > Grant, > > On 04/12/2019 23:49, Cory Hawkless wrote: >> >> Are they failing to contact the metadata service and hanging >> during the boot process while they try and receive metadata? >> >> From the VM can you hit http://169.254.169.254 – That’s the >> default IP of the metadata server, it should respond with a basic >> page showing some date based subdirectories >> >> If it doesn’t respond you can start following the metadata >> service path instead of DHCP >> >> Given that the machines come up with an IP eventually leads me to >> think the DHCP service is actually working ok. >> >> *From:*Grant Morley [mailto:grant at civo.com] >> *Sent:* Thursday, 5 December 2019 10:10 AM >> *To:* Eric K. Miller >> ; >> openstack-operators at lists.openstack.org >> >> *Cc:* Ian Banks >> *Subject:* Re: DHCP timeout when creating instances for specific >> tenants >> >> Hi Eric, >> >> Thanks for getting back to me. I am fairly sure it is a DHCP >> error. The instances are getting an IP when they eventually boot, >> it is just taking a long time for them to bring up networking. >> The strange thing is, it only seems to be new tenants. All >> existing tenants are absolutely fine. >> >> I can check DNS as well just to be on the safe side, however I >> wasn't seeing any errors in the Nova or Neutron logs when the >> instance(s) were being created. >> >> Regards, >> >> On 04/12/2019 22:47, Eric K. Miller wrote: >> >> Hi Grant, >> >> Are you sure this is a DHCP timeout and not a DNS resolution >> issue?  I ask because we have seen a strange DNS issue occur >> that can cause something similar. >> >> Are the VMs being assigned an IP after they finally boot? >> >> Eric K. Miller >> >> Genesis Hosting Solutions, LLC >> >> Try our Genesis Public Cloud - powered by OpenStack!eut >> >> https://genesishosting.com/ >> >> *From:*Grant Morley [mailto:grant at civo.com] >> *Sent:* Wednesday, December 04, 2019 11:00 AM >> *To:* openstack-operators at lists.openstack.org >> >> *Cc:* Ian Banks >> *Subject:* DHCP timeout when creating instances for specific >> tenants >> >> Hi all, >> >> I wonder if anyone can help shed any light on an odd issue we >> are seeing with only a couple of specific tenants. Basically >> if they launch an instance they are taking about 5 minutes to >> launch rather than our usual 30 second or so launch. >> >> We are seeing the following on the instance logs: >> >> https://pastebin.com/hDstsd8G >> >> Weirdly it only seems to be happening for 1 or 2 new tenants. >> I have tested this on our personal account and a few other >> customers have tested and their instances launch really >> quickly as expected. >> >> Is there anything specific during the tenant creation that >> can cause this issue? Or are there any logs in nova / neutron >> I should be looking out for that might shed some light? >> >> I haven't seen anything that is obvious. Any help would be >> much appreciated as we are a little stumped at the moment. >> >> Many thanks, >> >> -- >> >> Grant Morley >> >> Cloud Lead, Civo Ltd >> >> www.civo.com | Signup for an account! >> >> -- Grant Morley Cloud Lead, Civo Ltd www.civo.com | Signup for an account! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Thu Dec 5 13:58:00 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Thu, 5 Dec 2019 07:58:00 -0600 Subject: What's the best recommendation of cache solution for Openstack services? In-Reply-To: References: Message-ID: <3f4dc56a-5178-5781-8523-ecd705af6baa@gmail.com> On 12/4/2019 8:03 PM, 양유석 wrote: > Hi, stackers. > > I wonder how other system deal with the problem we encountered. Any > advice would be appreciated. > > He had a over 1000 hypervisors and hit some connection bottleneck in a > DB. The DB instance handle all Openstack services (nova, neutron, > cinder, glance ...) > > Also, we have some additional systems (kubernetes, consul...) which > periodically request openstack API. > > Since most of request from additional systems is READ, I am trying to > find cache approach for Openstack service. Although some services like > keystone has their own caching layer, I could not find general > solutions. > > Is there any project to solve this problem (reduce DB overhead in a > lot of read overhead) > I found oslo.cache only provide decorator for a function, so wonder > there is any service level approach. > > Thanks > It's not a cache but if you're looking for help with READ load you could setup a read-only copy of the nova database(s) and configure a slave connection: https://docs.openstack.org/nova/latest/configuration/config.html#api_database.slave_connection Like the oslo.cache usage in nova, that's also on a per-function/query basis and it's been awhile since there was a concentrated effort on making more queries able to read from the slave connection, but if there are more that could be identified as beneficial to use that read-only connection then I'm sure patches would be welcome. -- Thanks, Matt From zbitter at redhat.com Thu Dec 5 14:05:20 2019 From: zbitter at redhat.com (Zane Bitter) Date: Thu, 5 Dec 2019 09:05:20 -0500 Subject: [TC] Right-sizing the Technical Committee In-Reply-To: <4f280085f2b11505bfca42da6b36c30396c4e20c.camel@evrard.me> References: <20191204235045.dx3the2pjz6c26jw@yuggoth.org> <05c6d862-b769-47dd-2c0a-7b5591ae6082@redhat.com> <4f280085f2b11505bfca42da6b36c30396c4e20c.camel@evrard.me> Message-ID: <7e0383b0-5c4d-5bc6-6e3d-702b04fe759a@redhat.com> On 5/12/19 6:26 am, Jean-Philippe Evrard wrote: > On Wed, 2019-12-04 at 22:18 -0500, Zane Bitter wrote: >> On 4/12/19 6:50 pm, Jeremy Stanley wrote: >>> Tangential to election scheduling but still on the topic of >>> election >>> planning, last cycle a bunch of folks jumped on the "let's shrink >>> the TC!" bandwagon *while* the election process was already >>> underway. That was of course not an appropriate time to talk about >>> changes to election parameters. But now(ish) *is* the right time. >>> >>> So to reopen that discussion we previously put a pin in, how many >>> TC >>> seats should we fill in the coming election, and how many should we >>> delete? There were a few different suggestions, some following a >>> less aggressive timeline than others. We would normally have 7 >>> seats >>> up for grabs in the coming round... do we reduce it to 6 (and work >>> with an even-number-sized TC), or just 5 (targeting a TC of 11 for >>> Ussuri into "V")? Or something even more drastic like just letting >>> them all expire and filling none, immediately down-sizing to a TC >>> of >>> 6 members? Thoughts? >> >> This is pretty well-settled: >> >> https://review.opendev.org/681266 >> >> (At least assuming we ignore the fact that JP merged it when it had >> only >> 8 of the 9 required votes, which I only just noticed. Naughty JP.) >> >> > > You know I like being naughty! > However, I don't think I was it this time: For a TC of 13 members, 7 is > the simple majority. We had consensus too :) But amending the charter itself requires 2/3 majority: https://governance.openstack.org/tc/reference/charter.html#amendment > I wrote this email to (shameless plug) remind that only 5 votes are > technically required for motions when we have 13 members [1], but I > generally try to reach the simple majority if possible. > > Regards, > JP > > [1]: https://governance.openstack.org/tc/reference/charter.html#motions > > > From elod.illes at est.tech Thu Dec 5 14:21:11 2019 From: elod.illes at est.tech (=?utf-8?B?RWzDtWQgSWxsw6lz?=) Date: Thu, 5 Dec 2019 14:21:11 +0000 Subject: [cinder][ops][extended-maintenance-sig][public-cloud-sig][enterprise-wg] Cinder to EOL some branches In-Reply-To: References: <3c5fd6e6-8ae4-d300-71a7-97b22431cb3b@gmail.com> <7e8b4158-89b2-6207-6f06-215782e0b126@est.tech> <70f9cd7b-535d-8f6b-dbe2-3578bd21c346@gmail.com> <6ca22c6a-c0c8-711a-91b5-111702cc2d06@est.tech> Message-ID: Hi Brian, Thanks for the update! The plan looks good: - we are not really interested in keeping Newton and Mitaka open - but supporting to keep open Ocata and Pike - the branches in Extended Maintenance About how long should Ocata and Pike be open: I think as long as there are interest to push fixes there and is possible to keep the CI healthy with "reasonable efforts". That means, if for example the CI will be broken on Ocata (due to Zuul v3 changes, dependency issues, etc) and it won't be fixed in some time (a couple of weeks?) then it is ready to EOL. Just to clarify: we support this not only to keep the branches open until a specific date but to help and encourage others to step up and fix issues on branches in the frame of Extended Maintenance, to make EM visible and show that it's worth to keep that process going. Thanks, Előd On 2019. 12. 04. 16:40, Brian Rosmaita wrote: > The tl;dr is that we won't EOL anything until Monday 9 December at the > earliest; see below for details. > > On 12/3/19 10:00 AM, Elõd Illés wrote: >> Hi Brian, >> >> Thanks in advance for proposing in Cinder meeting to wait to EOL Pike >> and Ocata. >> >> So far I have been looking after stable/* branches. > > There seemed to be consensus at today's meeting that we could keep > stable/ocata and stable/pike in EM mode, which would be consistent > with other openstack projects. > >> >> I had a quick check and it seems that the CI on driverfixes/* branches >> are not in a good shape for some time. Those branches: >> >> - don't have periodic jobs, so we can't see how long they are in this >> faulty state >> - the test jobs are very limited >> - as I see there aren't really arriving patches lately (actually one >> arrived in newton ~2 months ago) >> >> I can try to search for solutions there to fix the gate, but maybe the >> activity there is less than ocata and pike and if there aren't others >> who are interested in driverfixes/* branches then maybe those reached >> the state to move them to EOL. > > I agree with your analysis of the state of the driverfixes branches. > Would you be interested in getting driverfixes/newton in working > order, or are you only interested in us keeping stable/o and /p open? > > To be clear, the current idea is: > - EOL driverfixes/mitaka and driverfixes/newton > - leave stable/ocata and stable/pike in EM mode > >> >> Thanks again, >> >> Előd >> >> >> On 2019. 12. 02. 16:25, Brian Rosmaita wrote: >>> On 11/27/19 12:02 PM, Elõd Illés wrote: >>>> Hi, >>>> >>>> First of all, sorry for the late response. >>>> >>>> About EOLing Ocata and Pike: Extended Maintenance was formed just to >>>> have a common place for interested parties, vendors, operators, to >>>> push >>>> bugfixes to. Currently these branches are in a good shape, check / >>>> gate >>>> jobs work and as far as I see there are a couple of backports, too >>>> (not >>>> too many, though). So hopefully it's not a waste of efforts. Why don't >>>> we keep them open as long as the CI works and there are patches? Of >>>> course, whenever e.g. Ocata branch / CI becomes unmaintainable >>>> (zuul v3 >>>> incompatibilities) or there aren't anyone who fixes the issues >>>> there, we >>>> can put it to EOL then. >>> >>> Quick question: the driverfixes/mitaka and driverfixes/newton gates >>> are broken now.  Do you have any interest in getting these working, or >>> are you only interested in stable/ocata and stable/pike? >>> >>>> I write this, because my employer supports Extended Maintenance, and I >>>> usually try to fix CI issues on stable branches and reviewing patches >>>> there. >>> >>> Thanks for following this strategy, it's good for the community, and >>> we appreciate it. >>> >>>> So maybe I can be some help here, too. Please consider leaving >>>> branches in Extended Maintenance open as long as they are in a good >>>> shape and there are bugfixes coming. >>> >>> This is good feedback.  As I said in the original email, we don't mind >>> keeping these open as long as people are actually using them. I'll >>> put a proposal on this week's Cinder meeting agenda to wait to EOL >>> ocata and pike until their gates break and no one steps up to fix >>> them, instead of proactively EOL'ing them now. >>> >>> Do let me know your position on the driverfixes branches right away, >>> though. >>> >>> >>> thanks, >>> brian >>> >>>> >>>> Thanks, >>>> >>>> Előd >>>> >>>> >>>> On 2019. 11. 25. 20:21, Brian Rosmaita wrote: >>>>> This is a courtesy notice that having received no responses to my >>>>> email of 28 October [0] proposing to EOL some currently open Cinder >>>>> branches, and following the policy articulated in [1], at today's >>>>> Virtual PTG meeting the Cinder project team has decided to put the >>>>> following stable branches into the End of Life state: >>>>>     driverfixes/mitaka >>>>>     driverfixes/newton >>>>>     stable/ocata >>>>>     stable/pike >>>>> >>>>> I will submit the paperwork to get this process moving one week from >>>>> today (2 December 2019). >>>>> >>>>> >>>>> cheers, >>>>> brian >>>>> >>>>> [0] >>>>> http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010385.html >>>>> >>>>> >>>>> [1] >>>>> https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-life >>>>> >>>>> >>>>> >>>> >>> >>> >> > From fungi at yuggoth.org Thu Dec 5 15:06:44 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Thu, 5 Dec 2019 15:06:44 +0000 Subject: [TC] Election terms In-Reply-To: <6fa1e07f-2147-9b1f-0326-371f81edc119@gmail.com> References: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> <6fa1e07f-2147-9b1f-0326-371f81edc119@gmail.com> Message-ID: <20191205150643.73yatpprlprlmxiv@yuggoth.org> On 2019-12-04 23:22:33 -0600 (-0600), Jay Bryant wrote: > On 12/4/2019 5:39 PM, Jeremy Stanley wrote: [...] > > 1. Push the TC election back further. This probably similarly > > implies backing the extra-ATC deadline up even more. > > I am guessing that this is hard for the election committee and > seems to avoid the fact that we seem to have more widely varying > release schedules. It's not really significantly harder on the election officials, but does possibly present challenges for the community if they need to identify additional extra-ATCs in advance of the TC election as they'd need to do it by, like, Ussuri milestone 2. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From eblock at nde.ag Thu Dec 5 15:25:24 2019 From: eblock at nde.ag (Eugen Block) Date: Thu, 05 Dec 2019 15:25:24 +0000 Subject: email alert settings are not applied (ask.openstack.org) Message-ID: <20191205152524.Horde.7IoBJehsJP-TNbVJXDxm3QV@webmail.nde.ag> Hi all, I'm not really sure if this is the right place to ask but I'll try it anyway. I'm regularly checking ask.openstack.org for interesting questions and sometimes I comment and respond. In my profile I configured instant email notification if someone comments or responds to my questions/comments. But this has never worked properly, I think this worked just once two or three years ago, never again. The only setting that seems to work is the weekly notification about new questions. Can anyone comment on this? Should I try to change the email address and see if that works? Any insight is appreciated! Regards, Eugen From sean.mcginnis at gmx.com Thu Dec 5 17:04:25 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 5 Dec 2019 11:04:25 -0600 Subject: [ptl] Train cycle-trailing release deadline Message-ID: <20191205170425.GA571363@sm-workstation> Hello all, For those teams with cycle-trailing deliverables, this is a reminder to make sure to get your Train series deliverables released. We like to have all of those out within three months of the coordinated release. For reference, here is a list of deliverables we have marked as Train cycle-trailing in the releases repo that do not appear to have a final release: cinderlib kayobe kolla-ansible kolla-cli kolla openstack-ansible tripleo-ui If these, or any others, need to be added or changed for Ussuri, please propose a placeholder deliverable file under deliverables/ussuri or update any existing ones with any needed changes. Thanks! Sean From rosmaita.fossdev at gmail.com Thu Dec 5 17:06:10 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Thu, 5 Dec 2019 12:06:10 -0500 Subject: [OSSN-0085] Cinder configuration option can leak secret key from Ceph backend Message-ID: <3ffd948f-83ac-75e3-d5d8-0323480601da@gmail.com> Cinder configuration option can leak secret key from Ceph backend --- ### Summary ### This vulnerability is present only when Cinder has a Ceph backend *and* the ``rbd_keyring_conf`` option in the Cinder configuration file is set. (The option is unset by default.) Under these circumstances, the keyring content may be leaked to a regular user. ### Affected Services / Software ### Cinder, Pike, Queens, Rocky, Stein, Train ### Discussion ### When the ``rbd_keyring_conf`` option is set, a user who creates a volume and then does a local attach of that volume using the Cinder REST API, may discover the keyring content for the Ceph cluster. (This does not apply to the normal Nova attach process. The user must contact the Cinder REST API directly for this exploit.) If this user then gains access to the Ceph cluster independently of Cinder, these credentials may be used to access any volume in the cluster. This issue was reported by Raphael Glon of OVH. NOTE: This issue is different from OSSA-2019-003, through which Ceph credentials could be leaked from the Nova REST API during error conditions, but we suggest you take this opportunity to review that security advisory and make sure your system has been updated or patched to address it: https://security.openstack.org/ossa/OSSA-2019-003.html ### Recommended Actions ### Any installation currently using the ``rbd_keyring_conf`` option should immediately stop using that option. In order for Cinder backups to continue working, the Ceph keyring secrets should be deployed directly on cinder-backup hosts in their standard location: /etc/ceph/.client..keyring The ``rbd_keyring_conf`` is deprecated in the Ussuri release and will be removed early in the 'V' development cycle. There is no replacement planned for this functionality. ### Contacts / References ### Author: Brian Rosmaita, Red Hat This OSSN : https://wiki.openstack.org/wiki/OSSN/OSSN-0085 Original LaunchPad Bug : https://bugs.launchpad.net/cinder/+bug/1849624 Mailing List : [Security] tag on openstack-discuss at lists.openstack.org OpenStack Security Project : https://launchpad.net/~openstack-ossg From jlibosva at redhat.com Thu Dec 5 09:34:51 2019 From: jlibosva at redhat.com (Jakub Libosvar) Date: Thu, 5 Dec 2019 10:34:51 +0100 Subject: [neutron] Proposing Jakub Libosvar as Neutron core reviewer In-Reply-To: References: <207982A0-CBE7-47D9-A19C-CCFCACCB34EF@redhat.com> Message-ID: Thanks Slawek and thanks everyone for your support :) I look forward working with all of you again. Kuba On 04/12/2019 17:01, Slawek Kaplonski wrote: > Hi, > > There was no any votes against during last week. > So welcome back in the Neutron core team Jakub :) > >> On 3 Dec 2019, at 09:17, Akihiro Motoki wrote: >> >> +1 from me. It would be nice to have him again. >> >> On Thu, Nov 28, 2019 at 6:01 PM Slawek Kaplonski wrote: >>> >>> Hi neutrinos, >>> >>> We already started process of migrating networking-ovn driver to be one of in-tree neutron drivers. Blueprint for that is [1]. >>> As part of this process I today proposed to include networking-ovn-core group into neutron-core group. Mail about it can be found at [2]. >>> One of persons in networking-ovn-group is Jakub Libosvar who was Neutron core for very long time in the past. He knows very well not only ovn related code but also have great knowledge about all Neutron code base. >>> So I would like to propose to Jakub as Neutron core reviewer again as he will be back working on neutron again now, after ovn will be in-tree driver. >>> What do You think about it? >>> I will wait for Your opinions for 1 week from now. Thx for all Your comments about it. >>> >>> [1] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge >>> [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011240.html >>> >>> — >>> Slawek Kaplonski >>> Senior software engineer >>> Red Hat >>> >>> >> > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From mobarak at plexus.com.bd Thu Dec 5 10:32:25 2019 From: mobarak at plexus.com.bd (Mobarak Hossain) Date: Thu, 5 Dec 2019 16:32:25 +0600 Subject: How Bank or Financial sector benefited from OpenStack ? Message-ID: Hi, I want to approach to Bank for Open Infrastructure . They are asking how a Bank get benefited from Open Infrastructure .Except open Source . Is there any special features for Bank . I saw a video that Amex is using . Is there suggestion for me please let me know . Thanks -- Mobarak Hossain -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdratlif at iu.edu Thu Dec 5 15:24:35 2019 From: jdratlif at iu.edu (Ratliff, John) Date: Thu, 5 Dec 2019 15:24:35 +0000 Subject: openstack volume snapshots and deleting Message-ID: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> If I create a volume, snapshot it, and create a new volume from that snapshot, I cannot seem to delete the snapshot until the volume is deleted. Is this intentional, or am I doing something wrong? Is the new volume based on the snapshot, and that's why I can't delete it? We're using CEPH as a storage backend if that makes a difference. openstack volume snapshot create -volume base-volume base-ss openstack volume create -snapshot base-ss new-volume openstack volume snapshot delete base-ss The last command is the one that fails. It gives no error message. It just doesn't actually delete the snapshot. But If I do this: openstack volume delete new-volume openstack volume snapshot delete base-ss Then it works. I also tried creating a snapshot, creating a volume from the snapshot, then creating a volume from that volume, but I still cannot delete the snapshot until both volumes are deleted. I do not want everlasting snapshots, so I hope I'm doing something wrong that someone can point out. Thanks. --John -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5696 bytes Desc: not available URL: From thierry at openstack.org Thu Dec 5 17:28:09 2019 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 5 Dec 2019 18:28:09 +0100 Subject: How Bank or Financial sector benefited from OpenStack ? In-Reply-To: References: Message-ID: <7e879f9f-7cc5-e5b5-7141-4d01aea2999d@openstack.org> Mobarak Hossain wrote: > I want to approach to Bank for Open Infrastructure . They are asking how > a Bank get benefited from Open Infrastructure .Except open Source . Is > there any special features for Bank . I saw a video that Amex is using . > > Is there suggestion for me please let me know . You can find a number of OpenStack users stories from the financial sector at: https://www.openstack.org/user-stories/ Hope this helps! -- Thierry Carrez (ttx) From aschultz at redhat.com Thu Dec 5 17:29:38 2019 From: aschultz at redhat.com (Alex Schultz) Date: Thu, 5 Dec 2019 10:29:38 -0700 Subject: [ptl] Train cycle-trailing release deadline In-Reply-To: <20191205170425.GA571363@sm-workstation> References: <20191205170425.GA571363@sm-workstation> Message-ID: On Thu, Dec 5, 2019 at 10:09 AM Sean McGinnis wrote: > Hello all, > > For those teams with cycle-trailing deliverables, this is a reminder to > make > sure to get your Train series deliverables released. We like to have all of > those out within three months of the coordinated release. > > For reference, here is a list of deliverables we have marked as Train > cycle-trailing in the releases repo that do not appear to have a final > release: > > cinderlib > kayobe > kolla-ansible > kolla-cli > kolla > openstack-ansible > tripleo-ui > tripleo-ui is no more. Thanks, -Alex > > If these, or any others, need to be added or changed for Ussuri, please > propose > a placeholder deliverable file under deliverables/ussuri or update any > existing > ones with any needed changes. > > Thanks! > Sean > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Thu Dec 5 17:58:16 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 5 Dec 2019 11:58:16 -0600 Subject: openstack volume snapshots and deleting In-Reply-To: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> References: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> Message-ID: <20191205175816.GA576175@sm-workstation> On Thu, Dec 05, 2019 at 03:24:35PM +0000, Ratliff, John wrote: > If I create a volume, snapshot it, and create a new volume from that > snapshot, I cannot seem to delete the snapshot until the volume is deleted. > Is this intentional, or am I doing something wrong? Is the new volume based > on the snapshot, and that's why I can't delete it? We're using CEPH as a > storage backend if that makes a difference. > That is the expected behavior as the snapshot is in-use by the volume created from it. So it can't be deleted until there isn't anything using it. Cloning would be the only way to get around this. There's not a quick and easy call to do this from a snapshot base, so unfortunately to get around that and get the final effect that it appears you are looking for, you would need to create a snapshot, create a volume from the snapshot, then *clone* yet another volume from that new one. Then delete the temporary volume and snapshot once the clone is done. As long as the backend driver is honoring the clone semantics, that should result in a fully independent volume that has its inital data set from the snapshot. So the process would look something like: openstack volume snapshot create --volume base-volume base-ss openstack volume create --snapshot base-ss temp-volume openstack volume create --volume temp-volume new-volume openstack volume delete temp-volume openstack volume snapshot delete base-ss Otherwise, it should be fairly safe to just keep the snapshot around. A lot of storage devices handle snapshots as internal pointers, so the only additional space consumption comes from anything new you end up writing to the new volume. Sean From gmann at ghanshyammann.com Thu Dec 5 18:00:38 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 05 Dec 2019 12:00:38 -0600 Subject: [all][requirements][drop-py27-support] Requirements is dropping cross tests with projects that will be dropping support for python2.7 this cycle In-Reply-To: <20191204222424.GA527721@sm-workstation> References: <20191204221436.onvqxqx2mt2g32qs@mthode.org> <20191204222424.GA527721@sm-workstation> Message-ID: <16ed7388ea9.ddfe43cc76964.5990476850283047557@ghanshyammann.com> ---- On Wed, 04 Dec 2019 16:24:24 -0600 Sean McGinnis wrote ---- > > > > The requirements project is dropping python2.7 cross tests we do for > > those projects that are also dropping python2.7 support this cycle. We > > are doing this as wel do not know when projects will drop support > > (neutron seems to have done so, cinder is soon as well). This was > > originally scheduled for later but need to be pushed up as we missed the > > dependency requirements has on those projects for cross jobs. > > > > Just to expand on this a little from the conversation we had in the > requirements channel... > > The schedule currently has libraries keeping py2 support until milestone 2, > which isn't until mid-February. So we're in a tough spot here as far as > requirements gating. > > We would like to continue testing lib updates with py27 to make sure nothing > breaks. But at the same time, services are no longer gating on py27, so > incompatible changes are creeping in and therefore we are not able to run the > full jobs to exercise the updated lib code under py27. > > Hopefully this is a non-issue and most libs have their own testing under py27 > that can give a reasonable assurance that things aren't completely broken. It's > a temporary problem, and I'm sure February will be here before we know it and > the issue will completely go away. > > In the meantime, please pay attention to lib changes to watch for issues. We > may end up getting to a point where we need to accelerate the timeline (and > probably deal with some broken gates), but I think we should be able to work > through most issues fairly quickly. +1. I think this is a fair adjustment. Cross testing jobs are the main challenges as not everyone dropping py2 at the same time so dropping those jobs from candidates of phase-2/3 is right adjustment. If the same issue on tempest-full, neutron-grenade then, we can convert these also to their py3 version. -gmann > > Thanks, > Sean > > From sean.mcginnis at gmx.com Thu Dec 5 18:01:46 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 5 Dec 2019 12:01:46 -0600 Subject: [ptl] Train cycle-trailing release deadline In-Reply-To: References: <20191205170425.GA571363@sm-workstation> Message-ID: <20191205180146.GB576175@sm-workstation> On Thu, Dec 05, 2019 at 10:29:38AM -0700, Alex Schultz wrote: > On Thu, Dec 5, 2019 at 10:09 AM Sean McGinnis wrote: > > > Hello all, > > > > For those teams with cycle-trailing deliverables, this is a reminder to > > make > > sure to get your Train series deliverables released. We like to have all of > > those out within three months of the coordinated release. > > > > tripleo-ui is no more. > > Thanks, > -Alex > Thanks Alex. Can I get your ack on https://review.opendev.org/697542 so it's recorded there that the team no longer needs those deliverables? Thanks! Sean From sean.mcginnis at gmx.com Thu Dec 5 18:03:55 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 5 Dec 2019 12:03:55 -0600 Subject: [release] Release countdown for week R-22 Message-ID: <20191205180355.GC576175@sm-workstation> Development Focus ----------------- The Ussuri-1 milestone is next week, on December 12! Project team plans for the Ussuri cycle should now be solidified. General Information ------------------- If you planned to change the release model for any of your deliverables this cycle, please remember to do so ASAP, before milestone-1. Libraries need to be released at least once per milestone period. Next week, the release team will propose releases for any library which had changes but has not been otherwise released since the Train release. PTL's or release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well, by posting a -1. If we do not hear anything at all by the end of the week, we will assume things are OK to proceed. NB: If one of your libraries is still releasing 0.x versions, start thinking about when it will be appropriate to do a 1.0 version. The version number does signal the state, real or perceived, of the library, so we strongly encourage going to a full major version once things are in a good and usable state. Upcoming Deadlines & Dates -------------------------- Ussuri-1 milestone: December 12 (R-22 week) From jdratlif at iu.edu Thu Dec 5 18:23:09 2019 From: jdratlif at iu.edu (Ratliff, John) Date: Thu, 5 Dec 2019 18:23:09 +0000 Subject: [External] Re: openstack volume snapshots and deleting In-Reply-To: <20191205175816.GA576175@sm-workstation> References: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> <20191205175816.GA576175@sm-workstation> Message-ID: <178818cb30464cbe9870ee7679af1303@IN-CCI-D1S14.ads.iu.edu> Yes, I tried this second thing, but it doesn't work either. I created a new instance called test1.example.net. I created a volume snapshot (test1.example.net_ss1) I created a volume from that snapshot (vol_from_ss) I created a volume from the volume that was from the snapshot (vol_from_vol_from_ss) I deleted the vol_from_ss (this works) I deleted the snapshot (this does not work) If I then delete the vol_from_vol_from_ss volume, I can then delete the snapshot. P.S. My openstack cli says the option is --source, not --volume. Here is my command output. $ openstack volume list | grep test1 | cc698f20-c56f-47c7-b767-3c554c8cb517 | test1.example.net | in-use | 20 | Attached to test1.example.net on /dev/sda | $ openstack volume snapshot list | grep test1 $ openstack volume snapshot create --volume test1.example.net test1.example.net_ss1 Invalid volume: Volume cc698f20-c56f-47c7-b767-3c554c8cb517 status must be available, but current status is: in-use. (HTTP 400) (Request-ID: req-1ab60e86-3809-4a3b-9ac3-98e7d09ae7e9) $ openstack volume snapshot create --force --volume test1.example.net test1.example.net_ss1 $ openstack volume create --snapshot test1.example.net_ss1 vol_from_ss $ openstack volume create --volume vol_from_ss vol_from_vol_from_ss usage: openstack volume create [-h] [-f {json,shell,table,value,yaml}] [-c COLUMN] [--max-width ] [--fit-width] [--print-empty] [--noindent] [--prefix PREFIX] [--size ] [--type ] [--image | --snapshot | --source ] [--description ] [--availability-zone ] [--consistency-group consistency-group>] [--property ] [--hint ] [--multi-attach] [--bootable | --non-bootable] [--read-only | --read-write] openstack volume create: error: unrecognized arguments: --volume vol_from_vol_from_ss $ openstack volume create --source vol_from_ss vol_from_vol_from_ss $ openstack volume delete vol_from_ss $ openstack volume snapshot delete test1.example.net_ss1 $ openstack volume snapshot list | grep test1 | 386186e9-6121-4222-b487-3dc026acf70d | test1.example.net_ss1 | None | available | 20 | $ openstack volume list | grep 'vol_\|test1' | ccbfdabe-d2ff-4bbd-a0ef-3b08935fe29a | vol_from_vol_from_ss | available | 20 | | | cc698f20-c56f-47c7-b767-3c554c8cb517 | test1.example.net | in-use | 20 | Attached to test1.example.net on /dev/sda | ---------------------------------------------------------------------------- ---- $ openstack volume delete vol_from_vol_from_ss $ openstack volume snapshot delete test1.example.net_ss1 $ openstack volume snapshot list | grep 'test1' $ openstack volume list | grep 'vol_\|test1' | cc698f20-c56f-47c7-b767-3c554c8cb517 | test1.example.net | in-use | 20 | Attached to test1.example.net on /dev/sda | -----Original Message----- From: Sean McGinnis Sent: Thursday, December 5, 2019 12:58 PM To: Ratliff, John Cc: openstack-discuss at lists.openstack.org Subject: [External] Re: openstack volume snapshots and deleting This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. ------- On Thu, Dec 05, 2019 at 03:24:35PM +0000, Ratliff, John wrote: > If I create a volume, snapshot it, and create a new volume from that > snapshot, I cannot seem to delete the snapshot until the volume is deleted. > Is this intentional, or am I doing something wrong? Is the new volume > based on the snapshot, and that's why I can't delete it? We're using > CEPH as a storage backend if that makes a difference. > That is the expected behavior as the snapshot is in-use by the volume created from it. So it can't be deleted until there isn't anything using it. Cloning would be the only way to get around this. There's not a quick and easy call to do this from a snapshot base, so unfortunately to get around that and get the final effect that it appears you are looking for, you would need to create a snapshot, create a volume from the snapshot, then *clone* yet another volume from that new one. Then delete the temporary volume and snapshot once the clone is done. As long as the backend driver is honoring the clone semantics, that should result in a fully independent volume that has its inital data set from the snapshot. So the process would look something like: openstack volume snapshot create --volume base-volume base-ss openstack volume create --snapshot base-ss temp-volume openstack volume create --volume temp-volume new-volume openstack volume delete temp-volume openstack volume snapshot delete base-ss Otherwise, it should be fairly safe to just keep the snapshot around. A lot of storage devices handle snapshots as internal pointers, so the only additional space consumption comes from anything new you end up writing to the new volume. Sean -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5696 bytes Desc: not available URL: From jdratlif at iu.edu Thu Dec 5 18:25:28 2019 From: jdratlif at iu.edu (Ratliff, John) Date: Thu, 5 Dec 2019 18:25:28 +0000 Subject: [External] Re: openstack volume snapshots and deleting In-Reply-To: <20191205175816.GA576175@sm-workstation> References: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> <20191205175816.GA576175@sm-workstation> Message-ID: <9e811d2eda9a41cf99444c0a58c5bbf4@IN-CCI-D1S14.ads.iu.edu> Yes, I tried this second thing, but it doesn't work either. I created a new instance called test1.example.net. I created a volume snapshot (test1.example.net_ss1) I created a volume from that snapshot (vol_from_ss) I created a volume from the volume that was from the snapshot (vol_from_vol_from_ss) I deleted the vol_from_ss (this works) I deleted the snapshot (this does not work) If I then delete the vol_from_vol_from_ss volume, I can then delete the snapshot. P.S. My openstack cli says the option is --source, not --volume. Here is my command output. $ openstack volume list | grep test1 | cc698f20-c56f-47c7-b767-3c554c8cb517 | test1.example.net | in-use | 20 | Attached to test1.example.net on /dev/sda | $ openstack volume snapshot list | grep test1 $ openstack volume snapshot create --volume test1.example.net test1.example.net_ss1 Invalid volume: Volume cc698f20-c56f-47c7-b767-3c554c8cb517 status must be available, but current status is: in-use. (HTTP 400) (Request-ID: req-1ab60e86-3809-4a3b-9ac3-98e7d09ae7e9) $ openstack volume snapshot create --force --volume test1.example.net test1.example.net_ss1 $ openstack volume create --snapshot test1.example.net_ss1 vol_from_ss $ openstack volume create --volume vol_from_ss vol_from_vol_from_ss usage: openstack volume create [-h] [-f {json,shell,table,value,yaml}] [-c COLUMN] [--max-width ] [--fit-width] [--print-empty] [--noindent] [--prefix PREFIX] [--size ] [--type ] [--image | --snapshot | --source ] [--description ] [--availability-zone ] [--consistency-group consistency-group>] [--property ] [--hint ] [--multi-attach] [--bootable | --non-bootable] [--read-only | --read-write] openstack volume create: error: unrecognized arguments: --volume vol_from_vol_from_ss $ openstack volume create --source vol_from_ss vol_from_vol_from_ss $ openstack volume delete vol_from_ss $ openstack volume snapshot delete test1.example.net_ss1 $ openstack volume snapshot list | grep test1 | 386186e9-6121-4222-b487-3dc026acf70d | test1.example.net_ss1 | None | available | 20 | $ openstack volume list | grep 'vol_\|test1' | ccbfdabe-d2ff-4bbd-a0ef-3b08935fe29a | vol_from_vol_from_ss | available | 20 | | | cc698f20-c56f-47c7-b767-3c554c8cb517 | test1.example.net | in-use | 20 | Attached to test1.example.net on /dev/sda | ---------------------------------------------------------------------------- ---- $ openstack volume delete vol_from_vol_from_ss $ openstack volume snapshot delete test1.example.net_ss1 $ openstack volume snapshot list | grep 'test1' $ openstack volume list | grep 'vol_\|test1' | cc698f20-c56f-47c7-b767-3c554c8cb517 | test1.example.net | in-use | 20 | Attached to test1.example.net on /dev/sda | --John -----Original Message----- From: Sean McGinnis Sent: Thursday, December 5, 2019 12:58 PM To: Ratliff, John Cc: openstack-discuss at lists.openstack.org Subject: [External] Re: openstack volume snapshots and deleting This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. ------- On Thu, Dec 05, 2019 at 03:24:35PM +0000, Ratliff, John wrote: > If I create a volume, snapshot it, and create a new volume from that > snapshot, I cannot seem to delete the snapshot until the volume is deleted. > Is this intentional, or am I doing something wrong? Is the new volume > based on the snapshot, and that's why I can't delete it? We're using > CEPH as a storage backend if that makes a difference. > That is the expected behavior as the snapshot is in-use by the volume created from it. So it can't be deleted until there isn't anything using it. Cloning would be the only way to get around this. There's not a quick and easy call to do this from a snapshot base, so unfortunately to get around that and get the final effect that it appears you are looking for, you would need to create a snapshot, create a volume from the snapshot, then *clone* yet another volume from that new one. Then delete the temporary volume and snapshot once the clone is done. As long as the backend driver is honoring the clone semantics, that should result in a fully independent volume that has its inital data set from the snapshot. So the process would look something like: openstack volume snapshot create --volume base-volume base-ss openstack volume create --snapshot base-ss temp-volume openstack volume create --volume temp-volume new-volume openstack volume delete temp-volume openstack volume snapshot delete base-ss Otherwise, it should be fairly safe to just keep the snapshot around. A lot of storage devices handle snapshots as internal pointers, so the only additional space consumption comes from anything new you end up writing to the new volume. Sean -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5696 bytes Desc: not available URL: From sean.mcginnis at gmx.com Thu Dec 5 18:58:47 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Thu, 5 Dec 2019 12:58:47 -0600 Subject: [External] Re: openstack volume snapshots and deleting In-Reply-To: <9e811d2eda9a41cf99444c0a58c5bbf4@IN-CCI-D1S14.ads.iu.edu> References: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> <20191205175816.GA576175@sm-workstation> <9e811d2eda9a41cf99444c0a58c5bbf4@IN-CCI-D1S14.ads.iu.edu> Message-ID: <20191205185847.GA579008@sm-workstation> On Thu, Dec 05, 2019 at 06:25:28PM +0000, Ratliff, John wrote: > Yes, I tried this second thing, but it doesn't work either. > > I created a new instance called test1.example.net. > I created a volume snapshot (test1.example.net_ss1) > I created a volume from that snapshot (vol_from_ss) > I created a volume from the volume that was from the snapshot > (vol_from_vol_from_ss) > I deleted the vol_from_ss (this works) > I deleted the snapshot (this does not work) > Looking at the RBD driver, it looks like it does not actually follow the expected clone semantics. Unless rbd_max_clone is set to 0, or the number of clones is less than whatever that is set to, it will actually do a COW volume on top of the snapshot. This is a performance optimization (it doesn't have to copy all of the data into the new volume), but is actually a bug, IMO. You can try changing that config option and restarting services to pick that up, then trying again. Otherwise, it looks like that drivers clone handling will create its own snapshot of a volume. So unless you need to keep that snapshot around for some other reason, it looks like you should be able to just: openstack volume create --source orig-volume new-volume And if I follow it right, it will create a snapshot under the covers, then use that to create the new volume. Hopefully one of those helps. Sean From gmann at ghanshyammann.com Thu Dec 5 19:06:31 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Thu, 05 Dec 2019 13:06:31 -0600 Subject: [goals][Drop Python 2.7 Support] Week R-23 Update Message-ID: <16ed774e177.ba8db98078182.5959834449939255336@ghanshyammann.com> Hello Everyone, Below is the progress on "Drop Python 2.7 Support" for R-23 week. We are in phase-1 which targets all the OpenStack services to drop the py2 support. Schedule: https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html#schedule NOTE: Phase-2 is for Library (including client library), QA tooling. Phase-3 is mainly for openstack/requirement and audit to all repo. (we can always adjust the schedule to make it smooth migration. For example, dropping a few testing jobs from phase-2/3 candidates if any cross-testing jobs are failing due to phase-1 candidates dropping the support[1].) Summary: ======= Patches to all services repo have been up for review[2]. The OpenStack services have not merged the py2 drop patches: * Adjutant * Barbican * Designate * ec2-api * Glance * Neutron * Freezer * Kolla * Monasca * Karbor * Congress * Telemetry * Storlets * Sahara * Blazar * Masakari * Placement * Heat (heat-dashboard patch is note merged ) * Tacker * Trove * Zaqar * Quinling * Manila * Tricircle * openstack-helm * python-openstackclient The OpenStack services dropped the py2 support: * Cinder * Nova * Ironic * Keystone * Octavia * Watcher * Vitrage * Zun * Winstackers * Mistral * Cyborg * Kuryr * Cloudkitty * Murano * Solum * Magnum * Senlin * Searchlight * openstack-ansible - I do not think we need any update in openstack-ansible? release notes jobs etc are already on py3. Rest all projects are targetted for phase-2 (after m-1). How you can help: ============== - Review the patches. Push the patches if I missed any repo targetted for phase-1. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011392.html [2] https://review.opendev.org/#/q/topic:drop-py27-support -gmann From zbitter at redhat.com Thu Dec 5 19:27:13 2019 From: zbitter at redhat.com (Zane Bitter) Date: Thu, 5 Dec 2019 14:27:13 -0500 Subject: [TC] Right-sizing the Technical Committee In-Reply-To: <7e0383b0-5c4d-5bc6-6e3d-702b04fe759a@redhat.com> References: <20191204235045.dx3the2pjz6c26jw@yuggoth.org> <05c6d862-b769-47dd-2c0a-7b5591ae6082@redhat.com> <4f280085f2b11505bfca42da6b36c30396c4e20c.camel@evrard.me> <7e0383b0-5c4d-5bc6-6e3d-702b04fe759a@redhat.com> Message-ID: On 5/12/19 9:05 am, Zane Bitter wrote: > On 5/12/19 6:26 am, Jean-Philippe Evrard wrote: >> On Wed, 2019-12-04 at 22:18 -0500, Zane Bitter wrote: >>> On 4/12/19 6:50 pm, Jeremy Stanley wrote: >>>> Tangential to election scheduling but still on the topic of >>>> election >>>> planning, last cycle a bunch of folks jumped on the "let's shrink >>>> the TC!" bandwagon *while* the election process was already >>>> underway. That was of course not an appropriate time to talk about >>>> changes to election parameters. But now(ish) *is* the right time. >>>> >>>> So to reopen that discussion we previously put a pin in, how many >>>> TC >>>> seats should we fill in the coming election, and how many should we >>>> delete? There were a few different suggestions, some following a >>>> less aggressive timeline than others. We would normally have 7 >>>> seats >>>> up for grabs in the coming round... do we reduce it to 6 (and work >>>> with an even-number-sized TC), or just 5 (targeting a TC of 11 for >>>> Ussuri into "V")? Or something even more drastic like just letting >>>> them all expire and filling none, immediately down-sizing to a TC >>>> of >>>> 6 members? Thoughts? >>> >>> This is pretty well-settled: >>> >>> https://review.opendev.org/681266 >>> >>> (At least assuming we ignore the fact that JP merged it when it had >>> only >>> 8 of the 9 required votes, which I only just noticed. Naughty JP.) >>> >>> >> >> You know I like being naughty! >> However, I don't think I was it this time: For a TC of 13 members, 7 is >> the simple majority. We had consensus too :) > > But amending the charter itself requires 2/3 majority: > > https://governance.openstack.org/tc/reference/charter.html#amendment Closing the loop on this, we got 3 more TC members to retroactively comment with their approval, so the rules are satisfied. I proposed a tweak to the documentation to help remind us that we should use the `charter-change` tag rather than `formal-vote` in future - it's currently easy to miss because of the non-locality of the documentation sections describing them: https://review.opendev.org/697511 - ZB From ignaziocassano at gmail.com Thu Dec 5 20:46:06 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Thu, 5 Dec 2019 21:46:06 +0100 Subject: [magnum][stein] broken pipe Message-ID: Hello, I installed magnum stein on centos 7 but magnum-api reports the following error in /var/log/messages: Dec 5 21:14:49 tst2-osctrl01 magnum-api: Traceback (most recent call last): Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 568, in process_request Dec 5 21:14:49 tst2-osctrl01 magnum-api: self.finish_request(request, client_address) Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request Dec 5 21:14:49 tst2-osctrl01 magnum-api: self.RequestHandlerClass(request, client_address, self) Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 651, in __init__ Dec 5 21:14:49 tst2-osctrl01 magnum-api: self.finish() Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/SocketServer.py", line 710, in finish Dec 5 21:14:49 tst2-osctrl01 magnum-api: self.wfile.close() Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/socket.py", line 279, in close Dec 5 21:14:49 tst2-osctrl01 magnum-api: self.flush() Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib64/python2.7/socket.py", line 303, in flush Dec 5 21:14:49 tst2-osctrl01 magnum-api: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 401, in sendall Dec 5 21:14:49 tst2-osctrl01 magnum-api: tail = self.send(data, flags) Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 395, in send Dec 5 21:14:49 tst2-osctrl01 magnum-api: return self._send_loop(self.fd.send, data, flags) Dec 5 21:14:49 tst2-osctrl01 magnum-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 382, in _send_loop Dec 5 21:14:49 tst2-osctrl01 magnum-api: return send_method(data, *args) Dec 5 21:14:49 tst2-osctrl01 magnum-api: error: [Errno 32] Broken pipe Anyone could help me please ? Regards Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From sshnaidm at redhat.com Thu Dec 5 22:03:04 2019 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Fri, 6 Dec 2019 00:03:04 +0200 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: References: Message-ID: Hi, all short minutes from the meeting today about Openstack Ansible modules. 1. Ansible 2.10 is going to move all modules to collections, so Openstack modules should find a new home in Openstack repos. 2. Namespace for openstack modules will be named "openstack.". What is coming after the dot is still under discussion. 3. Current modules will be migrated to collections in "openstack." as is with their names and will be still available for playbooks (via symlinking). It will avoid breaking people that use in their playbooks os_* modules now. 4. Old modules will be frozen after migrations and all development work will go in the new modules which will live aside. 5. Critical bugfixes to 2.9 versions will be done via Ansible GitHub repo as usual and synced manually to "openstack." collection. It must be a very exceptional case. 6. Migrations are set for mid of January 2020 approximately. 7. Modules should stay compatible with last Ansible and collections API changed. 8. Because current old modules are licensed with GPL and license of Openstack is Apache2, we need to figure out if we can either relicense them or develop new ones with different license or to continue to work on new ones with GPL in SIG repo. Agreed to ask on legal-discuss ML. Long minutes: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.00.html Logs: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.00.log.html Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules Next time Thursday 12 Dec 2019 4.00 PM UTC. Thanks On Tue, Dec 3, 2019 at 8:18 PM Sagi Shnaidman wrote: > Hi, all > In the meeting today we agreed to meet every Thursday starting *this week* > at 4.00 PM UTC on #openstack-sdks channel on Freenode. We'll discuss > everything related to Openstack Ansible modules. > Agenda and topics are in the etherpad: > https://etherpad.openstack.org/p/openstack-ansible-modules > (I've created a new one, because we don't limit to Ironic modules only, > it's about all of them in general) > > Short minutes from meeting today: > Organizational: > 1. We meet every Thursday from this week at 4.00 PM UTC on #openstack-sdks > 2. Interested parties for now are: Ironic, Tripleo, Openstack-Ansible, > Kolla-ansible, OpenstackSDK teams. Feel free to join and add yourself in > the etherpad. [1] > 3. We'll track our work in Storyboard for ansible-collections-openstack > (in progress) > 4. Openstack Ansible modules will live as collections under Ansible SIG in > repo openstack/ansible-collections-openstack [2] because there are issues > with different licensing: GPLv3 for Ansible in upstream and Openstack > license (Apache2). > 5. Ansible upstream Openstack modules will be merge-frozen when we'll have > our collections fully working and will be deprecated from Ansible at some > point in the future. > 6. Openstack Ansible collections will be published to Galaxy. > 7. There is a list of people that can be pinged for reviews in > ansible-collections-openstack project, feel free to join there [1] > > Technical: > 1. We use openstacksdk instead of [project]client modules. > 2. We will rename modules to be more like os_[service_type] named, > examples are in Ironic modules etherpad [3] > > Logs from meeting today you can find here: > http://eavesdrop.openstack.org/meetings/ansible_sig/2019/ansible_sig.2019-12-03-15.01.log.html > Please feel free to participate and add topics to agenda. [1] > > [1] https://etherpad.openstack.org/p/openstack-ansible-modules > [2] https://review.opendev.org/#/c/684740/ > [3] https://etherpad.openstack.org/p/ironic-ansible-modules > > Thanks > > On Wed, Nov 27, 2019 at 7:57 PM Sagi Shnaidman > wrote: > >> Hi, all >> >> in the light of finding the new home place for openstack related ansible >> modules [1] I'd like to discuss the best strategy to create Ironic ansible >> modules. Existing Ironic modules in Ansible repo don't cover even half of >> Ironic functionality, don't fit current needs and definitely require an >> additional work. There are a few topics that require attention and better >> be solved before modules are written to save additional work. We prepared >> an etherpad [2] with all these questions and if you have ideas or >> suggestions on how it should look you're welcome to update it. >> We'd like to decide the final place for them, name conventions (the most >> complex one!), what they should look like and how better to implement. >> Anybody interested in Ansible and baremetal management in Openstack, >> you're more than welcome to contribute. >> >> Thanks >> >> [1] https://review.opendev.org/#/c/684740/ >> [2] https://etherpad.openstack.org/p/ironic-ansible-modules >> >> -- >> Best regards >> Sagi Shnaidman >> > > > -- > Best regards > Sagi Shnaidman > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtroyer at gmail.com Fri Dec 6 00:39:37 2019 From: dtroyer at gmail.com (Dean Troyer) Date: Thu, 5 Dec 2019 18:39:37 -0600 Subject: [TC] Election terms In-Reply-To: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> References: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> Message-ID: <396572bf-7451-c895-9efe-8e65d09db911@gmail.com> On 12/4/19 5:39 PM, Jeremy Stanley wrote: > Just to clarify, this is what our (recently updated) Technical > Committee Member Policy appendix to the OSF Bylaws says about TC > term durations: > > 2.a. [...] Each Technical Committee member shall hold the seat > for a term not to exceed sixteen months, but may be re-elected > to the Technical Committee. After January 1, 2019, the term for > the members of the Technical Committee shall be approved by a > majority of the Technical Committee (“Term”) and shall be > published publicly before each Technical Committee election; if > no such Term is published the Term will be twelve calendar > months. [...] "published publicly before each TC election" ... stick a pin here for below... [...] > 3. Probably only a solution for 2021 and later elections, but start > declaring now that TC seat terms are something like "14 months or > until the date of the next election" so that the TC doesn't need to > predict election timing more than a year in advance (OSF bylaws > authorize us to make it up to 16 months for that matter). I recall this being the intent when the charter was last updated. Unless I am conflating this with StarlingX where we had similar discussions when setting up governance. I think it is the "published before" language that is the real issue. The terms have never been exactly 6 or 12 months and in practice the membership changes were made soon after the close of the election. Relaxing the wording to just refer to the (roughly) 6 month election cycle as the bounds for terms achieves both the historical practice and what I have always thought was the community consensus. This way the only time that terms would expire and seats be left unfilled is if the elections were to be delayed long enough for the OSF bylaws limit of 16 months to be reached. Summary: a) Given that elections hare held (roughly) every 6 months for all PTLs and approximately half of the TC; b) Define terms as running from the close and certification of an elections results until the close and certification of the next applicable election for a given seat, ie the next election about 6 months later for PTL terms and the second-following election about 12 months later for TC terms. dt -- Dean Troyer dtroyer at gmail.com From Albert.Braden at synopsys.com Fri Dec 6 02:07:59 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 6 Dec 2019 02:07:59 +0000 Subject: Rebooted VM disappears when HV lacks memory In-Reply-To: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> References: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> Message-ID: We changed cpu_allocation_ratio and ram_allocation_ratio to 1 on the hypervisors. Now I'm trying to clean up the mess. I'm trying to cold-migrate dead VMs from an oversubscribed hypervisor, but they are failing to migrate: root at us01odc-p02-ctrl1:~# os hypervisor list --long|grep hv199 | 218 | us01-p02-hv199.internal.synopsys.com | QEMU | 10.195.54.226 | up | 138 | 80 | 1122304 | 772691 | root at us01odc-p02-ctrl1:~# openstack server migrate fc250e76-4ca5-44b1-a6a0-8a7c7ad48c24 No valid host was found. No valid host found for cold migrate (HTTP 400) (Request-ID: req-cc048221-abaf-4be0-bc90-f28eecec3df5) But we have lots of hosts with space, and if I build a new VM with the same flavor it works. When I check the logs, it appears that the migration fails because the hypervisor we are moving from is oversubscribed: /var/log/nova/nova-conductor.log:2019-12-05 17:30:17.163 48989 WARNING nova.scheduler.client.report [req-cc048221-abaf-4be0-bc90-f28eecec3df5 5b288691527245bda715ab7744a193e9 deea2d8541f741eda6fb0d242d16bb23 - default 824941b2ad6a43d7984fab9390290f18] Unable to post allocations for instance 87608240-1ad2-45fc-b4f0-c118e3bf0262 (409 {"errors": [{"status": 409, "request_id": "req-e56d29f8-c97e-41ac-8481-0598f3c681b2", "detail": "There was a conflict when trying to complete your request.\n\n Unable to allocate inventory: Unable to create allocation for 'MEMORY_MB' on resource provider 'fde477cd-fd3d-4296-838c-f9c54b3e97ef'. The requested amount would exceed the capacity. ", "title": "Conflict"}]}) 'fde477cd-fd3d-4296-838c-f9c54b3e97ef' is the UUID of hv199: root at us01odc-p02-ctrl1:~# curl -s -X GET -H "X-Auth-Token: $token" -H "Content-Type: application/json" "http://us01odc-p02-lb.internal.synopsys.com:28778/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef";echo {"generation": 40, "uuid": "fde477cd-fd3d-4296-838c-f9c54b3e97ef", "links": [{"href": "/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef", "rel": "self"}, {"href": "/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef/inventories", "rel": "inventories"}, {"href": "/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef/usages", "rel": "usages"}], "name": "us01-p02-hv199.internal.synopsys.com"} What am I doing wrong? Do I need to move the live VMs away from hv199 to make room for the dead ones? -----Original Message----- From: Sean Mooney Sent: Wednesday, December 4, 2019 4:30 AM To: Albert Braden ; openstack-discuss at lists.openstack.org Subject: Re: Rebooted VM disappears when HV lacks memory On Wed, 2019-12-04 at 00:51 +0000, Albert Braden wrote: > We are running Rocky and we have the allocation bug because we set cpu_allocation_ratio and ram_allocation_ratio to 1 > on the controllers but not on the hypervisors, so the hypervisors are oversubscribed. When we try to reboot a VM when > the HV is short memory, the VM is deleted from the HV. It still exists in Openstack but not on the HV. Is it possible > to recover a VM from this state? so the vm is likely not deleted the harddisk and the libvirt domain will still be defiend. my guess is starting the vm cause the oom reaper to kill the qemu process when it was relaunched. if you are deploying with memory over subsription enabled you should configre enough swap space equal to total memory* ram_allocation_ratio. if you do that you you should be able to start the vm. now the other option is while the vm is stopped you can cold migrate it to another host and then start it there. From Albert.Braden at synopsys.com Fri Dec 6 02:22:14 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 6 Dec 2019 02:22:14 +0000 Subject: Rebooted VM disappears when HV lacks memory In-Reply-To: References: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> Message-ID: Update: It looks like attempting to migrate any VM, alive or dead, from an oversubscribed hypervisor throws the same error. Maybe I should have done the migrations before setting cpu_allocation_ratio and ram_allocation_ratio. Is there a way to recover from this besides deleting VMs from the oversubscribed hypervisors? -----Original Message----- From: Albert Braden Sent: Thursday, December 5, 2019 6:08 PM To: openstack-discuss at lists.openstack.org Subject: RE: Rebooted VM disappears when HV lacks memory We changed cpu_allocation_ratio and ram_allocation_ratio to 1 on the hypervisors. Now I'm trying to clean up the mess. I'm trying to cold-migrate dead VMs from an oversubscribed hypervisor, but they are failing to migrate: root at us01odc-p02-ctrl1:~# os hypervisor list --long|grep hv199 | 218 | us01-p02-hv199.internal.synopsys.com | QEMU | 10.195.54.226 | up | 138 | 80 | 1122304 | 772691 | root at us01odc-p02-ctrl1:~# openstack server migrate fc250e76-4ca5-44b1-a6a0-8a7c7ad48c24 No valid host was found. No valid host found for cold migrate (HTTP 400) (Request-ID: req-cc048221-abaf-4be0-bc90-f28eecec3df5) But we have lots of hosts with space, and if I build a new VM with the same flavor it works. When I check the logs, it appears that the migration fails because the hypervisor we are moving from is oversubscribed: /var/log/nova/nova-conductor.log:2019-12-05 17:30:17.163 48989 WARNING nova.scheduler.client.report [req-cc048221-abaf-4be0-bc90-f28eecec3df5 5b288691527245bda715ab7744a193e9 deea2d8541f741eda6fb0d242d16bb23 - default 824941b2ad6a43d7984fab9390290f18] Unable to post allocations for instance 87608240-1ad2-45fc-b4f0-c118e3bf0262 (409 {"errors": [{"status": 409, "request_id": "req-e56d29f8-c97e-41ac-8481-0598f3c681b2", "detail": "There was a conflict when trying to complete your request.\n\n Unable to allocate inventory: Unable to create allocation for 'MEMORY_MB' on resource provider 'fde477cd-fd3d-4296-838c-f9c54b3e97ef'. The requested amount would exceed the capacity. ", "title": "Conflict"}]}) 'fde477cd-fd3d-4296-838c-f9c54b3e97ef' is the UUID of hv199: root at us01odc-p02-ctrl1:~# curl -s -X GET -H "X-Auth-Token: $token" -H "Content-Type: application/json" "http://us01odc-p02-lb.internal.synopsys.com:28778/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef";echo {"generation": 40, "uuid": "fde477cd-fd3d-4296-838c-f9c54b3e97ef", "links": [{"href": "/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef", "rel": "self"}, {"href": "/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef/inventories", "rel": "inventories"}, {"href": "/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef/usages", "rel": "usages"}], "name": "us01-p02-hv199.internal.synopsys.com"} What am I doing wrong? Do I need to move the live VMs away from hv199 to make room for the dead ones? -----Original Message----- From: Sean Mooney Sent: Wednesday, December 4, 2019 4:30 AM To: Albert Braden ; openstack-discuss at lists.openstack.org Subject: Re: Rebooted VM disappears when HV lacks memory On Wed, 2019-12-04 at 00:51 +0000, Albert Braden wrote: > We are running Rocky and we have the allocation bug because we set cpu_allocation_ratio and ram_allocation_ratio to 1 > on the controllers but not on the hypervisors, so the hypervisors are oversubscribed. When we try to reboot a VM when > the HV is short memory, the VM is deleted from the HV. It still exists in Openstack but not on the HV. Is it possible > to recover a VM from this state? so the vm is likely not deleted the harddisk and the libvirt domain will still be defiend. my guess is starting the vm cause the oom reaper to kill the qemu process when it was relaunched. if you are deploying with memory over subsription enabled you should configre enough swap space equal to total memory* ram_allocation_ratio. if you do that you you should be able to start the vm. now the other option is while the vm is stopped you can cold migrate it to another host and then start it there. From brett.milford at canonical.com Fri Dec 6 04:19:45 2019 From: brett.milford at canonical.com (Brett Milford) Date: Fri, 6 Dec 2019 15:19:45 +1100 Subject: [nova][cinder][ops] question/confirmation of legacy vol attachment migration In-Reply-To: References: <37e953ee-f3c8-9797-446f-f3e3db9dcad6@gmail.com> <20191010100050.hn546tikeihaho7e@localhost> <20191017102419.pa3qqlqgrlp2b7qx@localhost> Message-ID: On Fri, Nov 22, 2019 at 3:54 AM Matt Riedemann wrote: > > On 10/17/2019 5:24 AM, Gorka Eguileor wrote: > > I stand by my initial recommendation, being able to update the existing > > attachment to add the connection information from Nova. > > OK, thanks for the input and thoughtfulness on this. I've abandoned my > change since I'm not going to be pushing this boulder anymore but left > notes in the change in case someone else wants to pick it up some day. > > Note to nova cores: this means we'll have legacy volume attachment > compat code around forever. > > -- > > Thanks, > > Matt > Hi Groka, Cinder & Nova devs, Following up this thread from the context of https://review.opendev.org/#/c/579004/ To summarise the discussion: - initialize_connection shouldn't be called more than once, as it may mess up some drivers. - To (safely) refresh connection_info a new api for cinder is required. - a patch to nova, such as #579004 could make a call to this new api to refresh connection info on events such as reboot. Is there any other context to this issue I've missed or an alternate path to solving this? Does the creation of a new api for cinder have any implications for being able to backport this patch for nova? What would be the process for me to kick off this work? Thanks for your help, Brett (bcm) -- ❯ brett From kennelson11 at gmail.com Fri Dec 6 07:15:45 2019 From: kennelson11 at gmail.com (Kendall Nelson) Date: Thu, 5 Dec 2019 23:15:45 -0800 Subject: [TC] 'V' Technical Elections In-Reply-To: References: Message-ID: Hello Everyone! To summarize the TC office hours discussions today[1]: - The majority of folks are in favor of a combined election - With the combined election dates above, those TC members seat that will be up for reelection will hit their term limite before we conclude the election so there will be a period of time where the TC will only be 6 members until the election concludes. - In an effor to avoid this expiration and gap in having a full TC in the future, we want to amend the charter to say something about term limits being '14 months or when the second election since being elected concludes' or something like that-- theres another thread for that discussion '[TC] Election Terms' - We will need to move the extra-atc deadline forward so that it is during the week of nominations so that they can be included in the electorates I think that covers all of it. If I missed anything, please feel free to expand :) If we, the election officials don't hear anything by the end of next week, we will move froward updating the election site and the release schedule. -Kendall [1] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-12-05.log.html#t2019-12-05T15:46:11 On Wed, Dec 4, 2019 at 3:01 PM Kendall Nelson wrote: > Hello! > > /me takes TC hat off and puts election official hat on > > Having run our handy dandy tool to project dates[1] for the elections, we > have some decisions to make again. > > At this point we have threeish options: > > 1. Run the elections with a week in between with the TC election happening > BEFORE the PTL election like the script generated. The TC election would > run March 10 to March 31 and the PTL Election would run April 7 to April > 21. The primary concern here is that it might lead to some election > burnout. > > 2. Run the elections back to back with the PTL election first (closer to > the historic norm). This would be tweaking the generated dates by pulling > the PTL election three weeks up and pushing back the TC dates by two weeks. > The PTL election would run March 3 to March 17 and the PTL Election would > run March 17 to April 7. Both elections would be concluded by milestone 3. > Similar concerns with this option to the first with election burnout. > > 3. Run the elections in parallel as a combined election, similar to how we > did for the Train elections. This is much easier for election officials > because we only need to generate one electorate. The two elections would > start March 10 and be done by March 31 following the TC election model with > a nominations period, a campaigning period (optional for PTL elections), > and then a polling period (assuming we need a runoff). > > Other info to keep in mind when formulating your opinions: > - Extra ATC deadline currently is March 23-27 which will likely need to > get bumped earlier for any of these options > - RC1 is the week of April 20-24 > - Milestone 3 is April 9 > - Release schedule for reference[2] > - For TC member terms, those closing in on their ~1 year terms would be > closer to 13 months since the train elections concluded March 5th of last > year > > -Kendall (diablo_rojo) & the Election Officials > > [1]http://paste.openstack.org/show/787114/ > [2]https://releases.openstack.org/ussuri/schedule.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Fri Dec 6 10:10:34 2019 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 6 Dec 2019 10:10:34 +0000 Subject: [ptl] Train cycle-trailing release deadline In-Reply-To: <20191205170425.GA571363@sm-workstation> References: <20191205170425.GA571363@sm-workstation> Message-ID: On Thu, 5 Dec 2019 at 17:05, Sean McGinnis wrote: > > Hello all, > > For those teams with cycle-trailing deliverables, this is a reminder to make > sure to get your Train series deliverables released. We like to have all of > those out within three months of the coordinated release. > > For reference, here is a list of deliverables we have marked as Train > cycle-trailing in the releases repo that do not appear to have a final release: > > cinderlib > kayobe > kolla-ansible > kolla-cli > kolla > openstack-ansible > tripleo-ui Thanks for the reminder. We (kolla) are about to propose some RC2s, which I expect to become GA. Kayobe is a little further behind, but I think we can finish it off in the next couple of weeks. > > If these, or any others, need to be added or changed for Ussuri, please propose > a placeholder deliverable file under deliverables/ussuri or update any existing > ones with any needed changes. > > Thanks! > Sean > From amotoki at gmail.com Fri Dec 6 11:18:15 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Fri, 6 Dec 2019 20:18:15 +0900 Subject: [horizon] Shanghai PTG recap Message-ID: Hi, I am summarizing the horizon PTG in Shanghai. One month passed from the PTG, but I believe the summary is still useful :-) Eight folks joined the session and we had productive discussions. Thanks all! - PTG etherpad https://etherpad.openstack.org/p/horizon-u-ptg - release priority etherpad https://etherpad.openstack.org/p/horizon-release-priorities [Priorities for Ussuri] After the PTG, we discussed the project priorities for Ussuri based on the PTG discussion and agreed the following as project priorities. We will cover them in the weekly team meetings. * High * Catch up the new policy mechanism * Bump and clean up Django versions * Error message refactoring * Medium * xstatic updates * ini-based-configuration * Chinese translation handling [PTG recap] * Train retrospective * To keep * We succeeded to restore integration tests in horizon and several plugins. They just cover basic GET scenarios. Most JS tests do not cover scenario testing so it would be a good step to avoid plugin breakage. * We can keep the project healthiness to some level regardless of the low number of contributors. * To improve * The bundle policy files in horizon is out of date from back-end services and there was no progress since Stein release. For example, the new policies (system scope, deprecated policies) from the keystone repo does not work with horizon. It should be priorities in Ussuri release. We will join the pop-up team for the policy enhancement. * Some community goal from queens has not completed. (Most of them have done.) * Supported versions of Django and Python * We discussed the plan to enable Django 2.2 (the current LTS) by default and drop Django 1.11 support. The current plan is to bump the default Django to 2.2 around milestone-1 and drop Django 1.11 support by milestone-2. The minimum goal is to support Django 2.2 as it will be the only LTS supported by Django community as of Ussuri release. * The version changes in Django may break horizon plugins so we need enough time to test them in real deployments. * Integration test coverage would help the transition of Django version and let's try integration tests more with our best. * X-Static updates * Many xstatic packages are not updated for long. They has has fixes including security fixes, so we need to update them. radomir volunteered for the effort. * JS loading * e0ne shared his experiences on his trial to implement a plugin using react. * The problem is that it is not easy to use newer JS libraries. An example of problmes with a current approach is https://review.opendev.org/#/c/652657/ * It comes from the gap between the xstatic way to use same versions for all JS applications and modern JS packaging like webpack. * There is no good conclusion in this topic, but a long-term goal would be to use modern packaging mechanism instead of xstatic. It could be an issue for distribution packagers. * Horizon plugins maintenance * This slot covered topics around horizon plugin ecosystem and their maintenance. * The main outcomes are: * we agreed to change the release model of horizon to cycle-with-intermediary. Horizon plugins use horizon as library and we need to release horizon promptly whenever we have changes which affect plugins. cycle-with-intermediary model would fit more for this purpose. * A concern on that multiple major version bump potentially happens was raised, but we agreed that the version number does not matter and the best practice to avoid this is to drop deprecated features at the beginning of the cycle. * horizon team asked individual project teams with horizon plugins to give +2 right to horizon cores to make plugin maintenance smooth. * Most team agreed and we had conversations with more team at the PTG. * One important note is that horizon team should shouldn’t merge anything to plugins without +2 from plugin team. * We will update our documentation to cover this change in the plugin maintenance and e0ne volunteered for it. * Some side-discussion happened. There was no decision but they might be worth considering in future. * An idea to split nova/cinder/neutron/keystone/.... support to plugins and make them mandatory dependencies. * Use of micro-frontend concept * Chinese translation handling * From horizon perspective we agreed that we would like to move to the new locale names (zh-hans/hant) both in horizon (+plugins). * The main questions are the transition plan and locales for documentations. Further discussion with the i18n team happened in Shanghai. The summary of the discussion will be coming soon, but the first step agreed is that the horizon team confirms the new locales work fine in horizon. * Feature Gaps * Brief discussions happened for features with volunteers. * We will visit the status of the gaps in the etherpad https://etherpad.openstack.org/p/horizon-feature-gap * Refactor Error Messages * We received a good number of bug reports which complain that horizon error messages are too general and hide error details. * We agree to show both error message summary (translated) and detail message from API. * An idea discussed is to have a collapsible box to show an error detail along with a summary message. This approach would bring us a good balance of translated message and error detail from back-end services. * It would be a nice improvement in Ussuri. * Deprecations and Removals * Short discussion covered potential feature deprecation and removals. * Cinder v2 support will be dropped soon. * ini-based-configuration * amotoki expressed that he continues the effort and hopefully completes it in Ussuri. * API versions of backend services * horizon uses the minimum microversions to communicate with back-end services unless new versions are required to support features. * The possibility to bump the minimum versions used (per cycle?) was discussed and we agreed to keep the current strategy considering dev resources and maintenance. * Adoption of openstacksdk was discussed. openstacksdk supports older APIs and it would help us keep the current strategy on API versions used. * To use openstacksdk, we need to explore how to instantiate keystonauth Auth object with a token (as horizon stores only token and does not store auth information). We can start the investigation on this and e0ne volunteered. That's all we discussed in Shanghai. I hope we will have good progress in Ussuri. Best Regards, Akihiro Motoki (irc: amotoki) From sean.mcginnis at gmx.com Fri Dec 6 12:53:31 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Fri, 6 Dec 2019 06:53:31 -0600 Subject: [TC] Election terms In-Reply-To: <396572bf-7451-c895-9efe-8e65d09db911@gmail.com> References: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> <396572bf-7451-c895-9efe-8e65d09db911@gmail.com> Message-ID: <20191206125331.GA670935@sm-workstation> On Thu, Dec 05, 2019 at 06:39:37PM -0600, Dean Troyer wrote: > On 12/4/19 5:39 PM, Jeremy Stanley wrote: > > Just to clarify, this is what our (recently updated) Technical > > Committee Member Policy appendix to the OSF Bylaws says about TC > > term durations: > > > > 2.a. [...] Each Technical Committee member shall hold the seat > > for a term not to exceed sixteen months, but may be re-elected > > to the Technical Committee. After January 1, 2019, the term for > > the members of the Technical Committee shall be approved by a > > majority of the Technical Committee (“Term”) and shall be > > published publicly before each Technical Committee election; if > > no such Term is published the Term will be twelve calendar > > months. [...] > > "published publicly before each TC election" ... stick a pin here for > below... > > [...] > > > 3. Probably only a solution for 2021 and later elections, but start > > declaring now that TC seat terms are something like "14 months or > > until the date of the next election" so that the TC doesn't need to > > predict election timing more than a year in advance (OSF bylaws > > authorize us to make it up to 16 months for that matter). > > I recall this being the intent when the charter was last updated. Unless I > am conflating this with StarlingX where we had similar discussions when > setting up governance. > That was my recollection as well. I think we had "not to exceed 16 months" as a way to have some flexibility because we knew that generally terms would be slightly longer than 12 months. The later phrasing of "will be twelve months" is the tricky part here, since that is what we ended up publishing. I don't recall it being our intent to have a strict 12 month term. From sean.mcginnis at gmx.com Fri Dec 6 12:56:23 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Fri, 6 Dec 2019 06:56:23 -0600 Subject: [TC] 'V' Technical Elections In-Reply-To: References: Message-ID: <20191206125623.GB670935@sm-workstation> On Thu, Dec 05, 2019 at 11:15:45PM -0800, Kendall Nelson wrote: > Hello Everyone! > > To summarize the TC office hours discussions today[1]: > > - The majority of folks are in favor of a combined election > - With the combined election dates above, those TC members seat that will > be up for reelection will hit their term limite before we conclude the > election so there will be a period of time where the TC will only be 6 > members until the election concludes. > - In an effor to avoid this expiration and gap in having a full TC in the > future, we want to amend the charter to say something about term limits > being '14 months or when the second election since being elected concludes' > or something like that-- theres another thread for that discussion '[TC] > Election Terms' Rather than stating an exact term length and running the risk of ending up in a similar situation in the future, I would recommend just stating term limits will be 'a minimum of 12 months not to exceed 16 months, based on when the next election aligns with overall community activities.' From thierry at openstack.org Fri Dec 6 13:46:57 2019 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 6 Dec 2019 14:46:57 +0100 Subject: email alert settings are not applied (ask.openstack.org) In-Reply-To: <20191205152524.Horde.7IoBJehsJP-TNbVJXDxm3QV@webmail.nde.ag> References: <20191205152524.Horde.7IoBJehsJP-TNbVJXDxm3QV@webmail.nde.ag> Message-ID: <2a9546e1-d88f-c82b-c8f6-7cd067b7641d@openstack.org> Eugen Block wrote: > I'm not really sure if this is the right place to ask but I'll try it > anyway. I'm regularly checking ask.openstack.org for interesting > questions and sometimes I comment and respond. In my profile I > configured instant email notification if someone comments or responds to > my questions/comments. But this has never worked properly, I think this > worked just once two or three years ago, never again. The only setting > that seems to work is the weekly notification about new questions. Can > anyone comment on this? Should I try to change the email address and see > if that works? Hi Eugen! You're right, there is some issue in our ask.o.o notifications that renders them useless. Ask.o.o is in a sad state right now due to lack of maintenance and interest in our community. This has prompted several discussions on the openstack-infra list in the past about dropping it, but so far no definitive action... Pointer to last discussion there: http://lists.openstack.org/pipermail/openstack-infra/2019-November/006503.html -- Thierry Carrez (ttx) From eblock at nde.ag Fri Dec 6 14:00:22 2019 From: eblock at nde.ag (Eugen Block) Date: Fri, 06 Dec 2019 14:00:22 +0000 Subject: email alert settings are not applied (ask.openstack.org) In-Reply-To: <2a9546e1-d88f-c82b-c8f6-7cd067b7641d@openstack.org> References: <20191205152524.Horde.7IoBJehsJP-TNbVJXDxm3QV@webmail.nde.ag> <2a9546e1-d88f-c82b-c8f6-7cd067b7641d@openstack.org> Message-ID: <20191206140022.Horde.i2W7CC0HIIPlFjvvu63ndRr@webmail.nde.ag> Thank you, Thierry! I didn't know ask.o.o was in such a state, sad to hear it. But thank you for the clarification. Regards, Eugen Zitat von Thierry Carrez : > Eugen Block wrote: >> I'm not really sure if this is the right place to ask but I'll try >> it anyway. I'm regularly checking ask.openstack.org for interesting >> questions and sometimes I comment and respond. In my profile I >> configured instant email notification if someone comments or >> responds to my questions/comments. But this has never worked >> properly, I think this worked just once two or three years ago, >> never again. The only setting that seems to work is the weekly >> notification about new questions. Can anyone comment on this? >> Should I try to change the email address and see if that works? > > Hi Eugen! > > You're right, there is some issue in our ask.o.o notifications that > renders them useless. Ask.o.o is in a sad state right now due to > lack of maintenance and interest in our community. This has prompted > several discussions on the openstack-infra list in the past about > dropping it, but so far no definitive action... > > Pointer to last discussion there: > > http://lists.openstack.org/pipermail/openstack-infra/2019-November/006503.html > > -- > Thierry Carrez (ttx) From mriedemos at gmail.com Fri Dec 6 14:29:51 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 6 Dec 2019 08:29:51 -0600 Subject: [nova][cinder][ops] question/confirmation of legacy vol attachment migration In-Reply-To: References: <37e953ee-f3c8-9797-446f-f3e3db9dcad6@gmail.com> <20191010100050.hn546tikeihaho7e@localhost> <20191017102419.pa3qqlqgrlp2b7qx@localhost> Message-ID: <86708731-997b-da03-1d70-543d41172dd2@gmail.com> On 12/5/2019 10:19 PM, Brett Milford wrote: > Does the creation of a new api for cinder have any implications for > being able to backport this patch for nova? The nova code would need to be tolerant of the cinder API microversion not being available. That could probably use the nova.volume.cinder.is_microversion_supported method. The Cinder API change would not be backportable though. > > What would be the process for me to kick off this work? I'm assuming you'd start with a cinder spec since it's a versioned API change over there. -- Thanks, Matt From dtroyer at gmail.com Fri Dec 6 14:58:00 2019 From: dtroyer at gmail.com (Dean Troyer) Date: Fri, 6 Dec 2019 08:58:00 -0600 Subject: [TC] 'V' Technical Elections In-Reply-To: <20191206125623.GB670935@sm-workstation> References: <20191206125623.GB670935@sm-workstation> Message-ID: On 12/6/19 6:56 AM, Sean McGinnis wrote: > On Thu, Dec 05, 2019 at 11:15:45PM -0800, Kendall Nelson wrote: >> Hello Everyone! >> >> To summarize the TC office hours discussions today[1]: >> >> - The majority of folks are in favor of a combined election >> - With the combined election dates above, those TC members seat that will >> be up for reelection will hit their term limite before we conclude the >> election so there will be a period of time where the TC will only be 6 >> members until the election concludes. >> - In an effor to avoid this expiration and gap in having a full TC in the >> future, we want to amend the charter to say something about term limits >> being '14 months or when the second election since being elected concludes' >> or something like that-- theres another thread for that discussion '[TC] >> Election Terms' > > Rather than stating an exact term length and running the risk of ending up in a > similar situation in the future, I would recommend just stating term limits > will be 'a minimum of 12 months not to exceed 16 months, based on when the next > election aligns with overall community activities.' The point I was making in another fork of this thread was to tie the exact term length to the election timing and only state approximate lengths here, ie including 'minimum' in the above will still set a bound that complicates accommodating another slightly smaller than usual cycle. dt -- Dean Troyer dtroyer at gmail.com From jdratlif at iu.edu Fri Dec 6 15:06:52 2019 From: jdratlif at iu.edu (Ratliff, John) Date: Fri, 6 Dec 2019 15:06:52 +0000 Subject: openstack cli server stop command Message-ID: <6725eae2c1a5409689d703888dac6e2f@IN-CCI-D1S14.ads.iu.edu> Does the openstack server stop command do an ACPI shutdown or does it do the equivalent of pulling the power plug? I want to know if it's safe to run this command rather than logging into the machine and running shutdown (for say a RHEL7 instance). If it does an ACPI shutdown, what happens if the machine doesn't respond? Is there a command to force the machine off? Thanks. --John -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5696 bytes Desc: not available URL: From gmann at ghanshyammann.com Fri Dec 6 15:08:19 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 06 Dec 2019 09:08:19 -0600 Subject: [TC] 'V' Technical Elections In-Reply-To: References: Message-ID: <16edbc128d0.d341dcf8109275.7298476469707499137@ghanshyammann.com> ---- On Fri, 06 Dec 2019 01:15:45 -0600 Kendall Nelson wrote ---- > Hello Everyone! > > To summarize the TC office hours discussions today[1]: > - The majority of folks are in favor of a combined election- With the combined election dates above, those TC members seat that will be up for reelection will hit their term limite before we conclude the election so there will be a period of time where the TC will only be 6 members until the election concludes. - In an effor to avoid this expiration and gap in having a full TC in the future, we want to amend the charter to say something about term limits being '14 months or when the second election since being elected concludes' or something like that-- theres another thread for that discussion '[TC] Election Terms'- We will need to move the extra-atc deadline forward so that it is during the week of nominations so that they can be included in the electorates > I think that covers all of it. If I missed anything, please feel free to expand :) I think that is all. During the half-tc period, we will not be able to merge any charter change which should be ok. -gmann > If we, the election officials don't hear anything by the end of next week, we will move froward updating the election site and the release schedule. > -Kendall > [1] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-12-05.log.html#t2019-12-05T15:46:11 > On Wed, Dec 4, 2019 at 3:01 PM Kendall Nelson wrote: > Hello! > > /me takes TC hat off and puts election official hat on > > Having run our handy dandy tool to project dates[1] for the elections, we have some decisions to make again. > > At this point we have threeish options: > > 1. Run the elections with a week in between with the TC election happening BEFORE the PTL election like the script generated. The TC election would run March 10 to March 31 and the PTL Election would run April 7 to April 21. The primary concern here is that it might lead to some election burnout. > > 2. Run the elections back to back with the PTL election first (closer to the historic norm). This would be tweaking the generated dates by pulling the PTL election three weeks up and pushing back the TC dates by two weeks. The PTL election would run March 3 to March 17 and the PTL Election would run March 17 to April 7. Both elections would be concluded by milestone 3. Similar concerns with this option to the first with election burnout. > > 3. Run the elections in parallel as a combined election, similar to how we did for the Train elections. This is much easier for election officials because we only need to generate one electorate. The two elections would start March 10 and be done by March 31 following the TC election model with a nominations period, a campaigning period (optional for PTL elections), and then a polling period (assuming we need a runoff). > > Other info to keep in mind when formulating your opinions: > - Extra ATC deadline currently is March 23-27 which will likely need to get bumped earlier for any of these options > - RC1 is the week of April 20-24 > - Milestone 3 is April 9 > - Release schedule for reference[2] > - For TC member terms, those closing in on their ~1 year terms would be closer to 13 months since the train elections concluded March 5th of last year > -Kendall (diablo_rojo) & the Election Officials > > [1]http://paste.openstack.org/show/787114/ > [2]https://releases.openstack.org/ussuri/schedule.html > From jdratlif at iu.edu Fri Dec 6 15:22:17 2019 From: jdratlif at iu.edu (Ratliff, John) Date: Fri, 6 Dec 2019 15:22:17 +0000 Subject: [External] Re: openstack volume snapshots and deleting In-Reply-To: <20191205185847.GA579008@sm-workstation> References: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> <20191205175816.GA576175@sm-workstation> <9e811d2eda9a41cf99444c0a58c5bbf4@IN-CCI-D1S14.ads.iu.edu> <20191205185847.GA579008@sm-workstation> Message-ID: Creating a volume from a volume with ceph, at least in our configuration (which I don't have any authority to change) will not create full volume, but rather a COW volume. However, I can use rbd flatten on the image, and then delete the snapshot. It takes time to copy the image if they're large, which makes sense as to why it doesn't do this by default. But that is a reasonable workaround. Thanks. --John -----Original Message----- From: Sean McGinnis Sent: Thursday, December 5, 2019 1:59 PM To: Ratliff, John Cc: openstack-discuss at lists.openstack.org Subject: Re: [External] Re: openstack volume snapshots and deleting On Thu, Dec 05, 2019 at 06:25:28PM +0000, Ratliff, John wrote: > Yes, I tried this second thing, but it doesn't work either. > > I created a new instance called test1.example.net. > I created a volume snapshot (test1.example.net_ss1) I created a volume > from that snapshot (vol_from_ss) I created a volume from the volume > that was from the snapshot > (vol_from_vol_from_ss) > I deleted the vol_from_ss (this works) I deleted the snapshot (this > does not work) > Looking at the RBD driver, it looks like it does not actually follow the expected clone semantics. Unless rbd_max_clone is set to 0, or the number of clones is less than whatever that is set to, it will actually do a COW volume on top of the snapshot. This is a performance optimization (it doesn't have to copy all of the data into the new volume), but is actually a bug, IMO. You can try changing that config option and restarting services to pick that up, then trying again. Otherwise, it looks like that drivers clone handling will create its own snapshot of a volume. So unless you need to keep that snapshot around for some other reason, it looks like you should be able to just: openstack volume create --source orig-volume new-volume And if I follow it right, it will create a snapshot under the covers, then use that to create the new volume. Hopefully one of those helps. Sean -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5696 bytes Desc: not available URL: From mriedemos at gmail.com Fri Dec 6 15:25:09 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 6 Dec 2019 09:25:09 -0600 Subject: openstack cli server stop command In-Reply-To: <6725eae2c1a5409689d703888dac6e2f@IN-CCI-D1S14.ads.iu.edu> References: <6725eae2c1a5409689d703888dac6e2f@IN-CCI-D1S14.ads.iu.edu> Message-ID: On 12/6/2019 9:06 AM, Ratliff, John wrote: > Does the openstack server stop command do an ACPI shutdown or does it do > the equivalent of pulling the power plug? > > I want to know if it’s safe to run this command rather than logging into > the machine and running shutdown (for say a RHEL7 instance). > > If it does an ACPI shutdown, what happens if the machine doesn’t > respond? Is there a command to force the machine off? > The client-side command doesn't have any say in that. On the server side, it depends. There is an image property "os_shutdown_timeout" which can override the server-side configured graceful shutdown timeout - which defaults to 60 seconds. So if you want to control that timeout in your images you can use the image property. See [1] for details. [1] https://docs.openstack.org/glance/latest/admin/useful-image-properties.html -- Thanks, Matt From mriedemos at gmail.com Fri Dec 6 15:28:45 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 6 Dec 2019 09:28:45 -0600 Subject: openstack cli server stop command In-Reply-To: References: <6725eae2c1a5409689d703888dac6e2f@IN-CCI-D1S14.ads.iu.edu> Message-ID: On 12/6/2019 9:25 AM, Matt Riedemann wrote: > which can override the server-side configured graceful shutdown timeout > - which defaults to 60 seconds. These are the config options in the nova-compute service: https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.shutdown_timeout https://docs.openstack.org/nova/latest/configuration/config.html#compute.shutdown_retry_interval -- Thanks, Matt From fungi at yuggoth.org Fri Dec 6 15:45:53 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 6 Dec 2019 15:45:53 +0000 Subject: [TC] Election terms In-Reply-To: <20191206125331.GA670935@sm-workstation> References: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> <396572bf-7451-c895-9efe-8e65d09db911@gmail.com> <20191206125331.GA670935@sm-workstation> Message-ID: <20191206154553.gc2yf437smylndqm@yuggoth.org> On 2019-12-06 06:53:31 -0600 (-0600), Sean McGinnis wrote: [...] > The later phrasing of "will be twelve months" is the tricky part > here, since that is what we ended up publishing. I don't recall it > being our intent to have a strict 12 month term. Yep, that's really the present challenge. The TC needs to have something in the charter which declares a longer (or maybe also shorter) possible term length. Otherwise they get the default "12 months" from the bylaws. They *can* work with the board to change that part of the bylaws more easily now, it no longer needs scheduling a vote of the OSF members and nail-biting over getting sufficient voter turnout at least, but it does still mean board meetings and similar lengthy process (compared to bylaws changes which the TC can in theory propose and approve within a week or so if they're quick to register their votes on it). So anyway, there's the rub, if the TC can fix this before March then the members elected in 2020 Q1 will not be subject to the same scheduling complications when their terms expire in 2021 Q1, but we're still going to have to deal with this problem through both 2020 elections I suspect. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Fri Dec 6 15:51:48 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 6 Dec 2019 15:51:48 +0000 Subject: [TC] 'V' Technical Elections In-Reply-To: <16edbc128d0.d341dcf8109275.7298476469707499137@ghanshyammann.com> References: <16edbc128d0.d341dcf8109275.7298476469707499137@ghanshyammann.com> Message-ID: <20191206155148.v4biqyn4nnatqff5@yuggoth.org> On 2019-12-06 09:08:19 -0600 (-0600), Ghanshyam Mann wrote: [...] > I think that is all. During the half-tc period, we will not be > able to merge any charter change which should be ok. [...] That is true if using a conservative interpretation of the TC Charter's wording, which I think is appropriate to err on the side of caution. However it *does* then follow that any changes needed in the charter to address scheduling challenges for 2021 elections must have their voting finalized before those other 7 members' terms expire. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From fungi at yuggoth.org Fri Dec 6 15:58:30 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 6 Dec 2019 15:58:30 +0000 Subject: [TC] Election terms In-Reply-To: <20191206154553.gc2yf437smylndqm@yuggoth.org> References: <20191204233931.h5awiuqa7bi2fp5d@yuggoth.org> <396572bf-7451-c895-9efe-8e65d09db911@gmail.com> <20191206125331.GA670935@sm-workstation> <20191206154553.gc2yf437smylndqm@yuggoth.org> Message-ID: <20191206155830.owwpsgk36faaxjzq@yuggoth.org> On 2019-12-06 15:45:53 +0000 (+0000), Jeremy Stanley wrote: [...] > They *can* work with the board to change that part of the bylaws > more easily now, it no longer needs scheduling a vote of the OSF > members and nail-biting over getting sufficient voter turnout at > least, but it does still mean board meetings and similar lengthy > process (compared to bylaws changes which the TC can in theory > propose and approve within a week or so if they're quick to > register their votes on it). [...] Just for clarification, I meant to say "...compared to CHARTER changes which the TC can in theory propose and approve within a week or so..." -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From mattia.belluco at uzh.ch Fri Dec 6 16:58:27 2019 From: mattia.belluco at uzh.ch (Mattia Belluco) Date: Fri, 6 Dec 2019 17:58:27 +0100 Subject: [External] Re: openstack volume snapshots and deleting In-Reply-To: References: <9a1f2d44a79d4517be7ab74528fdea7e@IN-CCI-D1S14.ads.iu.edu> <20191205175816.GA576175@sm-workstation> <9e811d2eda9a41cf99444c0a58c5bbf4@IN-CCI-D1S14.ads.iu.edu> <20191205185847.GA579008@sm-workstation> Message-ID: To avoid endless COW chains you could also set: rbd_flatten_volume_from_snapshot = True in your cinder.conf. Of course you have to be fine with all volumes created from snapshots to be flattened: depending on the size the volume creation process can require a sizable amount of time. Best, Mattia On 12/6/19 4:22 PM, Ratliff, John wrote: > Creating a volume from a volume with ceph, at least in our configuration > (which I don't have any authority to change) will not create full volume, > but rather a COW volume. > > However, I can use rbd flatten on the image, and then delete the snapshot. > It takes time to copy the image if they're large, which makes sense as to > why it doesn't do this by default. But that is a reasonable workaround. > > Thanks. > > --John > > > -----Original Message----- > From: Sean McGinnis > Sent: Thursday, December 5, 2019 1:59 PM > To: Ratliff, John > Cc: openstack-discuss at lists.openstack.org > Subject: Re: [External] Re: openstack volume snapshots and deleting > > On Thu, Dec 05, 2019 at 06:25:28PM +0000, Ratliff, John wrote: >> Yes, I tried this second thing, but it doesn't work either. >> >> I created a new instance called test1.example.net. >> I created a volume snapshot (test1.example.net_ss1) I created a volume >> from that snapshot (vol_from_ss) I created a volume from the volume >> that was from the snapshot >> (vol_from_vol_from_ss) >> I deleted the vol_from_ss (this works) I deleted the snapshot (this >> does not work) >> > > Looking at the RBD driver, it looks like it does not actually follow the > expected clone semantics. Unless rbd_max_clone is set to 0, or the number of > clones is less than whatever that is set to, it will actually do a COW > volume on top of the snapshot. > > This is a performance optimization (it doesn't have to copy all of the data > into the new volume), but is actually a bug, IMO. > > You can try changing that config option and restarting services to pick that > up, then trying again. > > Otherwise, it looks like that drivers clone handling will create its own > snapshot of a volume. So unless you need to keep that snapshot around for > some other reason, it looks like you should be able to just: > > openstack volume create --source orig-volume new-volume > > And if I follow it right, it will create a snapshot under the covers, then > use that to create the new volume. > > Hopefully one of those helps. > > Sean > -- Mattia Belluco S3IT Services and Support for Science IT Office Y11 F 52 University of Zürich Winterthurerstrasse 190, CH-8057 Zürich (Switzerland) Tel: +41 44 635 42 22 From ignaziocassano at gmail.com Fri Dec 6 17:46:14 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Fri, 6 Dec 2019 18:46:14 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> References: <690FE148-F072-49AD-9428-49396E8907D9@stackhpc.com> <9F782B93-031D-43DA-9A50-2835D3D7AA74@stackhpc.com> Message-ID: Hello Bharat, we wrote today on irc. On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. I tested it but master kubernetes node reports errors while is trying to tag nodes. If you want I could send you more details . Best Regards Ignazio Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: > Yes you can specify comma separated `labels` when you create a cluster > template/cluster from the dashboard. > > e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. > > Also make sure cluster_user_trust=True inside your magnum.conf. > > On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: > > The network plugin is flannel . > I do not kow what is kube_tag . I read that with it I can specify the > kubernetes version, right ? > I am using openstack dashboard for creating my cluster: where I can > specify heat_container_agent_tag: train-stable in the dashboard ? > Thanks > Ignazio > > Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar > ha scritto: > >> Hi Ignazio, >> >> What version of kube_tag are you trying to run? What network plugin? I >> suggest using `heat_container_agent_tag: train-stable` as its easier to >> debug. >> >> >> Best >> >> bharat >> >> > On 2 Dec 2019, at 15:08, Ignazio Cassano >> wrote: >> > >> > hello, I've just installed stein with magnum on centos 7. I am creating >> a kubernetes cluster but it is running since 22 minutes and I think it will >> not work because the heat stack is waiting OSSoftwareDeployment completes >> > I presume something is not working on heat with magnum. I ran several >> stack with Software Deployment without magnum, and they worked fine >> > >> > Thanks >> > Ignazio >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jungleboyj at gmail.com Fri Dec 6 18:54:00 2019 From: jungleboyj at gmail.com (Jay Bryant) Date: Fri, 6 Dec 2019 12:54:00 -0600 Subject: [TC] 'V' Technical Elections In-Reply-To: References: <20191206125623.GB670935@sm-workstation> Message-ID: <53db3047-9822-a160-0e96-29956f272191@gmail.com> > The point I was making in another fork of this thread was to tie the > exact term length to the election timing and only state approximate > lengths here, ie including 'minimum' in the above will still set a > bound that complicates accommodating another slightly smaller than > usual cycle. > I am in agreement with Dean that the easiest and most flexible solution is to tie the term lengths to the elections and not state a particular minimum or maximum. Thank you for proposing this solution! Jay > dt From fungi at yuggoth.org Fri Dec 6 20:02:07 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Fri, 6 Dec 2019 20:02:07 +0000 Subject: [TC] 'V' Technical Elections In-Reply-To: <53db3047-9822-a160-0e96-29956f272191@gmail.com> References: <20191206125623.GB670935@sm-workstation> <53db3047-9822-a160-0e96-29956f272191@gmail.com> Message-ID: <20191206200207.36kbck272qyj6xul@yuggoth.org> On 2019-12-06 12:54:00 -0600 (-0600), Jay Bryant wrote: > > > The point I was making in another fork of this thread was to tie the > > exact term length to the election timing and only state approximate > > lengths here, ie including 'minimum' in the above will still set a bound > > that complicates accommodating another slightly smaller than usual > > cycle. > > > > > I am in agreement with Dean that the easiest and most flexible > solution is to tie the term lengths to the elections and not state > a particular minimum or maximum. [...] Does this satisfy what's required by the bylaws though? the term for the members of the Technical Committee shall be approved by a majority of the Technical Committee (“Term”) and shall be published publicly before each Technical Committee election; if no such Term is published the Term will be twelve calendar months. https://www.openstack.org/legal/technical-committee-member-policy/ I guess it doesn't *strictly* require a term duration to be specified as an explicit measure of time, so it's possible to interpret it as allowing terms to be defined according to some other criteria as long as they don't also exceed 16 months. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From Albert.Braden at synopsys.com Fri Dec 6 20:59:45 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 6 Dec 2019 20:59:45 +0000 Subject: Rebooted VM disappears when HV lacks memory In-Reply-To: <95ff036f1afebe000e8857c01b8d33e355a4d8a1.camel@redhat.com> References: <34afe1661c48c80de98d9c0e56202367c098a5e7.camel@redhat.com> <95ff036f1afebe000e8857c01b8d33e355a4d8a1.camel@redhat.com> Message-ID: Hi Sean, I didn't want to mess with prod configs but shelve/unshelve works. Thanks for your advice! -----Original Message----- From: Sean Mooney Sent: Friday, December 6, 2019 3:53 AM To: Albert Braden ; openstack-discuss at lists.openstack.org Subject: Re: Rebooted VM disappears when HV lacks memory On Fri, 2019-12-06 at 02:22 +0000, Albert Braden wrote: > Update: It looks like attempting to migrate any VM, alive or dead, from an oversubscribed hypervisor throws the same > error. Maybe I should have done the migrations before setting cpu_allocation_ratio and ram_allocation_ratio. > > Is there a way to recover from this besides deleting VMs from the oversubscribed hypervisors? you could tempoerally set the allocation ratio higher to allow the vm to boot then try migrating it again. you could also maybe try shelving the vm, waiting for ti to be shelve offloaded then unshelve it. > > -----Original Message----- > From: Albert Braden > Sent: Thursday, December 5, 2019 6:08 PM > To: openstack-discuss at lists.openstack.org > Subject: RE: Rebooted VM disappears when HV lacks memory > > We changed cpu_allocation_ratio and ram_allocation_ratio to 1 on the hypervisors. Now I'm trying to clean up the mess. > > I'm trying to cold-migrate dead VMs from an oversubscribed hypervisor, but they are failing to migrate: > > root at us01odc-p02-ctrl1:~# os hypervisor list --long|grep hv199 > > 218 | us01-p02-hv199.internal.synopsys.com | QEMU | 10.195.54.226 | up | 138 | 80 > > | 1122304 | 772691 | > > root at us01odc-p02-ctrl1:~# openstack server migrate fc250e76-4ca5-44b1-a6a0-8a7c7ad48c24 > No valid host was found. No valid host found for cold migrate (HTTP 400) (Request-ID: req-cc048221-abaf-4be0-bc90- > f28eecec3df5) > > But we have lots of hosts with space, and if I build a new VM with the same flavor it works. When I check the logs, it > appears that the migration fails because the hypervisor we are moving from is oversubscribed: > > /var/log/nova/nova-conductor.log:2019-12-05 17:30:17.163 48989 WARNING nova.scheduler.client.report [req-cc048221- > abaf-4be0-bc90-f28eecec3df5 5b288691527245bda715ab7744a193e9 deea2d8541f741eda6fb0d242d16bb23 - default > 824941b2ad6a43d7984fab9390290f18] Unable to post allocations for instance 87608240-1ad2-45fc-b4f0-c118e3bf0262 (409 > {"errors": [{"status": 409, "request_id": "req-e56d29f8-c97e-41ac-8481-0598f3c681b2", "detail": "There was a conflict > when trying to complete your request.\n\n Unable to allocate inventory: Unable to create allocation for 'MEMORY_MB' on > resource provider 'fde477cd-fd3d-4296-838c-f9c54b3e97ef'. The requested amount would exceed the capacity. ", "title": > "Conflict"}]}) > > 'fde477cd-fd3d-4296-838c-f9c54b3e97ef' is the UUID of hv199: > > root at us01odc-p02-ctrl1:~# curl -s -X GET -H "X-Auth-Token: $token" -H "Content-Type: application/json" " > http://us01odc-p02-lb.internal.synopsys.com:28778/resource_providers/fde477cd-fd3d-4296-838c-f9c54b3e97ef";echo > {"generation": 40, "uuid": "fde477cd-fd3d-4296-838c-f9c54b3e97ef", "links": [{"href": "/resource_providers/fde477cd- > fd3d-4296-838c-f9c54b3e97ef", "rel": "self"}, {"href": "/resource_providers/fde477cd-fd3d-4296-838c- > f9c54b3e97ef/inventories", "rel": "inventories"}, {"href": "/resource_providers/fde477cd-fd3d-4296-838c- > f9c54b3e97ef/usages", "rel": "usages"}], "name": "us01-p02-hv199.internal.synopsys.com"} > > What am I doing wrong? Do I need to move the live VMs away from hv199 to make room for the dead ones? > > > > -----Original Message----- > From: Sean Mooney > Sent: Wednesday, December 4, 2019 4:30 AM > To: Albert Braden ; openstack-discuss at lists.openstack.org > Subject: Re: Rebooted VM disappears when HV lacks memory > > On Wed, 2019-12-04 at 00:51 +0000, Albert Braden wrote: > > We are running Rocky and we have the allocation bug because we set cpu_allocation_ratio and ram_allocation_ratio to > > 1 > > on the controllers but not on the hypervisors, so the hypervisors are oversubscribed. When we try to reboot a VM > > when > > the HV is short memory, the VM is deleted from the HV. It still exists in Openstack but not on the HV. Is it > > possible > > to recover a VM from this state? > > so the vm is likely not deleted the harddisk and the libvirt domain will still be defiend. my guess is starting the > vm cause the oom reaper to kill the qemu process when it was relaunched. > > if you are deploying with memory over subsription enabled you should configre enough swap space equal to total > memory* ram_allocation_ratio. > > if you do that you you should be able to start the vm. now the other option is while the vm is stopped you can > cold migrate it to another host and then start it there. From bharat at stackhpc.com Fri Dec 6 21:53:59 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Fri, 6 Dec 2019 21:53:59 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: More details is good. Cheers Bharat > On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: > >  > Hello Bharat, we wrote today on irc. > On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. > I tested it but master kubernetes node reports errors while is trying to tag nodes. > If you want I could send you more details . > Best Regards > Ignazio > > > Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. >> >> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >> >> Also make sure cluster_user_trust=True inside your magnum.conf. >> >>> On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: >>> >>> The network plugin is flannel . >>> I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? >>> I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? >>> Thanks >>> Ignazio >>> >>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: >>>> Hi Ignazio, >>>> >>>> What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. >>>> >>>> >>>> Best >>>> >>>> bharat >>>> >>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: >>>> > >>>> > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes >>>> > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine >>>> > >>>> > Thanks >>>> > Ignazio >>>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Fri Dec 6 21:53:59 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Fri, 6 Dec 2019 21:53:59 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: More details is good. Cheers Bharat > On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: > >  > Hello Bharat, we wrote today on irc. > On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. > I tested it but master kubernetes node reports errors while is trying to tag nodes. > If you want I could send you more details . > Best Regards > Ignazio > > > Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. >> >> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >> >> Also make sure cluster_user_trust=True inside your magnum.conf. >> >>> On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: >>> >>> The network plugin is flannel . >>> I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? >>> I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? >>> Thanks >>> Ignazio >>> >>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: >>>> Hi Ignazio, >>>> >>>> What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. >>>> >>>> >>>> Best >>>> >>>> bharat >>>> >>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: >>>> > >>>> > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes >>>> > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine >>>> > >>>> > Thanks >>>> > Ignazio >>>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Fri Dec 6 23:20:47 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 6 Dec 2019 17:20:47 -0600 Subject: [nova][ironic] Changing an owner of a provisioned node In-Reply-To: References: <5049a2b0-4ab8-261f-b17b-e99288646f97@gmail.com> Message-ID: <3317ff60-cabb-2f88-296f-562099a58afc@gmail.com> On 12/4/2019 1:56 PM, Tzu-Mainn Chen wrote: > I like the idea of something like baremetal:node:update_owner > defaulting to rule:is_admin (NOT to baremetal:node:update). I can > work on a patch tomorrow if nobody beats me to it. > > > I'm happy to take this on. Thanks! Thanks, I've created a story/bug report for tracking this: https://storyboard.openstack.org/#!/story/2006997 -- Thanks, Matt From Albert.Braden at synopsys.com Sat Dec 7 02:11:10 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Sat, 7 Dec 2019 02:11:10 +0000 Subject: neutron-metadata-agent broken pipe Message-ID: As our production cluster grows in size we are starting to have trouble with neutron-metadata-agent. After restarting it is happy for a minute and then it complains "2019-12-06 17:54:24.615 664587 WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 11, greater than warning threshold: 10. There could be a leak. Increasing threshold to: 20" It increases the threshold a couple of times and then after increasing to 40 we start to see errors: 2019-12-06 17:55:10.119 664578 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe It looks like increasing the threshold to 40 fails because it keeps trying: 2019-12-06 17:55:17.452 664597 WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 21, greater than warning threshold: 20. There could be a leak. Increasing threshold to: 40 And the errors increase until the log is nothing but errors, and VMs fail to boot. root at us01odc-p02-ctrl1:~# tail -f /var/log/neutron/neutron-metadata-agent.log self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe 2019-12-06 17:59:23.664 664597 INFO eventlet.wsgi.server [-] 10.195.73.174, "GET /latest/meta-data/instance-id HTTP/1.0" status: 200 len: 0 time: 62.1517761 2019-12-06 17:59:23.756 664583 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe 2019-12-06 17:59:23.757 664583 INFO eventlet.wsgi.server [-] 10.195.65.69, "GET /latest/meta-data/instance-id HTTP/1.0" status: 200 len: 0 time: 63.0419171 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent [-] Unexpected error.: MessagingTimeout: Timed out waiting for a reply to message ID 77551d4cf4394b7b9cdfad68c0be46e8 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent Traceback (most recent call last): 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 89, in __call__ 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent instance_id, tenant_id = self._get_instance_and_tenant_id(req) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 162, in _get_instance_and_tenant_id 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent ports = self._get_ports(remote_address, network_id, router_id) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 155, in _get_ports 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self._get_ports_for_remote_address(remote_address, networks) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/cache_utils.py", line 116, in __call__ 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self.func(target_self, *args, **kwargs) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 137, in _get_ports_for_remote_address 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent ip_address=remote_address) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 106, in _get_ports_from_server 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self.plugin_rpc.get_ports(self.context, filters) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 72, in get_ports 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return cctxt.call(context, 'get_ports', filters=filters) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 173, in call 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent time.sleep(wait) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent self.force_reraise() 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent six.reraise(self.type_, self.value, self.tb) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 150, in call 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self._original_context.call(ctxt, method, **kwargs) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent retry=self.retry) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent retry=retry) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent call_monitor_timeout, retry=retry) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 573, in _send 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent call_monitor_timeout) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 459, in wait 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent message = self.waiters.get(msg_id, timeout=timeout) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 336, in get 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent 'to message ID %s' % msg_id) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent MessagingTimeout: Timed out waiting for a reply to message ID 77551d4cf4394b7b9cdfad68c0be46e8 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent 2019-12-06 17:59:23.778 664595 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) What's causing this? Are we overloading RMQ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rony.khan at brilliant.com.bd Sat Dec 7 05:55:20 2019 From: rony.khan at brilliant.com.bd (Md. Farhad Hasan Khan) Date: Sat, 7 Dec 2019 11:55:20 +0600 Subject: High CPU utilization by systemd process || debian 8 Message-ID: <0ff601d5acc2$e92a15e0$bb7e41a0$@brilliant.com.bd> Hi, We are observing high CPU utilization by systemd process. 100% CPU utilize by systemd process when no user is being logged in to the server but CPU utilization became 0% while any user logged in to the server. What cloud be possible reason behind this issue and solution? Log collected via login to the server via ssh or console: (0% CPU utilization) top -b -n 1 | grep systemd | grep 612 612 root 20 0 3386692 2.051g 4932 T 0.0 6.5 22029:26 systemd Log collected without login to the server : (100% CPU utilization) # ssh user at 192.168.18.11 top -b -n 1 | grep systemd | grep 612 user at 192.168.18.11's password: 612 root 20 0 3386692 2.051g 4932 S 789.2 6.5 22031:31 systemd Please find the server information: lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 8.11 (jessie) Release: 8.11 Codename: jessie uname -r 3.16.0-6-amd64 Thanks & B’Rgds, Rony -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sat Dec 7 17:35:43 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sat, 7 Dec 2019 18:35:43 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: Hello Bharat, the following are last lines of the master node /var/log/cloud-init-output.log file: ystemctl enable heat-container-agent + systemctl start heat-container-agent starting services activating service etcd activating service docker activating service kube-apiserver activating service kube-controller-manager activating service kube-scheduler activating service kubelet activating service kube-proxy Trying to label master node with node-role.kubernetes.io/master="" Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" error: resource(s) were provided, but no name, label selector, or --all flag specified Trying to label master node with node-role.kubernetes.io/master="" Regards Ignazio Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar ha scritto: > More details is good. > > Cheers > > Bharat > > On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: > >  > Hello Bharat, we wrote today on irc. > On rdo stein testing repo the new openstack magnum version has ben > released at 2:27 Pm. > I tested it but master kubernetes node reports errors while is trying to > tag nodes. > If you want I could send you more details . > Best Regards > Ignazio > > > Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: > >> Yes you can specify comma separated `labels` when you create a cluster >> template/cluster from the dashboard. >> >> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >> >> Also make sure cluster_user_trust=True inside your magnum.conf. >> >> On 2 Dec 2019, at 15:26, Ignazio Cassano >> wrote: >> >> The network plugin is flannel . >> I do not kow what is kube_tag . I read that with it I can specify the >> kubernetes version, right ? >> I am using openstack dashboard for creating my cluster: where I can >> specify heat_container_agent_tag: train-stable in the dashboard ? >> Thanks >> Ignazio >> >> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar < >> bharat at stackhpc.com> ha scritto: >> >>> Hi Ignazio, >>> >>> What version of kube_tag are you trying to run? What network plugin? I >>> suggest using `heat_container_agent_tag: train-stable` as its easier to >>> debug. >>> >>> >>> Best >>> >>> bharat >>> >>> > On 2 Dec 2019, at 15:08, Ignazio Cassano >>> wrote: >>> > >>> > hello, I've just installed stein with magnum on centos 7. I am >>> creating a kubernetes cluster but it is running since 22 minutes and I >>> think it will not work because the heat stack is waiting >>> OSSoftwareDeployment completes >>> > I presume something is not working on heat with magnum. I ran several >>> stack with Software Deployment without magnum, and they worked fine >>> > >>> > Thanks >>> > Ignazio >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rony.khan at brilliant.com.bd Sat Dec 7 18:39:33 2019 From: rony.khan at brilliant.com.bd (Md. Farhad Hasan Khan) Date: Sun, 8 Dec 2019 00:39:33 +0600 Subject: High CPU utilization by systemd process || debian 8 Message-ID: <04d301d5ad2d$ab549b80$01fdd280$@brilliant.com.bd> Hi, Please help to solve this. Thanks/Rony From: Md. Farhad Hasan Khan [mailto:rony.khan at brilliant.com.bd] Sent: Saturday, December 7, 2019 11:55 AM To: 'OpenStack Discuss' Subject: High CPU utilization by systemd process || debian 8 Hi, We are observing high CPU utilization by systemd process. 100% CPU utilize by systemd process when no user is being logged in to the server but CPU utilization became 0% while any user logged in to the server. What cloud be possible reason behind this issue and solution? Log collected via login to the server via ssh or console: (0% CPU utilization) top -b -n 1 | grep systemd | grep 612 612 root 20 0 3386692 2.051g 4932 T 0.0 6.5 22029:26 systemd Log collected without login to the server : (100% CPU utilization) # ssh user at 192.168.18.11 top -b -n 1 | grep systemd | grep 612 user at 192.168.18.11's password: 612 root 20 0 3386692 2.051g 4932 S 789.2 6.5 22031:31 systemd Please find the server information: lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 8.11 (jessie) Release: 8.11 Codename: jessie uname -r 3.16.0-6-amd64 Thanks & B’Rgds, Rony -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Sat Dec 7 23:06:03 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Sat, 7 Dec 2019 23:06:03 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: <65AE0E3B-B3AF-4EAA-A50A-8F9B7D5BEB40@stackhpc.com> Do you have this change in the version of magnum you installed? If you do, it should definitely work! https://review.opendev.org/#/c/695632/ Sent from my iPhone > On 7 Dec 2019, at 17:35, Ignazio Cassano wrote: > >  > Hello Bharat, > the following are last lines of the master node /var/log/cloud-init-output.log file: > ystemctl enable heat-container-agent > + systemctl start heat-container-agent > starting services > activating service etcd > activating service docker > activating service kube-apiserver > activating service kube-controller-manager > activating service kube-scheduler > activating service kubelet > activating service kube-proxy > Trying to label master node with node-role.kubernetes.io/master="" > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > > Regards > Ignazio > >> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar ha scritto: >> More details is good. >> >> Cheers >> >> Bharat >> >>>> On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: >>>> >>>  >>> Hello Bharat, we wrote today on irc. >>> On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. >>> I tested it but master kubernetes node reports errors while is trying to tag nodes. >>> If you want I could send you more details . >>> Best Regards >>> Ignazio >>> >>> >>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >>>> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. >>>> >>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>> >>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>> >>>>> On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: >>>>> >>>>> The network plugin is flannel . >>>>> I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? >>>>> I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? >>>>> Thanks >>>>> Ignazio >>>>> >>>>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: >>>>>> Hi Ignazio, >>>>>> >>>>>> What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. >>>>>> >>>>>> >>>>>> Best >>>>>> >>>>>> bharat >>>>>> >>>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: >>>>>> > >>>>>> > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes >>>>>> > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine >>>>>> > >>>>>> > Thanks >>>>>> > Ignazio >>>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Sat Dec 7 23:06:03 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Sat, 7 Dec 2019 23:06:03 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: <65AE0E3B-B3AF-4EAA-A50A-8F9B7D5BEB40@stackhpc.com> Do you have this change in the version of magnum you installed? If you do, it should definitely work! https://review.opendev.org/#/c/695632/ Sent from my iPhone > On 7 Dec 2019, at 17:35, Ignazio Cassano wrote: > >  > Hello Bharat, > the following are last lines of the master node /var/log/cloud-init-output.log file: > ystemctl enable heat-container-agent > + systemctl start heat-container-agent > starting services > activating service etcd > activating service docker > activating service kube-apiserver > activating service kube-controller-manager > activating service kube-scheduler > activating service kubelet > activating service kube-proxy > Trying to label master node with node-role.kubernetes.io/master="" > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all flag specified > Trying to label master node with node-role.kubernetes.io/master="" > > Regards > Ignazio > >> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar ha scritto: >> More details is good. >> >> Cheers >> >> Bharat >> >>>> On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: >>>> >>>  >>> Hello Bharat, we wrote today on irc. >>> On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. >>> I tested it but master kubernetes node reports errors while is trying to tag nodes. >>> If you want I could send you more details . >>> Best Regards >>> Ignazio >>> >>> >>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >>>> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. >>>> >>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>> >>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>> >>>>> On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: >>>>> >>>>> The network plugin is flannel . >>>>> I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? >>>>> I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? >>>>> Thanks >>>>> Ignazio >>>>> >>>>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: >>>>>> Hi Ignazio, >>>>>> >>>>>> What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. >>>>>> >>>>>> >>>>>> Best >>>>>> >>>>>> bharat >>>>>> >>>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: >>>>>> > >>>>>> > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes >>>>>> > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine >>>>>> > >>>>>> > Thanks >>>>>> > Ignazio >>>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sun Dec 8 10:55:46 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 8 Dec 2019 11:55:46 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: <65AE0E3B-B3AF-4EAA-A50A-8F9B7D5BEB40@stackhpc.com> References: <65AE0E3B-B3AF-4EAA-A50A-8F9B7D5BEB40@stackhpc.com> Message-ID: No, I don't. I'll apply this patch asap. Thanks Ignazio Il Dom 8 Dic 2019, 00:06 Bharat Kunwar ha scritto: > Do you have this change in the version of magnum you installed? If you do, > it should definitely work! > > https://review.opendev.org/#/c/695632/ > > Sent from my iPhone > > On 7 Dec 2019, at 17:35, Ignazio Cassano wrote: > >  > Hello Bharat, > the following are last lines of the master node > /var/log/cloud-init-output.log file: > ystemctl enable heat-container-agent > + systemctl start heat-container-agent > starting services > activating service etcd > activating service docker > activating service kube-apiserver > activating service kube-controller-manager > activating service kube-scheduler > activating service kubelet > activating service kube-proxy > Trying to label master node with node-role.kubernetes.io/master="" > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > error: resource(s) were provided, but no name, label selector, or --all > flag specified > Trying to label master node with node-role.kubernetes.io/master="" > > Regards > Ignazio > > Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar > ha scritto: > >> More details is good. >> >> Cheers >> >> Bharat >> >> On 6 Dec 2019, at 17:46, Ignazio Cassano >> wrote: >> >>  >> Hello Bharat, we wrote today on irc. >> On rdo stein testing repo the new openstack magnum version has ben >> released at 2:27 Pm. >> I tested it but master kubernetes node reports errors while is trying to >> tag nodes. >> If you want I could send you more details . >> Best Regards >> Ignazio >> >> >> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >> >>> Yes you can specify comma separated `labels` when you create a cluster >>> template/cluster from the dashboard. >>> >>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>> >>> Also make sure cluster_user_trust=True inside your magnum.conf. >>> >>> On 2 Dec 2019, at 15:26, Ignazio Cassano >>> wrote: >>> >>> The network plugin is flannel . >>> I do not kow what is kube_tag . I read that with it I can specify the >>> kubernetes version, right ? >>> I am using openstack dashboard for creating my cluster: where I can >>> specify heat_container_agent_tag: train-stable in the dashboard ? >>> Thanks >>> Ignazio >>> >>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar < >>> bharat at stackhpc.com> ha scritto: >>> >>>> Hi Ignazio, >>>> >>>> What version of kube_tag are you trying to run? What network plugin? I >>>> suggest using `heat_container_agent_tag: train-stable` as its easier to >>>> debug. >>>> >>>> >>>> Best >>>> >>>> bharat >>>> >>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano >>>> wrote: >>>> > >>>> > hello, I've just installed stein with magnum on centos 7. I am >>>> creating a kubernetes cluster but it is running since 22 minutes and I >>>> think it will not work because the heat stack is waiting >>>> OSSoftwareDeployment completes >>>> > I presume something is not working on heat with magnum. I ran several >>>> stack with Software Deployment without magnum, and they worked fine >>>> > >>>> > Thanks >>>> > Ignazio >>>> >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Sun Dec 8 12:37:21 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Sun, 8 Dec 2019 12:37:21 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: It should have been installed with stein 8.2.0. It sounds like you haven’t installed the right version of the package. > On 8 Dec 2019, at 10:55, Ignazio Cassano wrote: > >  > No, I don't. I'll apply this patch asap. > Thanks > Ignazio > > > Il Dom 8 Dic 2019, 00:06 Bharat Kunwar ha scritto: >> Do you have this change in the version of magnum you installed? If you do, it should definitely work! >> >> https://review.opendev.org/#/c/695632/ >> >> Sent from my iPhone >> >>>> On 7 Dec 2019, at 17:35, Ignazio Cassano wrote: >>>> >>>  >>> Hello Bharat, >>> the following are last lines of the master node /var/log/cloud-init-output.log file: >>> ystemctl enable heat-container-agent >>> + systemctl start heat-container-agent >>> starting services >>> activating service etcd >>> activating service docker >>> activating service kube-apiserver >>> activating service kube-controller-manager >>> activating service kube-scheduler >>> activating service kubelet >>> activating service kube-proxy >>> Trying to label master node with node-role.kubernetes.io/master="" >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> >>> Regards >>> Ignazio >>> >>>> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar ha scritto: >>>> More details is good. >>>> >>>> Cheers >>>> >>>> Bharat >>>> >>>>>> On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: >>>>>> >>>>>  >>>>> Hello Bharat, we wrote today on irc. >>>>> On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. >>>>> I tested it but master kubernetes node reports errors while is trying to tag nodes. >>>>> If you want I could send you more details . >>>>> Best Regards >>>>> Ignazio >>>>> >>>>> >>>>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >>>>>> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. >>>>>> >>>>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>>>> >>>>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>>>> >>>>>>> On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: >>>>>>> >>>>>>> The network plugin is flannel . >>>>>>> I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? >>>>>>> I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? >>>>>>> Thanks >>>>>>> Ignazio >>>>>>> >>>>>>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: >>>>>>>> Hi Ignazio, >>>>>>>> >>>>>>>> What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. >>>>>>>> >>>>>>>> >>>>>>>> Best >>>>>>>> >>>>>>>> bharat >>>>>>>> >>>>>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: >>>>>>>> > >>>>>>>> > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes >>>>>>>> > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine >>>>>>>> > >>>>>>>> > Thanks >>>>>>>> > Ignazio >>>>>>>> >>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bharat at stackhpc.com Sun Dec 8 12:37:21 2019 From: bharat at stackhpc.com (Bharat Kunwar) Date: Sun, 8 Dec 2019 12:37:21 +0000 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: It should have been installed with stein 8.2.0. It sounds like you haven’t installed the right version of the package. > On 8 Dec 2019, at 10:55, Ignazio Cassano wrote: > >  > No, I don't. I'll apply this patch asap. > Thanks > Ignazio > > > Il Dom 8 Dic 2019, 00:06 Bharat Kunwar ha scritto: >> Do you have this change in the version of magnum you installed? If you do, it should definitely work! >> >> https://review.opendev.org/#/c/695632/ >> >> Sent from my iPhone >> >>>> On 7 Dec 2019, at 17:35, Ignazio Cassano wrote: >>>> >>>  >>> Hello Bharat, >>> the following are last lines of the master node /var/log/cloud-init-output.log file: >>> ystemctl enable heat-container-agent >>> + systemctl start heat-container-agent >>> starting services >>> activating service etcd >>> activating service docker >>> activating service kube-apiserver >>> activating service kube-controller-manager >>> activating service kube-scheduler >>> activating service kubelet >>> activating service kube-proxy >>> Trying to label master node with node-role.kubernetes.io/master="" >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> >>> Regards >>> Ignazio >>> >>>> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar ha scritto: >>>> More details is good. >>>> >>>> Cheers >>>> >>>> Bharat >>>> >>>>>> On 6 Dec 2019, at 17:46, Ignazio Cassano wrote: >>>>>> >>>>>  >>>>> Hello Bharat, we wrote today on irc. >>>>> On rdo stein testing repo the new openstack magnum version has ben released at 2:27 Pm. >>>>> I tested it but master kubernetes node reports errors while is trying to tag nodes. >>>>> If you want I could send you more details . >>>>> Best Regards >>>>> Ignazio >>>>> >>>>> >>>>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >>>>>> Yes you can specify comma separated `labels` when you create a cluster template/cluster from the dashboard. >>>>>> >>>>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>>>> >>>>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>>>> >>>>>>> On 2 Dec 2019, at 15:26, Ignazio Cassano wrote: >>>>>>> >>>>>>> The network plugin is flannel . >>>>>>> I do not kow what is kube_tag . I read that with it I can specify the kubernetes version, right ? >>>>>>> I am using openstack dashboard for creating my cluster: where I can specify heat_container_agent_tag: train-stable in the dashboard ? >>>>>>> Thanks >>>>>>> Ignazio >>>>>>> >>>>>>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar ha scritto: >>>>>>>> Hi Ignazio, >>>>>>>> >>>>>>>> What version of kube_tag are you trying to run? What network plugin? I suggest using `heat_container_agent_tag: train-stable` as its easier to debug. >>>>>>>> >>>>>>>> >>>>>>>> Best >>>>>>>> >>>>>>>> bharat >>>>>>>> >>>>>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano wrote: >>>>>>>> > >>>>>>>> > hello, I've just installed stein with magnum on centos 7. I am creating a kubernetes cluster but it is running since 22 minutes and I think it will not work because the heat stack is waiting OSSoftwareDeployment completes >>>>>>>> > I presume something is not working on heat with magnum. I ran several stack with Software Deployment without magnum, and they worked fine >>>>>>>> > >>>>>>>> > Thanks >>>>>>>> > Ignazio >>>>>>>> >>>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sun Dec 8 15:12:03 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 8 Dec 2019 16:12:03 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: Yes, you are right, I think they releases patched packages today Ignazio Il giorno dom 8 dic 2019 alle ore 13:37 Bharat Kunwar ha scritto: > It should have been installed with stein 8.2.0. It sounds like you haven’t > installed the right version of the package. > > On 8 Dec 2019, at 10:55, Ignazio Cassano wrote: > >  > No, I don't. I'll apply this patch asap. > Thanks > Ignazio > > > Il Dom 8 Dic 2019, 00:06 Bharat Kunwar ha scritto: > >> Do you have this change in the version of magnum you installed? If you >> do, it should definitely work! >> >> https://review.opendev.org/#/c/695632/ >> >> Sent from my iPhone >> >> On 7 Dec 2019, at 17:35, Ignazio Cassano >> wrote: >> >>  >> Hello Bharat, >> the following are last lines of the master node >> /var/log/cloud-init-output.log file: >> ystemctl enable heat-container-agent >> + systemctl start heat-container-agent >> starting services >> activating service etcd >> activating service docker >> activating service kube-apiserver >> activating service kube-controller-manager >> activating service kube-scheduler >> activating service kubelet >> activating service kube-proxy >> Trying to label master node with node-role.kubernetes.io/master="" >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> >> Regards >> Ignazio >> >> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar < >> bharat at stackhpc.com> ha scritto: >> >>> More details is good. >>> >>> Cheers >>> >>> Bharat >>> >>> On 6 Dec 2019, at 17:46, Ignazio Cassano >>> wrote: >>> >>>  >>> Hello Bharat, we wrote today on irc. >>> On rdo stein testing repo the new openstack magnum version has ben >>> released at 2:27 Pm. >>> I tested it but master kubernetes node reports errors while is trying to >>> tag nodes. >>> If you want I could send you more details . >>> Best Regards >>> Ignazio >>> >>> >>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >>> >>>> Yes you can specify comma separated `labels` when you create a cluster >>>> template/cluster from the dashboard. >>>> >>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>> >>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>> >>>> On 2 Dec 2019, at 15:26, Ignazio Cassano >>>> wrote: >>>> >>>> The network plugin is flannel . >>>> I do not kow what is kube_tag . I read that with it I can specify the >>>> kubernetes version, right ? >>>> I am using openstack dashboard for creating my cluster: where I can >>>> specify heat_container_agent_tag: train-stable in the dashboard ? >>>> Thanks >>>> Ignazio >>>> >>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar < >>>> bharat at stackhpc.com> ha scritto: >>>> >>>>> Hi Ignazio, >>>>> >>>>> What version of kube_tag are you trying to run? What network plugin? I >>>>> suggest using `heat_container_agent_tag: train-stable` as its easier to >>>>> debug. >>>>> >>>>> >>>>> Best >>>>> >>>>> bharat >>>>> >>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano >>>>> wrote: >>>>> > >>>>> > hello, I've just installed stein with magnum on centos 7. I am >>>>> creating a kubernetes cluster but it is running since 22 minutes and I >>>>> think it will not work because the heat stack is waiting >>>>> OSSoftwareDeployment completes >>>>> > I presume something is not working on heat with magnum. I ran >>>>> several stack with Software Deployment without magnum, and they worked fine >>>>> > >>>>> > Thanks >>>>> > Ignazio >>>>> >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sun Dec 8 15:20:01 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 8 Dec 2019 16:20:01 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: New rpm packages contains only the path for the second file related to the bug you sent me, any case I am going to try patching by hand Il giorno dom 8 dic 2019 alle ore 13:37 Bharat Kunwar ha scritto: > It should have been installed with stein 8.2.0. It sounds like you haven’t > installed the right version of the package. > > On 8 Dec 2019, at 10:55, Ignazio Cassano wrote: > >  > No, I don't. I'll apply this patch asap. > Thanks > Ignazio > > > Il Dom 8 Dic 2019, 00:06 Bharat Kunwar ha scritto: > >> Do you have this change in the version of magnum you installed? If you >> do, it should definitely work! >> >> https://review.opendev.org/#/c/695632/ >> >> Sent from my iPhone >> >> On 7 Dec 2019, at 17:35, Ignazio Cassano >> wrote: >> >>  >> Hello Bharat, >> the following are last lines of the master node >> /var/log/cloud-init-output.log file: >> ystemctl enable heat-container-agent >> + systemctl start heat-container-agent >> starting services >> activating service etcd >> activating service docker >> activating service kube-apiserver >> activating service kube-controller-manager >> activating service kube-scheduler >> activating service kubelet >> activating service kube-proxy >> Trying to label master node with node-role.kubernetes.io/master="" >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> error: resource(s) were provided, but no name, label selector, or --all >> flag specified >> Trying to label master node with node-role.kubernetes.io/master="" >> >> Regards >> Ignazio >> >> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar < >> bharat at stackhpc.com> ha scritto: >> >>> More details is good. >>> >>> Cheers >>> >>> Bharat >>> >>> On 6 Dec 2019, at 17:46, Ignazio Cassano >>> wrote: >>> >>>  >>> Hello Bharat, we wrote today on irc. >>> On rdo stein testing repo the new openstack magnum version has ben >>> released at 2:27 Pm. >>> I tested it but master kubernetes node reports errors while is trying to >>> tag nodes. >>> If you want I could send you more details . >>> Best Regards >>> Ignazio >>> >>> >>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha scritto: >>> >>>> Yes you can specify comma separated `labels` when you create a cluster >>>> template/cluster from the dashboard. >>>> >>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>> >>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>> >>>> On 2 Dec 2019, at 15:26, Ignazio Cassano >>>> wrote: >>>> >>>> The network plugin is flannel . >>>> I do not kow what is kube_tag . I read that with it I can specify the >>>> kubernetes version, right ? >>>> I am using openstack dashboard for creating my cluster: where I can >>>> specify heat_container_agent_tag: train-stable in the dashboard ? >>>> Thanks >>>> Ignazio >>>> >>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar < >>>> bharat at stackhpc.com> ha scritto: >>>> >>>>> Hi Ignazio, >>>>> >>>>> What version of kube_tag are you trying to run? What network plugin? I >>>>> suggest using `heat_container_agent_tag: train-stable` as its easier to >>>>> debug. >>>>> >>>>> >>>>> Best >>>>> >>>>> bharat >>>>> >>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano >>>>> wrote: >>>>> > >>>>> > hello, I've just installed stein with magnum on centos 7. I am >>>>> creating a kubernetes cluster but it is running since 22 minutes and I >>>>> think it will not work because the heat stack is waiting >>>>> OSSoftwareDeployment completes >>>>> > I presume something is not working on heat with magnum. I ran >>>>> several stack with Software Deployment without magnum, and they worked fine >>>>> > >>>>> > Thanks >>>>> > Ignazio >>>>> >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Sun Dec 8 15:43:07 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Sun, 8 Dec 2019 16:43:07 +0100 Subject: [magnum] [stein] kubernetes stack does not terminate In-Reply-To: References: Message-ID: Hello, applyng pathes you sent me, only master node is tagged while minion are not tagged: [centos at kubectl ~]$ kubectl get nodes NAME STATUS ROLES AGE VERSION kube-vkfsogbuz4we-master-0 Ready master 10m v1.14.8 kube-vkfsogbuz4we-minion-0 Ready 10m v1.14.8 [centos at kubectl ~]$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-865bd969f-cldrp 0/1 Pending 0 10m heapster-7bf5794cc7-hkhcl 0/1 Pending 0 10m k8s-keystone-auth-pdcxt 1/1 Running 0 10m kube-dns-autoscaler-6c6b4b7cf7-pwxtl 0/1 Pending 0 10m kube-flannel-ds-amd64-k5r97 1/1 Running 0 10m kube-flannel-ds-amd64-s2vrm 1/1 Running 0 10m kubernetes-dashboard-d48c76949-wvgr4 0/1 Pending 0 10m openstack-cloud-controller-manager-sxxkd 1/1 Running 0 10m Ignazio Il giorno dom 8 dic 2019 alle ore 16:20 Ignazio Cassano < ignaziocassano at gmail.com> ha scritto: > New rpm packages contains only the path for the second file related to > the bug you sent me, any case I am going to try patching by hand > > Il giorno dom 8 dic 2019 alle ore 13:37 Bharat Kunwar > ha scritto: > >> It should have been installed with stein 8.2.0. It sounds like you >> haven’t installed the right version of the package. >> >> On 8 Dec 2019, at 10:55, Ignazio Cassano >> wrote: >> >>  >> No, I don't. I'll apply this patch asap. >> Thanks >> Ignazio >> >> >> Il Dom 8 Dic 2019, 00:06 Bharat Kunwar ha scritto: >> >>> Do you have this change in the version of magnum you installed? If you >>> do, it should definitely work! >>> >>> https://review.opendev.org/#/c/695632/ >>> >>> Sent from my iPhone >>> >>> On 7 Dec 2019, at 17:35, Ignazio Cassano >>> wrote: >>> >>>  >>> Hello Bharat, >>> the following are last lines of the master node >>> /var/log/cloud-init-output.log file: >>> ystemctl enable heat-container-agent >>> + systemctl start heat-container-agent >>> starting services >>> activating service etcd >>> activating service docker >>> activating service kube-apiserver >>> activating service kube-controller-manager >>> activating service kube-scheduler >>> activating service kubelet >>> activating service kube-proxy >>> Trying to label master node with node-role.kubernetes.io/master="" >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> error: resource(s) were provided, but no name, label selector, or --all >>> flag specified >>> Trying to label master node with node-role.kubernetes.io/master="" >>> >>> Regards >>> Ignazio >>> >>> Il giorno ven 6 dic 2019 alle ore 22:54 Bharat Kunwar < >>> bharat at stackhpc.com> ha scritto: >>> >>>> More details is good. >>>> >>>> Cheers >>>> >>>> Bharat >>>> >>>> On 6 Dec 2019, at 17:46, Ignazio Cassano >>>> wrote: >>>> >>>>  >>>> Hello Bharat, we wrote today on irc. >>>> On rdo stein testing repo the new openstack magnum version has ben >>>> released at 2:27 Pm. >>>> I tested it but master kubernetes node reports errors while is trying >>>> to tag nodes. >>>> If you want I could send you more details . >>>> Best Regards >>>> Ignazio >>>> >>>> >>>> Il Lun 2 Dic 2019, 16:29 Bharat Kunwar ha >>>> scritto: >>>> >>>>> Yes you can specify comma separated `labels` when you create a cluster >>>>> template/cluster from the dashboard. >>>>> >>>>> e.g. heat_container_agent_tag=train-stable,kube_tag=v1.14.8. >>>>> >>>>> Also make sure cluster_user_trust=True inside your magnum.conf. >>>>> >>>>> On 2 Dec 2019, at 15:26, Ignazio Cassano >>>>> wrote: >>>>> >>>>> The network plugin is flannel . >>>>> I do not kow what is kube_tag . I read that with it I can specify the >>>>> kubernetes version, right ? >>>>> I am using openstack dashboard for creating my cluster: where I can >>>>> specify heat_container_agent_tag: train-stable in the dashboard ? >>>>> Thanks >>>>> Ignazio >>>>> >>>>> Il giorno lun 2 dic 2019 alle ore 16:11 Bharat Kunwar < >>>>> bharat at stackhpc.com> ha scritto: >>>>> >>>>>> Hi Ignazio, >>>>>> >>>>>> What version of kube_tag are you trying to run? What network plugin? >>>>>> I suggest using `heat_container_agent_tag: train-stable` as its easier to >>>>>> debug. >>>>>> >>>>>> >>>>>> Best >>>>>> >>>>>> bharat >>>>>> >>>>>> > On 2 Dec 2019, at 15:08, Ignazio Cassano >>>>>> wrote: >>>>>> > >>>>>> > hello, I've just installed stein with magnum on centos 7. I am >>>>>> creating a kubernetes cluster but it is running since 22 minutes and I >>>>>> think it will not work because the heat stack is waiting >>>>>> OSSoftwareDeployment completes >>>>>> > I presume something is not working on heat with magnum. I ran >>>>>> several stack with Software Deployment without magnum, and they worked fine >>>>>> > >>>>>> > Thanks >>>>>> > Ignazio >>>>>> >>>>>> >>>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Sun Dec 8 20:52:48 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Sun, 8 Dec 2019 21:52:48 +0100 Subject: [all] OpenStack V Release Naming Poll Message-ID: Hey everyone, This is to officially announce the beginning of the CIVS poll to determine the "V" release name. Since the Train release, we use a public poll. This means that rather than having a unique voting URL for each person, all voting can be done using the following URL: https://civs.cs.cornell.edu/cgi-bin/vote.pl?id=E_13ccd49b66cfd1b4&akey=39d9234da798e6a3 A public poll can mean that users behind NAT, proxy servers or firewalls may receive an error message saying that your vote has already been lodged. If this happens please try another IP. Voting will officially end December 16, 2019 at 23:59 UTC. The results of the voting will be shared soon afterwards, but keep in mind there will need to be a marketing and legal review of any selected names before it is finalized. Import Note On Name Review -------------------------- There are a couple steps that I am consolidating from our official process [0]. I believe this to be harmless, but if anyone has any concerns about this please do raise them and we can discuss and avoid any issues for future elections. I will be proposing some updates to clarify our election process either way. First, there is a discrete step for marketing review. I believe that can be done as part of the post-voting name review. Second is to consult with the TC to have names removed. I also believe that can be done as part of the post-voting review, but also think that due to past concerns about this process it is best to allow voting on all of the names first, then determine if something is not appropriate. I think this should be more efficient, but ultimately I determined this course of action due to not allowing enough time for step 6 of the process. For that I want to apologize to everyone sincerely. I do think (and hope) this will not be an issue, but again, please do not hesitate to raise concerns about this. Sean [0] https://governance.openstack.org/tc/reference/release-naming.html From dtroyer at gmail.com Sun Dec 8 22:10:58 2019 From: dtroyer at gmail.com (Dean Troyer) Date: Sun, 8 Dec 2019 16:10:58 -0600 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: References: Message-ID: On Sun, Dec 8, 2019 at 2:57 PM Sean McGinnis wrote: > This is to officially announce the beginning of the CIVS poll to determine the "V" release name. It appears Vidette is on the list twice? > Second is to consult with the TC to have names removed. I also believe that can be done as part of the post-voting review, but also think that due to past concerns about this process it is best to allow voting on all of the names first, then determine if something is not appropriate. FWIW this seems reasonable to me now, I have a hunch that if someone is not happy about their favorite name being removed it may not matter if it occurs before or after voting... dt -- Dean Troyer dtroyer at gmail.com From thierry at openstack.org Mon Dec 9 08:21:25 2019 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Dec 2019 09:21:25 +0100 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: References: Message-ID: Dean Troyer wrote: > [...] >> Second is to consult with the TC to have names removed. I also believe that can be done as part of the post-voting review, but also think that due to past concerns about this process it is best to allow voting on all of the names first, then determine if something is not appropriate. > > FWIW this seems reasonable to me now, I have a hunch that if someone > is not happy about their favorite name being removed it may not matter > if it occurs before or after voting... Speaking from experience, it does matter: they are usually more pissed if their favorite name wins the vote *and* gets removed. -- Thierry From katonalala at gmail.com Mon Dec 9 09:26:43 2019 From: katonalala at gmail.com (Lajos Katona) Date: Mon, 9 Dec 2019 10:26:43 +0100 Subject: High CPU utilization by systemd process || debian 8 In-Reply-To: <0ff601d5acc2$e92a15e0$bb7e41a0$@brilliant.com.bd> References: <0ff601d5acc2$e92a15e0$bb7e41a0$@brilliant.com.bd> Message-ID: Hi, This seems to be unrelated to openstack, perhaps try on some debian or systemd issue tracker (see: https://github.com/systemd/systemd/issues, https://www.debian.org/Bugs/) Regards Lajos Md. Farhad Hasan Khan ezt írta (időpont: 2019. dec. 7., Szo, 7:00): > Hi, > > > > We are observing high CPU utilization by systemd process. > > > > 100% CPU utilize by systemd process when no user is being logged in to the > server but CPU utilization became 0% while any user logged in to the > server. What cloud be possible reason behind this issue and solution? > > > > *Log collected via login to the server via** ssh or console**: (0% CPU > utilization)* > > > > top -b -n 1 | grep systemd | grep 612 > 612 root 20 0 3386692 2.051g 4932 T *0.0* 6.5 22029:26 > systemd > > > > > > *Log collected without login to the server : (100% CPU utilization)* > > > > # ssh user at 192.168.18.11 top -b -n 1 | grep systemd | grep 612 > user at 192.168.18.11's password: > 612 root 20 0 3386692 2.051g 4932 S *789.2* 6.5 22031:31 > systemd > > > > > > *Please find the server information:* > > > > lsb_release -a > No LSB modules are available. > Distributor ID: Debian > Description: Debian GNU/Linux 8.11 (jessie) > Release: 8.11 > Codename: jessie > > *uname -r* > 3.16.0-6-amd64 > > > > Thanks & B’Rgds, > > Rony > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 1306890576 at qq.com Mon Dec 9 12:52:01 2019 From: 1306890576 at qq.com (=?gb18030?B?uqO357S1?=) Date: Mon, 9 Dec 2019 20:52:01 +0800 Subject: =?gb18030?B?ob5pcm9uaWOhvyBib290aW5nIGZyb20gdm9sdW1l?= Message-ID: hi:    The pike version of ironic support the feature: booting from volume. There is a question: when booting from volume, the baremetal node should be reachable to the storage node,but when deploy done, the baremetal's network  completely belong to the vpc. How could the baremetal node booting from volume??     I should add the extar storage nic to communicate with storage node??        expecting for the answer thanks guo -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Mon Dec 9 13:54:36 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 9 Dec 2019 14:54:36 +0100 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: References: Message-ID: > > It appears Vidette is on the list twice? > [head -> desk] Apparently another thing to apologize for. I must have looked at that list a dozen times over the past week watching for updates, but failed to notice it had been added twice. There are quite a few votes cast already. I'd like to hear feedback on how this should be handled. I can stop the current poll, give us a few days, then start a new one. Or we could see what the combined votes look like and if it appears close enough to the top (top 50%?) run a secondary poll to get a cleaner ranking of the top names? Any other ideas? Sorry for not catching that. Sean From sean.mcginnis at gmx.com Mon Dec 9 13:57:20 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 9 Dec 2019 14:57:20 +0100 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: References: Message-ID: > > > > FWIW this seems reasonable to me now, I have a hunch that if someone > > is not happy about their favorite name being removed it may not matter > > if it occurs before or after voting... > > Speaking from experience, it does matter: they are usually more pissed > if their favorite name wins the vote *and* gets removed. > My take was we had the opposite experience the last time around. There were more people upset about the pre-filtering that happened. I think really, no one is going to be happy if the name they like best is removed, whether before or after. I'm hoping we have a clear enough preference here that that doesn't become an issue. From mordred at inaugust.com Mon Dec 9 15:45:45 2019 From: mordred at inaugust.com (Monty Taylor) Date: Mon, 9 Dec 2019 10:45:45 -0500 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: References: Message-ID: <503C9E04-F39B-4D94-8629-2351D5924A57@inaugust.com> > On Dec 9, 2019, at 8:54 AM, Sean McGinnis wrote: > >> >> It appears Vidette is on the list twice? >> > > [head -> desk] > > Apparently another thing to apologize for. I must have looked at that list a dozen times over the past week watching for updates, but failed to notice it had been added twice. > > There are quite a few votes cast already. I'd like to hear feedback on how this should be handled. I can stop the current poll, give us a few days, then start a new one. Or we could see what the combined votes look like and if it appears close enough to the top (top 50%?) run a secondary poll to get a cleaner ranking of the top names? I think that’s the right choice - there’s no need to stress over it until it’s an actual issue. > Any other ideas? > > Sorry for not catching that. > > Sean > > From jeremyfreudberg at gmail.com Mon Dec 9 16:20:26 2019 From: jeremyfreudberg at gmail.com (Jeremy Freudberg) Date: Mon, 9 Dec 2019 11:20:26 -0500 Subject: [sahara] Next Sahara meeting date Message-ID: As a reminder, the Sahara team currently meets every two weeks. A meeting was not held 28 Nov due to the Thanksgiving holiday. A meeting will not be held 12 Dec as I am on PTO. In order to determine the next meeting date we need to answer two questions: - Is there desire to meet on 26 Dec, despite the nearby holidays? - Is a gap in meetings from 14 Nov to 9 Jan acceptable? Depending on those answers, the next Sahara meeting will either be on 26 Dec or 9 Jan. Let me know. Thanks, Jeremy From thierry at openstack.org Mon Dec 9 17:18:55 2019 From: thierry at openstack.org (Thierry Carrez) Date: Mon, 9 Dec 2019 18:18:55 +0100 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: <503C9E04-F39B-4D94-8629-2351D5924A57@inaugust.com> References: <503C9E04-F39B-4D94-8629-2351D5924A57@inaugust.com> Message-ID: <33891169-0e98-d2a8-9e1e-de29048a4a23@openstack.org> Monty Taylor wrote: > > >> On Dec 9, 2019, at 8:54 AM, Sean McGinnis wrote: >> >>> >>> It appears Vidette is on the list twice? >>> >> >> [head -> desk] >> >> Apparently another thing to apologize for. I must have looked at that list a dozen times over the past week watching for updates, but failed to notice it had been added twice. >> >> There are quite a few votes cast already. I'd like to hear feedback on how this should be handled. I can stop the current poll, give us a few days, then start a new one. Or we could see what the combined votes look like and if it appears close enough to the top (top 50%?) run a secondary poll to get a cleaner ranking of the top names? > > I think that’s the right choice - there’s no need to stress over it until it’s an actual issue. +1 In theory it should not change the results at all (as people should rank both Videttes at the exact same rank). -- Thierry From gmann at ghanshyammann.com Mon Dec 9 18:11:47 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 09 Dec 2019 12:11:47 -0600 Subject: [TC] 'V' Technical Elections In-Reply-To: <20191206200207.36kbck272qyj6xul@yuggoth.org> References: <20191206125623.GB670935@sm-workstation> <53db3047-9822-a160-0e96-29956f272191@gmail.com> <20191206200207.36kbck272qyj6xul@yuggoth.org> Message-ID: <16eebdc3553.1128a1503179767.754308089648715960@ghanshyammann.com> ---- On Fri, 06 Dec 2019 14:02:07 -0600 Jeremy Stanley wrote ---- > On 2019-12-06 12:54:00 -0600 (-0600), Jay Bryant wrote: > > > > > The point I was making in another fork of this thread was to tie the > > > exact term length to the election timing and only state approximate > > > lengths here, ie including 'minimum' in the above will still set a bound > > > that complicates accommodating another slightly smaller than usual > > > cycle. > > > > > > > > > I am in agreement with Dean that the easiest and most flexible > > solution is to tie the term lengths to the elections and not state > > a particular minimum or maximum. > [...] > > Does this satisfy what's required by the bylaws though? > > the term for the members of the Technical Committee shall be > approved by a majority of the Technical Committee (“Term”) and > shall be published publicly before each Technical Committee > election; if no such Term is published the Term will be twelve > calendar months. > > https://www.openstack.org/legal/technical-committee-member-policy/ > > I guess it doesn't *strictly* require a term duration to be > specified as an explicit measure of time, so it's possible to > interpret it as allowing terms to be defined according to some other > criteria as long as they don't also exceed 16 months. I agree with mapping the TC term with the election. bylaws say: " the elections for the Technical Committee shall be held in two phases: the first election being for at least half of the members of the Technical Committee and the second election being for the remaining members of Technical Committee." Because Elections dates are not explicitly mentioned (as we do not know the future events final dates) and are divided into two-phase. Current TC charter states 'The election is held no later than 6 weeks prior to each OpenStack Summit' which makes the term length etc tricky. If we can define the specific week for both phases then it can be consistent. I feel R-4 can be fixed week: - 4th weeks prior to each cycle final release date (R-4) is the fixed week to conduct the election. - And make PTL and TC election as a combine election always at R-4. TC term can be documented as a "two-cycle term". [2] https://governance.openstack.org/tc/reference/charter.html -gmann > -- > Jeremy Stanley > From jeremyfreudberg at gmail.com Mon Dec 9 18:27:09 2019 From: jeremyfreudberg at gmail.com (Jeremy Freudberg) Date: Mon, 9 Dec 2019 13:27:09 -0500 Subject: [sahara] Next Sahara meeting date In-Reply-To: <13492368.YTXk2pfZSh@whitebase.usersys.redhat.com> References: <13492368.YTXk2pfZSh@whitebase.usersys.redhat.com> Message-ID: Oops, I meant to write in my original message that 19 Dec is also an option. I think I would also prefer 19 Dec for the date of the next meeting. Let's allow some time for thoughts from the rest of the team but likely I will announce an exceptional meeting for 19 Dec 1400UTC. Not sure if #openstack-meeting-3 is actually available or if we are even allowed to use it for an exceptional meeting. But we can figure that out later... On Mon, Dec 9, 2019 at 1:22 PM Luigi Toscano wrote: > > On Monday, 9 December 2019 17:20:26 CET Jeremy Freudberg wrote: > > As a reminder, the Sahara team currently meets every two weeks. A > > meeting was not held 28 Nov due to the Thanksgiving holiday. A meeting > > will not be held 12 Dec as I am on PTO. > > > > In order to determine the next meeting date we need to answer two questions: > > - Is there desire to meet on 26 Dec, despite the nearby holidays? > > - Is a gap in meetings from 14 Nov to 9 Jan acceptable? > > > > Depending on those answers, the next Sahara meeting will either be on > > 26 Dec or 9 Jan. > > Uhm, could we exceptionally met on Dec 19 ? That would probably solve the > issue. > > I can't totally guarantee to be around on Dec 26, even though it shouldn't be > a big problem, so that's my emergency scenario. > > Ciao > -- > Luigi > > From Albert.Braden at synopsys.com Mon Dec 9 18:40:00 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Mon, 9 Dec 2019 18:40:00 +0000 Subject: neutron-metadata-agent broken pipe Message-ID: Is anyone else seeing this? We are running Rocky. As our production cluster grows in size we are starting to have trouble with neutron-metadata-agent. After restarting it is happy for a minute and then it complains "2019-12-06 17:54:24.615 664587 WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 11, greater than warning threshold: 10. There could be a leak. Increasing threshold to: 20" It increases the threshold a couple of times and then after increasing to 40 we start to see errors: 2019-12-06 17:55:10.119 664578 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe It looks like increasing the threshold to 40 fails because it keeps trying: 2019-12-06 17:55:17.452 664597 WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 21, greater than warning threshold: 20. There could be a leak. Increasing threshold to: 40 And the errors increase until the log is nothing but errors, and VMs fail to boot. root at us01odc-p02-ctrl1:~# tail -f /var/log/neutron/neutron-metadata-agent.log self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe 2019-12-06 17:59:23.664 664597 INFO eventlet.wsgi.server [-] 10.195.73.174, "GET /latest/meta-data/instance-id HTTP/1.0" status: 200 len: 0 time: 62.1517761 2019-12-06 17:59:23.756 664583 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe 2019-12-06 17:59:23.757 664583 INFO eventlet.wsgi.server [-] 10.195.65.69, "GET /latest/meta-data/instance-id HTTP/1.0" status: 200 len: 0 time: 63.0419171 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent [-] Unexpected error.: MessagingTimeout: Timed out waiting for a reply to message ID 77551d4cf4394b7b9cdfad68c0be46e8 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent Traceback (most recent call last): 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 89, in __call__ 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent instance_id, tenant_id = self._get_instance_and_tenant_id(req) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 162, in _get_instance_and_tenant_id 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent ports = self._get_ports(remote_address, network_id, router_id) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 155, in _get_ports 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self._get_ports_for_remote_address(remote_address, networks) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/cache_utils.py", line 116, in __call__ 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self.func(target_self, *args, **kwargs) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 137, in _get_ports_for_remote_address 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent ip_address=remote_address) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 106, in _get_ports_from_server 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self.plugin_rpc.get_ports(self.context, filters) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/agent/metadata/agent.py", line 72, in get_ports 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return cctxt.call(context, 'get_ports', filters=filters) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 173, in call 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent time.sleep(wait) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent self.force_reraise() 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent six.reraise(self.type_, self.value, self.tb) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/neutron/common/rpc.py", line 150, in call 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent return self._original_context.call(ctxt, method, **kwargs) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 179, in call 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent retry=self.retry) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 133, in _send 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent retry=retry) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 584, in send 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent call_monitor_timeout, retry=retry) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 573, in _send 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent call_monitor_timeout) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 459, in wait 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent message = self.waiters.get(msg_id, timeout=timeout) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 336, in get 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent 'to message ID %s' % msg_id) 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent MessagingTimeout: Timed out waiting for a reply to message ID 77551d4cf4394b7b9cdfad68c0be46e8 2019-12-06 17:59:23.776 664595 ERROR neutron.agent.metadata.agent 2019-12-06 17:59:23.778 664595 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) What's causing this? Are we overloading RMQ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennelson11 at gmail.com Mon Dec 9 18:41:13 2019 From: kennelson11 at gmail.com (Kendall Nelson) Date: Mon, 9 Dec 2019 10:41:13 -0800 Subject: [TC] 'V' Technical Elections In-Reply-To: <16eebdc3553.1128a1503179767.754308089648715960@ghanshyammann.com> References: <20191206125623.GB670935@sm-workstation> <53db3047-9822-a160-0e96-29956f272191@gmail.com> <20191206200207.36kbck272qyj6xul@yuggoth.org> <16eebdc3553.1128a1503179767.754308089648715960@ghanshyammann.com> Message-ID: On Mon, Dec 9, 2019 at 10:13 AM Ghanshyam Mann wrote: > ---- On Fri, 06 Dec 2019 14:02:07 -0600 Jeremy Stanley > wrote ---- > > On 2019-12-06 12:54:00 -0600 (-0600), Jay Bryant wrote: > > > > > > > The point I was making in another fork of this thread was to tie the > > > > exact term length to the election timing and only state approximate > > > > lengths here, ie including 'minimum' in the above will still set a > bound > > > > that complicates accommodating another slightly smaller than usual > > > > cycle. > > > > > > > > > > > > > I am in agreement with Dean that the easiest and most flexible > > > solution is to tie the term lengths to the elections and not state > > > a particular minimum or maximum. > > [...] > > > > Does this satisfy what's required by the bylaws though? > > > > the term for the members of the Technical Committee shall be > > approved by a majority of the Technical Committee (“Term”) and > > shall be published publicly before each Technical Committee > > election; if no such Term is published the Term will be twelve > > calendar months. > > > > https://www.openstack.org/legal/technical-committee-member-policy/ > > > > I guess it doesn't *strictly* require a term duration to be > > specified as an explicit measure of time, so it's possible to > > interpret it as allowing terms to be defined according to some other > > criteria as long as they don't also exceed 16 months. > > I agree with mapping the TC term with the election. > > bylaws say: > " the elections for the Technical Committee shall be held in two phases: > the first > election being for at least half of the members of the Technical > Committee and > the second election being for the remaining members of Technical > Committee." > > Because Elections dates are not explicitly mentioned (as we do not know > the future events final dates) > and are divided into two-phase. Current TC charter states 'The election is > held no later than 6 weeks prior to each OpenStack Summit' > which makes the term length etc tricky. If we can define the specific week > for both phases then it can be > consistent. > Just throwing this idea out there (its probably already been discussed at some point).. but what if we just tie it to releases instead of to events since events are way more variable than releases? -Kendall (diablo_rojo) > I feel R-4 can be fixed week: > - 4th weeks prior to each cycle final release date (R-4) is the fixed > week to conduct the election. > - And make PTL and TC election as a combine election always at R-4. > > TC term can be documented as a "two-cycle term". > > [2] https://governance.openstack.org/tc/reference/charter.html > > -gmann > > > -- > > Jeremy Stanley > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Albert.Braden at synopsys.com Mon Dec 9 20:20:24 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Mon, 9 Dec 2019 20:20:24 +0000 Subject: neutron-metadata-agent broken pipe In-Reply-To: References: Message-ID: When I try to build a VM I see this in the VM logs: 2019-12-09 20:02:21,396 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: bad status code [503] 2019-12-09 20:03:41,084 - util.py[WARNING]: Failed fetching userdata from url http://169.254.169.254/2009-04-04/user-data 2019-12-09 12:03:53,041 - util.py[WARNING]: Failed running /var/lib/cloud/scripts/per-boot/config_instance.sh [1] 2019-12-09 12:03:53,043 - cc_scripts_per_boot.py[WARNING]: Failed to run module scripts-per-boot (per-boot in /var/lib/cloud/scripts/per-boot) This is the failing line from the script: name=`curl -s http://169.254.169.254/2009-04-04/meta-data/hostname` When I try this from the VM I get this error: albertb@

503:~ $ curl -s http://169.254.169.254/2009-04-04/meta-data/hostname

503 Service Unavailable

No server is available to handle this request. When I check neutron-metadata-agent.log for the time when the VM was failing I see the "broken pipe" errors: 2019-12-09 11:56:00.075 664593 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe Why is my neutron-metadata server failing? Has anyone else seen this problem? We are running Rocky with about 200 hypervisors; it started after we added 100. From: Albert Braden Sent: Monday, December 9, 2019 10:40 AM To: openstack-discuss at lists.openstack.org Subject: neutron-metadata-agent broken pipe Is anyone else seeing this? We are running Rocky. As our production cluster grows in size we are starting to have trouble with neutron-metadata-agent. After restarting it is happy for a minute and then it complains "2019-12-06 17:54:24.615 664587 WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 11, greater than warning threshold: 10. There could be a leak. Increasing threshold to: 20" It increases the threshold a couple of times and then after increasing to 40 we start to see errors: 2019-12-06 17:55:10.119 664578 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe -------------- next part -------------- An HTML attachment was scrubbed... URL: From 1306890576 at qq.com Mon Dec 9 09:35:38 2019 From: 1306890576 at qq.com (=?gb18030?B?uqO357S1?=) Date: Mon, 9 Dec 2019 17:35:38 +0800 Subject: a question about ironic booting from volume Message-ID: hi:    The pike version of ironic support the feature: booting from volume. There is a question: when booting from volume, the baremetal node should be reachable to the storage node,but when deploy done, the baremetal's network  completely belong to the vpc. How could the baremetal node booting from volume??      I should add the extar storage nic to communicate with storage node??         expecting for the answer thanks guo -------------- next part -------------- An HTML attachment was scrubbed... URL: From rishabh.sikka at hpe.com Mon Dec 9 10:32:19 2019 From: rishabh.sikka at hpe.com (Sikka, Rishabh) Date: Mon, 9 Dec 2019 10:32:19 +0000 Subject: Openstack CI Failing from Friday(6-Dec-2019) Message-ID: Dear All , >From Saturday onwards our openstack CI is failing for ISCSI driver at very early stage of devstack setup and for FC driver it is getting queued. Please let us know if there is any changes are done from community end. >From the failure log and last success log we can conclude with below error -: Failure log -: http://54.201.44.218/53/696853/1/check/3par-iscsi-driver-master-client-pip-ssa02-dsvm/aaeae9a/logs/devstack-early.txt.gz + ./stack.sh:main:224 : [[ ! xenial =~ (bionic|stretch|jessie|f29|opensuse-15.0|opensuse-15.1|opensuse-tumbleweed|rhel7) ]] + ./stack.sh:main:225 : echo 'WARNING: this script has not been tested on xenial' WARNING: this script has not been tested on xenial + ./stack.sh:main:226 : [[ '' != \y\e\s ]] + ./stack.sh:main:227 : die 227 'If you wish to run this script anyway run with FORCE=yes' + functions-common:die:195 : local exitcode=0 + functions-common:die:196 : set +o xtrace [Call Trace] ./stack.sh:227:die [ERROR] ./stack.sh:227 If you wish to run this script anyway run with FORCE=yesnon-zero return code ################################## Success log-: http://54.201.44.218/14/685914/9/check/3par-iscsi-driver-master-client-pip-ssa02-dsvm/21d03ca/logs/devstack-early.txt.gz + ./stack.sh:main:224 : [[ ! xenial =~ (xenial|artful|bionic|stretch|jessie|f29|opensuse-15.0|opensuse-15.1|opensuse-tumbleweed|rhel7) ]] + ./stack.sh:main:235 : export_proxy_variables + functions-common:export_proxy_variables:2154 : isset http_proxy + functions-common:isset:171 : [[ -v http_proxy ]] + functions-common:export_proxy_variables:2155 : export http_proxy=http://192.168.75.1:8118/ + functions-common:export_proxy_variables:2155 : http_proxy=http://192.168.75.1:8118/ + functions-common:export_proxy_variables:2157 : isset https_proxy + functions-common:isset:171 : [[ -v https_proxy ]] -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Mon Dec 9 20:50:17 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Mon, 9 Dec 2019 14:50:17 -0600 Subject: Openstack CI Failing from Friday(6-Dec-2019) In-Reply-To: References: Message-ID: <35b4433d-1189-3fff-7714-f0b025adc294@gmail.com> On 12/9/2019 4:32 AM, Sikka, Rishabh wrote: > From Saturday onwards our openstack CI is failing for ISCSI driver at > very early stage of devstack setup and for FC driver it is getting > queued. Please let us know if there is any changes are done from > community end. > > From the failure log and last success log we can  conclude with below > error -: > > *Failure log -:* > http://54.201.44.218/53/696853/1/check/3par-iscsi-driver-master-client-pip-ssa02-dsvm/aaeae9a/logs/devstack-early.txt.gz > > + ./stack.sh:main:224                      :   [[ ! xenial =~ > (bionic|stretch|jessie|f29|opensuse-15.0|opensuse-15.1|opensuse-tumbleweed|rhel7) > ]] > > + ./stack.sh:main:225                      :   echo 'WARNING: this > script has not been tested on xenial' > > WARNING: this script has not been tested on xenial > > + ./stack.sh:main:226                      :   [[ '' != \y\e\s ]] > > + ./stack.sh:main:227                      :   die 227 'If you wish to > run this script anyway run with FORCE=yes' > > + functions-common:die:195                 :   local exitcode=0 > > + functions-common:die:196                 :   set +o xtrace > > [Call Trace] > > ./stack.sh:227:die > > [ERROR] ./stack.sh:227 If you wish to run this script anyway run with > FORCE=yesnon-zero return code > This looks like your problem: https://github.com/openstack/devstack/commit/2e6677869925c86c01cae883b3dde6cccad81d30 -- Thanks, Matt From jungleboyj at gmail.com Mon Dec 9 21:02:34 2019 From: jungleboyj at gmail.com (Jay Bryant) Date: Mon, 9 Dec 2019 15:02:34 -0600 Subject: [TC] 'V' Technical Elections In-Reply-To: References: <20191206125623.GB670935@sm-workstation> <53db3047-9822-a160-0e96-29956f272191@gmail.com> <20191206200207.36kbck272qyj6xul@yuggoth.org> <16eebdc3553.1128a1503179767.754308089648715960@ghanshyammann.com> Message-ID: <49d95fd8-ab4b-512c-de9f-17f7b5cdc384@gmail.com> On 12/9/2019 12:41 PM, Kendall Nelson wrote: > > On Mon, Dec 9, 2019 at 10:13 AM Ghanshyam Mann > > wrote: > >  ---- On Fri, 06 Dec 2019 14:02:07 -0600 Jeremy Stanley > > wrote ---- >  > On 2019-12-06 12:54:00 -0600 (-0600), Jay Bryant wrote: >  > > >  > > > The point I was making in another fork of this thread was > to tie the >  > > > exact term length to the election timing and only state > approximate >  > > > lengths here, ie including 'minimum' in the above will > still set a bound >  > > > that complicates accommodating another slightly smaller > than usual >  > > > cycle. >  > > > >  > > >  > > >  > > I am in agreement with Dean that the easiest and most flexible >  > > solution is to tie the term lengths to the elections and not > state >  > > a particular minimum or maximum. >  > [...] >  > >  > Does this satisfy what's required by the bylaws though? >  > >  >     the term for the members of the Technical Committee shall be >  >     approved by a majority of the Technical Committee (“Term”) and >  >     shall be published publicly before each Technical Committee >  >     election; if no such Term is published the Term will be twelve >  >     calendar months. >  > >  > https://www.openstack.org/legal/technical-committee-member-policy/ >  > >  > I guess it doesn't *strictly* require a term duration to be >  > specified as an explicit measure of time, so it's possible to >  > interpret it as allowing terms to be defined according to some > other >  > criteria as long as they don't also exceed 16 months. > > I agree with mapping the TC term with the election. > > bylaws say: > " the elections for the Technical Committee shall be held in two > phases: the first >  election being for at least half of the members of the Technical > Committee and > the second election being for the remaining members of Technical > Committee." > > Because Elections dates are not explicitly mentioned (as we do not > know the future events final dates) > and are divided into two-phase. Current TC charter states 'The > election is held no later than 6 weeks prior to each OpenStack Summit' > which makes the term length etc tricky. If we can define the > specific week for both phases then it can be > consistent. > > > Just throwing this idea out there (its probably already been discussed > at some point).. but what if we just tie it to releases instead of to > events since events are way more variable than releases? > > -Kendall (diablo_rojo) Good point.  Given the recent changes and the fact that even schedules have become more variable, it may make sense to follow the release schedule that is remaining relatively static. Jay > > > I feel R-4 can be fixed week: >  - 4th weeks prior to each cycle final release date (R-4) is the > fixed week to conduct the election. >  - And make PTL and TC election as a combine election always at R-4. > > TC term can be documented as a "two-cycle term". > >  [2] https://governance.openstack.org/tc/reference/charter.html > > -gmann > >  > -- >  > Jeremy Stanley >  > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Dec 9 21:07:12 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 9 Dec 2019 22:07:12 +0100 Subject: [stein][octavia] api error Message-ID: Hello All, I've just installed octavia on centos 7 stein, but when I call the octavia command line, the octavia api.log reports: 2019-12-09 22:03:40.986 7925 WARNING keystonemiddleware.auth_token [req-1dea61b8-5c61-4b97-9be5-b1fd8c273984 - - - - -] Identity response: {"error":{"code":401,"message":"The request you have made requires authentication.","title":"Unauthorized"}} : Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-4264ea33-ad5c-46a7-99af-def5570e7e47) 2019-12-09 22:03:41.008 7925 WARNING keystonemiddleware.auth_token [req-1dea61b8-5c61-4b97-9be5-b1fd8c273984 - - - - -] Identity response: {"error":{"code":401,"message":"The request you have made requires authentication.","title":"Unauthorized"}} : Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-02b07d72-431b-421c-9735-3289228e7df1) 2019-12-09 22:03:41.008 7925 CRITICAL keystonemiddleware.auth_token [req-1dea61b8-5c61-4b97-9be5-b1fd8c273984 - - - - -] Unable to validate token: Identity server rejected authorization necessary to fetch token data: ServiceError: Identity server rejected authorization necessary to fetch token data Please, any help ? Regards Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From Albert.Braden at synopsys.com Mon Dec 9 21:11:20 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Mon, 9 Dec 2019 21:11:20 +0000 Subject: neutron-metadata-agent broken pipe In-Reply-To: References: Message-ID: >From my VM, if I try the metadata server repeatedly, it eventually gets a response: albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id

503 Service Unavailable

No server is available to handle this request. albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id

504 Gateway Time-out

The server didn't respond in time. albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id i-000017ccalbertb@

503:~ $ Then I see this in neutron-metadata-agent.log: 2019-12-09 12:41:55.213 266833 INFO eventlet.wsgi.server [-] 10.195.72.75, "GET /2009-04-04/meta-data/instance-id HTTP/1.1" status: 200 len: 146 time: 0.3922560 When it fails (503/504) nothing is logged by Neutron but I see haproxy logs: Dec 9 13:06:14 us01odc-p02-ctrl3 haproxy-metadata-proxy-569e2c42-3935-48e9-ae41-074650e57b2b[267096]: 10.195.72.75:53266 [09/Dec/2019:13:06:14.475] listener listener/metadata 0/0/-1/-1/0 503 212 - - SC-- 58/58/57/57/3 0/0 "GET /2009-04-04/meta-data/instance-id HTTP/1.1" Dec 9 13:06:59 us01odc-p02-ctrl3 haproxy-metadata-proxy-569e2c42-3935-48e9-ae41-074650e57b2b[267096]: 10.195.72.75:53268 [09/Dec/2019:13:06:27.067] listener listener/metadata 0/0/0/-1/32001 504 194 - - sH-- 90/90/89/89/0 0/0 "GET /2009-04-04/meta-data/instance-id HTTP/1.1" The load on my controllers is around 15 but they have 48 CPU so 15 should be OK. When I look at RMQ I see everything fine except for the q-plugin queue; it has 3 consumers but there are thousands of unacked messages and the number gradually increases. Could that be causing the neutron-metadata issue? From: Albert Braden Sent: Monday, December 9, 2019 12:20 PM To: openstack-discuss at lists.openstack.org Subject: RE: neutron-metadata-agent broken pipe When I try to build a VM I see this in the VM logs: 2019-12-09 20:02:21,396 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: bad status code [503] 2019-12-09 20:03:41,084 - util.py[WARNING]: Failed fetching userdata from url http://169.254.169.254/2009-04-04/user-data 2019-12-09 12:03:53,041 - util.py[WARNING]: Failed running /var/lib/cloud/scripts/per-boot/config_instance.sh [1] 2019-12-09 12:03:53,043 - cc_scripts_per_boot.py[WARNING]: Failed to run module scripts-per-boot (per-boot in /var/lib/cloud/scripts/per-boot) This is the failing line from the script: name=`curl -s http://169.254.169.254/2009-04-04/meta-data/hostname` When I try this from the VM I get this error: albertb@

503:~ $ curl -s http://169.254.169.254/2009-04-04/meta-data/hostname

503 Service Unavailable

No server is available to handle this request. When I check neutron-metadata-agent.log for the time when the VM was failing I see the "broken pipe" errors: 2019-12-09 11:56:00.075 664593 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe Why is my neutron-metadata server failing? Has anyone else seen this problem? We are running Rocky with about 200 hypervisors; it started after we added 100. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Mon Dec 9 22:15:34 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 9 Dec 2019 23:15:34 +0100 Subject: [stein][octavia] api error In-Reply-To: References: Message-ID: Sorry, I wrote project_name = services instead of service in octavia.conf Ignazio Il Lun 9 Dic 2019, 22:07 Ignazio Cassano ha scritto: > Hello All, > I've just installed octavia on centos 7 stein, but when I call the octavia > command line, the octavia api.log reports: > > 2019-12-09 22:03:40.986 7925 WARNING keystonemiddleware.auth_token > [req-1dea61b8-5c61-4b97-9be5-b1fd8c273984 - - - - -] Identity response: > {"error":{"code":401,"message":"The request you have made requires > authentication.","title":"Unauthorized"}} > : Unauthorized: The request you have made requires authentication. (HTTP > 401) (Request-ID: req-4264ea33-ad5c-46a7-99af-def5570e7e47) > 2019-12-09 22:03:41.008 7925 WARNING keystonemiddleware.auth_token > [req-1dea61b8-5c61-4b97-9be5-b1fd8c273984 - - - - -] Identity response: > {"error":{"code":401,"message":"The request you have made requires > authentication.","title":"Unauthorized"}} > : Unauthorized: The request you have made requires authentication. (HTTP > 401) (Request-ID: req-02b07d72-431b-421c-9735-3289228e7df1) > 2019-12-09 22:03:41.008 7925 CRITICAL keystonemiddleware.auth_token > [req-1dea61b8-5c61-4b97-9be5-b1fd8c273984 - - - - -] Unable to validate > token: Identity server rejected authorization necessary to fetch token > data: ServiceError: Identity server rejected authorization necessary to > fetch token data > > Please, any help ? > > Regards > Ignazio > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Tue Dec 10 01:57:07 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 10 Dec 2019 02:57:07 +0100 Subject: [all] OpenStack V Release Naming Poll In-Reply-To: <33891169-0e98-d2a8-9e1e-de29048a4a23@openstack.org> References: <503C9E04-F39B-4D94-8629-2351D5924A57@inaugust.com> <33891169-0e98-d2a8-9e1e-de29048a4a23@openstack.org> Message-ID: > >> > >> There are quite a few votes cast already. I'd like to hear feedback on how this should be handled. I can stop the current poll, give us a few days, then start a new one. Or we could see what the combined votes look like and if it appears close enough to the top (top 50%?) run a secondary poll to get a cleaner ranking of the top names? > > > > I think that’s the right choice - there’s no need to stress over it until it’s an actual issue. > > +1 > > In theory it should not change the results at all (as people should rank > both Videttes at the exact same rank). > Thanks for the feedback. I think that's the best route at this point too. I'm sure we'll all be aware of it, but I will make sure to remind the TC about this when the poll closes and share the results. Then based on those final results, we can determine if there is any further action needed. Thanks! Sean From ignaziocassano at gmail.com Tue Dec 10 10:18:41 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 10 Dec 2019 11:18:41 +0100 Subject: [octavia][stein issue Message-ID: Hello, I've just installed stein on centos 7 with octavia. I created the amphora image based on centos , but when a load balance is created the amphora instance does note have any proces on port 9443 and the worker reports: 2019-12-10 08:40:14.456 3710 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: ConnectTimeout: HTTPSConnectionPool(host='10.102.143.94', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(, 'Connection to 10.102.143.94 timed out. (connect timeout=10.0)')) Network connectivity and security groups are ok. I connected on the instance via ssh and I cannot see any listening 9443 port. Anyone can help me, please ? Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From josephine.seifert at secustack.com Tue Dec 10 13:57:47 2019 From: josephine.seifert at secustack.com (Josephine Seifert) Date: Tue, 10 Dec 2019 14:57:47 +0100 Subject: [Image-Encryption] No meeting until January Message-ID: <137964ff-ee1e-d477-4579-3632102a2714@secustack.com> Hello, I will have vacation until January, so the next meeting will take place at 6th of Januar. I wish you all nice holidays and a happy new year :) From rosmaita.fossdev at gmail.com Tue Dec 10 14:56:19 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Tue, 10 Dec 2019 09:56:19 -0500 Subject: [cinder] weekly meeting change reminder Message-ID: (No, I'm not planning to send this out every week ... this will be the final reminder email.) This is a quick reminder that this week's Cinder meeting will be held as usual on Wednesday, but: new time: 1400 UTC new channel: #openstack-meeting-4 See you there! brian From ignaziocassano at gmail.com Tue Dec 10 15:52:42 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 10 Dec 2019 16:52:42 +0100 Subject: [neutron][queens] dhcp issue Message-ID: Hello Everyone, I have an openstack queens installation based on centos 7. I created a shared vlan network and multiple subnet under it. When I start an instance on one of the subnet network it gets the correct ip based on its subnet but the gateway seems random (one of the subnet of the vlan). Any suggestion, please ? Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmail.com Tue Dec 10 16:01:55 2019 From: sean.mcginnis at gmail.com (Sean McGinnis) Date: Tue, 10 Dec 2019 10:01:55 -0600 Subject: [Foundation Board] [OpenStack Foundation] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign In-Reply-To: <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> References: <057BD0AC-451C-4E64-9D30-6CC6A20DFA0F@demarco.com> <1595EA09-2B8C-45ED-B836-5751FC79DBA1@openstack.org> <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> Message-ID: Thanks Allison. And thanks Graham for raising this in the community. On Tue, Dec 10, 2019, 10:53 Allison Randal wrote: > A few more news posts about the topic: > > https://www.theregister.co.uk/2019/11/20/org_registry_sale_shambles/ > > > https://gizmodo.com/private-equity-ghouls-buy-non-profit-that-handles-org-1839860118 > > > http://blogs.harvard.edu/sj/2019/11/23/a-tale-of-icann-and-regulatory-capture-the-dot-org-heist/ > > > https://arstechnica.com/tech-policy/2019/11/private-equity-firm-buys-org-domain-months-after-icann-lifted-price-caps/ > > > https://medium.com/@jacobmalthouse/in-which-i-explain-why-all-of-andrew-sullivans-reasons-for-selling-org-are-wrong-449997d42cac > > https://www.bbc.com/news/technology-50515786 > > HTH, > Allison > > On 11/26/19 12:33 PM, Graham Hayes wrote: > > On 26/11/2019 16:44, Alan Clark wrote: > >> Who owns savedotorg.org? I see EFF references on the page but when I > look it up the domain it appears to all be "REDACTED". > > > > From https://savedotorg.org/index.php/about/ - it is ran by > > NTEN (https://www.nten.org/) > > > > It is supported by the EFF (among others) > > > >> > >>> -----Original Message----- > >>> From: Graham Hayes [mailto:gr at ham.ie] > >>> Sent: Tuesday, November 26, 2019 9:35 AM > >>> To: Jimmy McArthur > >>> Cc: openstack-discuss at lists.openstack.org; foundation- > >>> board at lists.openstack.org; foundation at lists.openstack.org > >>> Subject: Re: [OpenStack Foundation] [tc][board][all] - Adding OpenStack > >>> community support to the savedotorg campaign > >>> > >>> On 26/11/2019 16:16, Jimmy McArthur wrote: > >>>> From my reading of the situation, it’s a done deal. It seems like > ICAAN just > >>> crammed it through. Is there new info out that could reverse it? > >>>> > >>>> Thanks, > >>>> Jimmy > >>> > >>> My understanding is it looks bleak, but there is a chance to shame the > owners > >>> into halting the sale. > >>> > >>> I think ICAAN can still block the same (until December 13th, 30 days > after they > >>> were notified), and through pure poor marketing, and raising our > voices, this > >>> could help convince the current owners of PIR (who is ISOC) to > re-consider the > >>> idea. > >>> > >>> > >>>>> On Nov 26, 2019, at 8:50 AM, Amy wrote: > >>>>> > >>>>> I agree with the OSF submitting on behalf of OpenStack and the > other projects. > >>> I also like the idea of individuals signing as contributors. > >>>>> > >>>>> Amy Marrich (spotz) > >>>>> > >>>>>>> On Nov 26, 2019, at 8:17 AM, Sean McGinnis < > sean.mcginnis at gmail.com> > >>> wrote: > >>>>>>> > >>>>>>> On Tue, Nov 26, 2019 at 7:00 AM Graham Hayes wrote: > >>>>>>> > >>>>>>> Hey All, > >>>>>>> > >>>>>>> I am not sure if this has been seen by everyone or not - but there > >>>>>>> is a change in how the .org top level domain is being ran in the > >>>>>>> works, in a way that may not be in the best interests of the non > >>>>>>> profits that it was created to facilitate. [1] > >>>>>>> > >>>>>>> A lot of well known non profits have already joined, and as a > >>>>>>> community that has an interest in the internet as a whole, and uses > >>>>>>> a .org domain, I think we should add our voice in support. > >>>>>>> > >>>>>>> What do people think? Are we happy to have the TC use its voice on > >>>>>>> behalf of the OpenStack project, or do we think the board should > >>>>>>> use its voice on behalf of the entire foundation? > >>>>>>> > >>>>>>> - Graham > >>>>>>> > >>>>>>> > >>>>>>> 1 - https://savedotorg.org/ > >>>>>>> > >>>>>> > >>>>>> I agree this is a Foundation level thing (though individuals can add > >>>>>> their names to the petition as well). I would support adding OSF to > >>>>>> the list of organizations. > >>>>>> > >>>>>> Sean > >>>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> Foundation mailing list > >>>>> Foundation at lists.openstack.org > >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation > >>>> > >>> > >>> > >>> _______________________________________________ > >>> Foundation mailing list > >>> Foundation at lists.openstack.org > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation > > > > > > _______________________________________________ > > Foundation mailing list > > Foundation at lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation > > > > _______________________________________________ > Foundation-board mailing list > Foundation-board at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation-board > -------------- next part -------------- An HTML attachment was scrubbed... URL: From colleen at gazlene.net Tue Dec 10 17:03:19 2019 From: colleen at gazlene.net (Colleen Murphy) Date: Tue, 10 Dec 2019 09:03:19 -0800 Subject: [keystone] Meeting time change Message-ID: <5308b221-3ba1-4598-9bab-f177fb95a7d9@www.fastmail.com> As discussed in today's meeting[1], I would like to propose we move the team meeting to one hour later, 17:00 UTC, same day same channel. Everyone in attendance today was okay with the change but I wanted to give anyone who couldn't attend today the chance to object. Let me know if this change would prevent you from attending the meeting. If there are no objections this week, I'll propose a change to the eavesdrop calendar. For the moment, assume the meeting next week (December 17) will be at 17:00. Colleen [1] http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-12-10-16.00.log.html#l-18 From johnsomor at gmail.com Tue Dec 10 17:04:02 2019 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 10 Dec 2019 09:04:02 -0800 Subject: [octavia][stein issue In-Reply-To: References: Message-ID: This warning is normal for compute nodes that are slow to boot an instance. As it states, it is "Retrying". I would wait a while and see if the compute service will finish booting the instance. You can also watch it progress by using the compute console. If you are running this compute node inside virtualbox or on some other nested virtualization (without vmx/smx support) scenario nova can take up to 18-20 minutes to fully boot an instance. If the instance does not finish booting and this times out, the load balancer will go into an ERROR state. If that is the case, we will have to debug nova and the image you are using. Michael On Tue, Dec 10, 2019 at 2:21 AM Ignazio Cassano wrote: > > Hello, I've just installed stein on centos 7 with octavia. > I created the amphora image based on centos , but when a load balance is created the amphora instance does note have any proces on port 9443 and the worker reports: > > 2019-12-10 08:40:14.456 3710 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: ConnectTimeout: HTTPSConnectionPool(host='10.102.143.94', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(, 'Connection to 10.102.143.94 timed out. (connect timeout=10.0)')) > > Network connectivity and security groups are ok. > I connected on the instance via ssh and I cannot see any listening 9443 port. > Anyone can help me, please ? > Ignazio From allison at lohutok.net Tue Dec 10 15:52:28 2019 From: allison at lohutok.net (Allison Randal) Date: Tue, 10 Dec 2019 10:52:28 -0500 Subject: [OpenStack Foundation] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign In-Reply-To: <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> References: <057BD0AC-451C-4E64-9D30-6CC6A20DFA0F@demarco.com> <1595EA09-2B8C-45ED-B836-5751FC79DBA1@openstack.org> <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> Message-ID: <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> A few more news posts about the topic: https://www.theregister.co.uk/2019/11/20/org_registry_sale_shambles/ https://gizmodo.com/private-equity-ghouls-buy-non-profit-that-handles-org-1839860118 http://blogs.harvard.edu/sj/2019/11/23/a-tale-of-icann-and-regulatory-capture-the-dot-org-heist/ https://arstechnica.com/tech-policy/2019/11/private-equity-firm-buys-org-domain-months-after-icann-lifted-price-caps/ https://medium.com/@jacobmalthouse/in-which-i-explain-why-all-of-andrew-sullivans-reasons-for-selling-org-are-wrong-449997d42cac https://www.bbc.com/news/technology-50515786 HTH, Allison On 11/26/19 12:33 PM, Graham Hayes wrote: > On 26/11/2019 16:44, Alan Clark wrote: >> Who owns savedotorg.org? I see EFF references on the page but when I look it up the domain it appears to all be "REDACTED". > > From https://savedotorg.org/index.php/about/ - it is ran by > NTEN (https://www.nten.org/) > > It is supported by the EFF (among others) > >> >>> -----Original Message----- >>> From: Graham Hayes [mailto:gr at ham.ie] >>> Sent: Tuesday, November 26, 2019 9:35 AM >>> To: Jimmy McArthur >>> Cc: openstack-discuss at lists.openstack.org; foundation- >>> board at lists.openstack.org; foundation at lists.openstack.org >>> Subject: Re: [OpenStack Foundation] [tc][board][all] - Adding OpenStack >>> community support to the savedotorg campaign >>> >>> On 26/11/2019 16:16, Jimmy McArthur wrote: >>>> From my reading of the situation, it’s a done deal. It seems like ICAAN just >>> crammed it through. Is there new info out that could reverse it? >>>> >>>> Thanks, >>>> Jimmy >>> >>> My understanding is it looks bleak, but there is a chance to shame the owners >>> into halting the sale. >>> >>> I think ICAAN can still block the same (until December 13th, 30 days after they >>> were notified), and through pure poor marketing, and raising our voices, this >>> could help convince the current owners of PIR (who is ISOC) to re-consider the >>> idea. >>> >>> >>>>> On Nov 26, 2019, at 8:50 AM, Amy wrote: >>>>> >>>>> I agree with the OSF submitting on behalf of OpenStack and the other projects. >>> I also like the idea of individuals signing as contributors. >>>>> >>>>> Amy Marrich (spotz) >>>>> >>>>>>> On Nov 26, 2019, at 8:17 AM, Sean McGinnis >>> wrote: >>>>>>> >>>>>>> On Tue, Nov 26, 2019 at 7:00 AM Graham Hayes wrote: >>>>>>> >>>>>>> Hey All, >>>>>>> >>>>>>> I am not sure if this has been seen by everyone or not - but there >>>>>>> is a change in how the .org top level domain is being ran in the >>>>>>> works, in a way that may not be in the best interests of the non >>>>>>> profits that it was created to facilitate. [1] >>>>>>> >>>>>>> A lot of well known non profits have already joined, and as a >>>>>>> community that has an interest in the internet as a whole, and uses >>>>>>> a .org domain, I think we should add our voice in support. >>>>>>> >>>>>>> What do people think? Are we happy to have the TC use its voice on >>>>>>> behalf of the OpenStack project, or do we think the board should >>>>>>> use its voice on behalf of the entire foundation? >>>>>>> >>>>>>> - Graham >>>>>>> >>>>>>> >>>>>>> 1 - https://savedotorg.org/ >>>>>>> >>>>>> >>>>>> I agree this is a Foundation level thing (though individuals can add >>>>>> their names to the petition as well). I would support adding OSF to >>>>>> the list of organizations. >>>>>> >>>>>> Sean >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Foundation mailing list >>>>> Foundation at lists.openstack.org >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation >>>> >>> >>> >>> _______________________________________________ >>> Foundation mailing list >>> Foundation at lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation > > > _______________________________________________ > Foundation mailing list > Foundation at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation > From fungi at yuggoth.org Tue Dec 10 17:32:14 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 10 Dec 2019 17:32:14 +0000 Subject: [dev] Counting the chaff Message-ID: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> It happens with some regularity that I or others are asked for code change volume numbers on OpenStack to compare its activity level against similar numbers published by other open-source projects, and I pretty much always wind up needing to couch my answers with the fact that commit counts published by projects following different workflows aren't directly comparable. Most often, these other projects employ an additive pull request model common to GitHub and similar platforms, which causes many times more commits to appear in their branch histories than happens with a rebase or squash model. (Note that there *are* plenty of high-profile projects on GitHub like Kubernetes using more of a rebase model, with PR owners typically force-pushing changes to address reviewer comments or CI results, and also projects where it's their custom to squash all commits in a PR before merging, so in those cases our normal change counts are more applicable for ~1:1 comparisons.) In order to be able to provide more useful data, I decided to try and find a way of approximating the "chaff" we discard when we merge the "wheat" of our rebased code changes. The Gerrit API provides information on the revision count for changes, so adding the number of revisions for each merged change basically gives us the answer. OpenStack's election tooling already tallies counts of changes merged, so adding an aggregate revision counter to that was fairly trivial: https://review.opendev.org/698291 A word on merge commits: While a direct count of Git commits will be additionally inflated by the merge commits used to stitch changes into a branch, and approach a 1:1 ratio with the number of changes merged in higher-volume projects (except possibly in projects squashing or cherry-picking/rebasing at merge time), GitHub's commit counts explicitly exclude merge commits in order to provide more useful metrics, therefore we don't need to factor them into our calculations: https://developer.github.com/v3/repos/statistics/#statistics-exclude-some-types-of-commits So with all that explanation out of the way (and thanks to those of you who read this far!), it's time for some numbers. Looking at the past five years, here's the breakdown... 1409697 revisions / 459770 changes ~= 3 Now this was calculated using the current OpenStack governance data so the exact change counts aren't official numbers because they could include data from a slightly different set of Git repositories than were official in those years, but the goal of this exercise was to obtain a rough coefficient so I think that's a reasonable simplification. This "GitHub coefficient" of 3 is then a number by which OpenStack change volume can be multiplied to provide a rough approximation for comparison with activity numbers for projects following the common GitHub iterative PR workflow (at least for projects which see similar levels of PR updating to OpenStack's change revision frequency). Out of curiosity, I also took a look at how that coefficient has evolved over time... 2015: 3.35 2016: 3.12 2017: 3.05 2018: 2.77 2019: 2.76 It's interesting to note that OpenStack's overall chaff-to-wheat ratio has been trending downward. This could be due to a number of factors. Are contributors getting better instruction and feedback to help them understand what reviewers expect? Is our project gating automation catching more errors and wasting less developer time on each change? Are reviewers possibly becoming more lax and fixing nits in subsequent changes rather than demanding more revisions to otherwise "good enough for now" changes? Are we seeing a rise in projects with lower affiliation diversity resulting in changes merging with minimal or no review? It could be any or all of these. I tried to dig back even farther to see if this pattern was consistent. What I found, in retrospect, should perhaps not be all that surprising... 2011: 1.79 2012: 2.12 2013: 2.72 2014: 3.18 Okay, so 2011/2012 are much lower, perhaps because contributors were still for the most part getting used to the ideas of code review and pre-merge testing in the first few years of the project. The data we have in Gerrit about this earlier period is also somewhat less reliable anyway, but it's worth pointing out that the curve for revisions-per-change roughly follows the curve for total change volume (peaking around 2014-2016). So maybe the answer is that when we try to move faster we become less efficient? Additional rebases due to a higher incidence of merge conflicts could be an explanation there. The patch which adds revision counting makes it possible to calculate change revisions for deliverables of different teams as well. These are the five teams with the highest and five with the lowest coefficients for this year... 12.5: I18n 5.58: Nova 5.33: OpenStack Helm 5.15: Cyborg 4.27: Storlets ... 1.74: SIGs (constructed "team" encompassing all SIG-owned repos) 1.73: Murano 1.64: Release Management 1.56: Puppet OpenStack 1.56: Requirements You can of course draw all sorts of conclusions from a comparison like this. Clearly there's something about the I18n team's changes which result in a very high revision count, but it's not immediately clear to me what that is (they only have a two repositories, one of which is basically unused and the other mostly just merges infrequent bot-proposed changes). It's also possible that the Nova team's reputation for being thorough reviewers is a well-deserved one. On the other hand some teams like Release Management and Requirements with well-structured, policy-oriented changes tend to merge at the first or second iteration on average. Hopefully some of you find this useful. For me, it's an interesting way to see evidence of some of the additional work our community is doing when it comes to making changes to its software, which isn't necessarily evident from the basic change counts we normally see published. It likely raises more questions than it answers, but I think the introspection it drives is also a healthy exercise for the community. One final note, I've got a trimmed-down and compressed archive of the data from which the above numbers were extracted I can forward to anyone wants to do their own digging, just let me know if you would like a copy. I was going to attach it to this analysis but that would have resulted in a >300KB message to the mailing list, so I thought better of that plan. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From jeremyfreudberg at gmail.com Tue Dec 10 17:35:09 2019 From: jeremyfreudberg at gmail.com (Jeremy Freudberg) Date: Tue, 10 Dec 2019 12:35:09 -0500 Subject: [sahara] Sahara meetings for the rest of 2019 - dates confirmed Message-ID: The next Sahara meeting will be December 19, 2019 at 1400 UTC in #openstack-sahara. (There will be no Sahara meeting December 12, 2019 or December 26, 2019.) From ignaziocassano at gmail.com Tue Dec 10 18:09:31 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Tue, 10 Dec 2019 19:09:31 +0100 Subject: [octavia][stein issue In-Reply-To: References: Message-ID: Hello Michael, I created the amphora image with the package supplyed by stein on centos ( I created an amphora centos). Using the git version of octavia on ubuntu, I created an ubuntu image and it works fine. Thanks ignazio Il giorno mar 10 dic 2019 alle ore 18:04 Michael Johnson < johnsomor at gmail.com> ha scritto: > This warning is normal for compute nodes that are slow to boot an > instance. As it states, it is "Retrying". > I would wait a while and see if the compute service will finish > booting the instance. You can also watch it progress by using the > compute console. > > If you are running this compute node inside virtualbox or on some > other nested virtualization (without vmx/smx support) scenario nova > can take up to 18-20 minutes to fully boot an instance. > > If the instance does not finish booting and this times out, the load > balancer will go into an ERROR state. If that is the case, we will > have to debug nova and the image you are using. > > Michael > > On Tue, Dec 10, 2019 at 2:21 AM Ignazio Cassano > wrote: > > > > Hello, I've just installed stein on centos 7 with octavia. > > I created the amphora image based on centos , but when a load balance > is created the amphora instance does note have any proces on port 9443 and > the worker reports: > > > > 2019-12-10 08:40:14.456 3710 WARNING > octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to > instance. Retrying.: ConnectTimeout: > HTTPSConnectionPool(host='10.102.143.94', port=9443): Max retries exceeded > with url: // (Caused by > ConnectTimeoutError( 0x7f4a04de0d10>, 'Connection to 10.102.143.94 timed out. (connect > timeout=10.0)')) > > > > Network connectivity and security groups are ok. > > I connected on the instance via ssh and I cannot see any listening 9443 > port. > > Anyone can help me, please ? > > Ignazio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Albert.Braden at synopsys.com Tue Dec 10 18:10:02 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 10 Dec 2019 18:10:02 +0000 Subject: neutron-metadata-agent broken pipe In-Reply-To: References: Message-ID: It looks like we may be encountering this bug: https://bugs.launchpad.net/neutron/+bug/1853071 I'm testing this patch now. https://review.opendev.org/#/c/697405/ From: Albert Braden Sent: Monday, December 9, 2019 1:11 PM To: openstack-discuss at lists.openstack.org Subject: RE: neutron-metadata-agent broken pipe >From my VM, if I try the metadata server repeatedly, it eventually gets a response: albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id

503 Service Unavailable

No server is available to handle this request. albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id

504 Gateway Time-out

The server didn't respond in time. albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id i-000017ccalbertb@

503:~ $ Then I see this in neutron-metadata-agent.log: 2019-12-09 12:41:55.213 266833 INFO eventlet.wsgi.server [-] 10.195.72.75, "GET /2009-04-04/meta-data/instance-id HTTP/1.1" status: 200 len: 146 time: 0.3922560 When it fails (503/504) nothing is logged by Neutron but I see haproxy logs: Dec 9 13:06:14 us01odc-p02-ctrl3 haproxy-metadata-proxy-569e2c42-3935-48e9-ae41-074650e57b2b[267096]: 10.195.72.75:53266 [09/Dec/2019:13:06:14.475] listener listener/metadata 0/0/-1/-1/0 503 212 - - SC-- 58/58/57/57/3 0/0 "GET /2009-04-04/meta-data/instance-id HTTP/1.1" Dec 9 13:06:59 us01odc-p02-ctrl3 haproxy-metadata-proxy-569e2c42-3935-48e9-ae41-074650e57b2b[267096]: 10.195.72.75:53268 [09/Dec/2019:13:06:27.067] listener listener/metadata 0/0/0/-1/32001 504 194 - - sH-- 90/90/89/89/0 0/0 "GET /2009-04-04/meta-data/instance-id HTTP/1.1" The load on my controllers is around 15 but they have 48 CPU so 15 should be OK. When I look at RMQ I see everything fine except for the q-plugin queue; it has 3 consumers but there are thousands of unacked messages and the number gradually increases. Could that be causing the neutron-metadata issue? From: Albert Braden > Sent: Monday, December 9, 2019 12:20 PM To: openstack-discuss at lists.openstack.org Subject: RE: neutron-metadata-agent broken pipe When I try to build a VM I see this in the VM logs: 2019-12-09 20:02:21,396 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: bad status code [503] 2019-12-09 20:03:41,084 - util.py[WARNING]: Failed fetching userdata from url http://169.254.169.254/2009-04-04/user-data 2019-12-09 12:03:53,041 - util.py[WARNING]: Failed running /var/lib/cloud/scripts/per-boot/config_instance.sh [1] 2019-12-09 12:03:53,043 - cc_scripts_per_boot.py[WARNING]: Failed to run module scripts-per-boot (per-boot in /var/lib/cloud/scripts/per-boot) This is the failing line from the script: name=`curl -s http://169.254.169.254/2009-04-04/meta-data/hostname` When I try this from the VM I get this error: albertb@

503:~ $ curl -s http://169.254.169.254/2009-04-04/meta-data/hostname

503 Service Unavailable

No server is available to handle this request. When I check neutron-metadata-agent.log for the time when the VM was failing I see the "broken pipe" errors: 2019-12-09 11:56:00.075 664593 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe Why is my neutron-metadata server failing? Has anyone else seen this problem? We are running Rocky with about 200 hypervisors; it started after we added 100. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mgagne at calavera.ca Tue Dec 10 18:31:08 2019 From: mgagne at calavera.ca (=?UTF-8?Q?Mathieu_Gagn=C3=A9?=) Date: Tue, 10 Dec 2019 13:31:08 -0500 Subject: [searchlight] Elasticsearch 5 and latest support Message-ID: Hi, I'm reading the documentation for Searchlight and it's recommending ElasticSearch 5.x: https://docs.openstack.org/searchlight/latest/install/elasticsearch.html I installed and tried to use Elasticsearch 5 and it doesn't work. The queries sent by Searchlight aren't accepted by Elasticsearch. For examples: When listing facets: TransportError(400, u'parsing_exception', u'[terms] failed to parse field [size]'): RequestError: TransportError(400, u'parsing_exception', u'[terms] failed to parse field [size]') When searching for images: No registered plugin for type owner: Exception: No registered plugin for type owner Is there any work in progress to add support for Elasticsearch 5? In fact, Elasticsearch 5 is EOL. Are there plans to support later versions? Thanks -- Mathieu From mriedemos at gmail.com Tue Dec 10 18:55:17 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 10 Dec 2019 12:55:17 -0600 Subject: [dev] Counting the chaff In-Reply-To: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> References: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> Message-ID: <3a5554f6-221d-0e86-edc7-dc695db954eb@gmail.com> On 12/10/2019 11:32 AM, Jeremy Stanley wrote: > It's also possible that the Nova > team's reputation for being thorough reviewers is a well-deserved > one. Hell yeah! -- Thanks, Matt From Albert.Braden at synopsys.com Tue Dec 10 19:03:50 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 10 Dec 2019 19:03:50 +0000 Subject: neutron-metadata-agent broken pipe In-Reply-To: References: Message-ID: That patch didn't fix it. It looks like we have a different issue. Can anyone help? From: Albert Braden Sent: Tuesday, December 10, 2019 10:10 AM To: openstack-discuss at lists.openstack.org Subject: RE: neutron-metadata-agent broken pipe It looks like we may be encountering this bug: https://bugs.launchpad.net/neutron/+bug/1853071 I'm testing this patch now. https://review.opendev.org/#/c/697405/ From: Albert Braden Sent: Monday, December 9, 2019 1:11 PM To: openstack-discuss at lists.openstack.org Subject: RE: neutron-metadata-agent broken pipe >From my VM, if I try the metadata server repeatedly, it eventually gets a response: albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id

503 Service Unavailable

No server is available to handle this request. albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id

504 Gateway Time-out

The server didn't respond in time. albertb@

503:~ $ curl http://169.254.169.254/2009-04-04/meta-data/instance-id i-000017ccalbertb@

503:~ $ Then I see this in neutron-metadata-agent.log: 2019-12-09 12:41:55.213 266833 INFO eventlet.wsgi.server [-] 10.195.72.75, "GET /2009-04-04/meta-data/instance-id HTTP/1.1" status: 200 len: 146 time: 0.3922560 When it fails (503/504) nothing is logged by Neutron but I see haproxy logs: Dec 9 13:06:14 us01odc-p02-ctrl3 haproxy-metadata-proxy-569e2c42-3935-48e9-ae41-074650e57b2b[267096]: 10.195.72.75:53266 [09/Dec/2019:13:06:14.475] listener listener/metadata 0/0/-1/-1/0 503 212 - - SC-- 58/58/57/57/3 0/0 "GET /2009-04-04/meta-data/instance-id HTTP/1.1" Dec 9 13:06:59 us01odc-p02-ctrl3 haproxy-metadata-proxy-569e2c42-3935-48e9-ae41-074650e57b2b[267096]: 10.195.72.75:53268 [09/Dec/2019:13:06:27.067] listener listener/metadata 0/0/0/-1/32001 504 194 - - sH-- 90/90/89/89/0 0/0 "GET /2009-04-04/meta-data/instance-id HTTP/1.1" The load on my controllers is around 15 but they have 48 CPU so 15 should be OK. When I look at RMQ I see everything fine except for the q-plugin queue; it has 3 consumers but there are thousands of unacked messages and the number gradually increases. Could that be causing the neutron-metadata issue? From: Albert Braden > Sent: Monday, December 9, 2019 12:20 PM To: openstack-discuss at lists.openstack.org Subject: RE: neutron-metadata-agent broken pipe When I try to build a VM I see this in the VM logs: 2019-12-09 20:02:21,396 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: bad status code [503] 2019-12-09 20:03:41,084 - util.py[WARNING]: Failed fetching userdata from url http://169.254.169.254/2009-04-04/user-data 2019-12-09 12:03:53,041 - util.py[WARNING]: Failed running /var/lib/cloud/scripts/per-boot/config_instance.sh [1] 2019-12-09 12:03:53,043 - cc_scripts_per_boot.py[WARNING]: Failed to run module scripts-per-boot (per-boot in /var/lib/cloud/scripts/per-boot) This is the failing line from the script: name=`curl -s http://169.254.169.254/2009-04-04/meta-data/hostname` When I try this from the VM I get this error: albertb@

503:~ $ curl -s http://169.254.169.254/2009-04-04/meta-data/hostname

503 Service Unavailable

No server is available to handle this request. When I check neutron-metadata-agent.log for the time when the VM was failing I see the "broken pipe" errors: 2019-12-09 11:56:00.075 664593 INFO eventlet.wsgi.server [-] Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 521, in handle_one_response write(b''.join(towrite)) File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 462, in write wfile.flush() File "/usr/lib/python2.7/socket.py", line 307, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 390, in sendall tail = self.send(data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send return self._send_loop(self.fd.send, data, flags) File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop return send_method(data, *args) error: [Errno 32] Broken pipe Why is my neutron-metadata server failing? Has anyone else seen this problem? We are running Rocky with about 200 hypervisors; it started after we added 100. -------------- next part -------------- An HTML attachment was scrubbed... URL: From openstack at fried.cc Tue Dec 10 19:11:52 2019 From: openstack at fried.cc (Eric Fried) Date: Tue, 10 Dec 2019 13:11:52 -0600 Subject: [dev] Counting the chaff In-Reply-To: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> References: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> Message-ID: <9c3b6328-5176-b60a-b1fb-777b9d1ae4e9@fried.cc> Are we counting number of commits, or lines of code? If the latter, the calculation of the coefficient ought to take into account that larger commits almost certainly have more revisions (I'm assuming -- I haven't checked). efried . From fungi at yuggoth.org Tue Dec 10 19:21:45 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 10 Dec 2019 19:21:45 +0000 Subject: [dev] Counting the chaff In-Reply-To: <9c3b6328-5176-b60a-b1fb-777b9d1ae4e9@fried.cc> References: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> <9c3b6328-5176-b60a-b1fb-777b9d1ae4e9@fried.cc> Message-ID: <20191210192145.np3wkgssmlttusea@yuggoth.org> On 2019-12-10 13:11:52 -0600 (-0600), Eric Fried wrote: > Are we counting number of commits, or lines of code? If the > latter, the calculation of the coefficient ought to take into > account that larger commits almost certainly have more revisions > (I'm assuming -- I haven't checked). This was purely the number of times any change was updated with a new revision prior to merging. Yes absolutely there are a ton of hidden variables there and I agree that larger changes are more prone to see more revisions due to there being more to get wrong in them and the increased chance that they'll end up in conflict with other changes which merge ahead of them. However, I don't think there's anything significantly different about that particular detail when you're comparing a Gerrit change review workflow to a GitHub additive commit review workflow, which was the main point of this analysis. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From dvallis at amperecomputing.com Tue Dec 10 18:25:12 2019 From: dvallis at amperecomputing.com (Darrin Vallis) Date: Tue, 10 Dec 2019 18:25:12 +0000 Subject: Work around for bug 1851516: grub2-mkconfig with ARM ? Message-ID: Hello We are using diskimage-builder to generate an ARM UEFI ISO for the Akraino REC blueprint. We have exactly the same problem documented here: https://bugs.launchpad.net/diskimage-builder/+bug/1851516 When our script uses grub2-mkconfig, grub.cfg is not relocated as shown grub2-mkconfig -0 /boot/efi/EFI/centos/grub.cfg The image is built with /boot/grub2/grub.cfg instead. Any thoughts on a work around for this? Thanks Darrin Vallis Ampere FAE | Austin TX 512-791-5908 CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and contains information that is confidential and proprietary to Ampere Computing or its subsidiaries. It is to be used solely for the purpose of furthering the parties' business relationship. Any review, copying, or distribution of this email (or any attachments thereto) is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email and any attachments thereto. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dvallis at amperecomputing.com Tue Dec 10 18:33:00 2019 From: dvallis at amperecomputing.com (Darrin Vallis) Date: Tue, 10 Dec 2019 18:33:00 +0000 Subject: Work around for bug 1851516: grub2-mkconfig with ARM ? (Resubmit) Message-ID: Hello We are using diskimage-builder to generate an ARM UEFI ISO for the Akraino REC blueprint. We have exactly the same problem documented here: https://bugs.launchpad.net/diskimage-builder/+bug/1851516 When our script uses grub2-mkconfig, grub.cfg is not relocated as shown grub2-mkconfig -0 /boot/efi/EFI/centos/grub.cfg The image is built with /boot/grub2/grub.cfg instead. Any thoughts on a work around for this? Thanks Darrin Vallis Ampere FAE | Austin TX 512-791-5908 CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and contains information that is confidential and proprietary to Ampere Computing or its subsidiaries. It is to be used solely for the purpose of furthering the parties' business relationship. Any review, copying, or distribution of this email (or any attachments thereto) is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email and any attachments thereto. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cboylan at sapwetik.org Tue Dec 10 20:14:46 2019 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 10 Dec 2019 12:14:46 -0800 Subject: Work around for bug 1851516: grub2-mkconfig with ARM ? In-Reply-To: References: Message-ID: <427864c3-f522-438d-b0ff-3790469a3f4c@www.fastmail.com> On Tue, Dec 10, 2019, at 10:25 AM, Darrin Vallis wrote: > > Hello > > > We are using diskimage-builder to generate an ARM UEFI ISO for the > Akraino REC blueprint. We have exactly the same problem documented here: > > > https://bugs.launchpad.net/diskimage-builder/+bug/1851516 > > > When our script uses grub2-mkconfig, grub.cfg is not relocated as shown > > > grub2-mkconfig -0 /boot/efi/EFI/centos/grub.cfg > > > The image is built with /boot/grub2/grub.cfg instead. > > > Any thoughts on a work around for this? Are you able to share the element list that is being used to produce the image? Also, it would be helpful if you can share the build logs as well. Quickly looking at DIB's code I think the path you don't want may be hardcoded by the bootloader element [0]. Are you running grub2-mkconfig outside of this element? If so it will need to run in the finalise stage with a prefix that sorts after 50. Other idea: We know this generally works on Debian and Ubuntu because our CI system builds UEFI arm64 images for Debian and Ubuntu and boots them to run tests. This could be a behavior difference on CentOS we need to track down. [0] https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/bootloader/finalise.d/50-bootloader#L161-L209 > > > Thanks > > Darrin Vallis > > Ampere FAE | Austin TX > > 512-791-5908 From dangtrinhnt at gmail.com Wed Dec 11 01:32:19 2019 From: dangtrinhnt at gmail.com (Trinh Nguyen) Date: Wed, 11 Dec 2019 10:32:19 +0900 Subject: [searchlight] Elasticsearch 5 and latest support In-Reply-To: References: Message-ID: Hi Mathieu, We just found a bug and I'm trying to fix it. We do have a plan to support a later version like 6.x or 7.x but It's become a little bit complicated when I just changed job a couple of months ago. If you or someone can give me a hand, it would be much appreciated. Bests, On Wed, Dec 11, 2019 at 3:35 AM Mathieu Gagné wrote: > Hi, > > I'm reading the documentation for Searchlight and it's recommending > ElasticSearch 5.x: > https://docs.openstack.org/searchlight/latest/install/elasticsearch.html > > I installed and tried to use Elasticsearch 5 and it doesn't work. The > queries sent by Searchlight aren't accepted by Elasticsearch. For > examples: > > When listing facets: > TransportError(400, u'parsing_exception', u'[terms] failed to parse > field [size]'): RequestError: TransportError(400, > u'parsing_exception', u'[terms] failed to parse field [size]') > > When searching for images: > No registered plugin for type owner: Exception: No registered plugin > for type owner > > Is there any work in progress to add support for Elasticsearch 5? > In fact, Elasticsearch 5 is EOL. Are there plans to support later versions? > > Thanks > > -- > Mathieu > > -- *Trinh Nguyen* *www.edlab.xyz * -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Wed Dec 11 09:10:17 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Wed, 11 Dec 2019 10:10:17 +0100 Subject: [neutron][queens] dhcp issue In-Reply-To: References: Message-ID: Slawek, I got same errors in /var/log/messages: Dec 11 09:49:06 podto2-osctrl01 dnsmasq-dhcp[70093]: cannot send DHCP/BOOTP option 121: no space left in packet Il giorno mer 11 dic 2019 alle ore 09:19 Slawek Kaplonski < skaplons at redhat.com> ha scritto: > Hi, > > Can You maybe give us some more details? Like how exactly this network and > subnets looks like (neutron net-show and neutron subnet-show), and please > also check dnsmasq config files - those are located on node where DHCP > agent is running. If You will do something like: > > ps aux | grep > > You will find dnsmasq process for this network. > > > On 10 Dec 2019, at 16:52, Ignazio Cassano > wrote: > > > > Hello Everyone, > > I have an openstack queens installation based on centos 7. > > I created a shared vlan network and multiple subnet under it. > > When I start an instance on one of the subnet network it gets the > correct ip based on its subnet but the gateway seems random (one of the > subnet of the vlan). > > > > Any suggestion, please ? > > > > Ignazio > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Wed Dec 11 15:23:32 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 11 Dec 2019 15:23:32 +0000 Subject: [neutron][queens] dhcp issue In-Reply-To: References: Message-ID: <20191211152331.xmre63uzald5t4tg@yuggoth.org> On 2019-12-11 10:10:17 +0100 (+0100), Ignazio Cassano wrote: [...] > Dec 11 09:49:06 podto2-osctrl01 dnsmasq-dhcp[70093]: cannot send > DHCP/BOOTP option 121: no space left in packet [...] This error is fairly straightforward. DHCP replies are limited to what you can fit in a single UDP packet and how large of a datagram the client agrees to accept. The number of routes you're trying to include in via option 121 could be the problem (is it a bunch?) but also the other fields and their lengths can also contribute, as can the MTU on that network segment: https://tools.ietf.org/html/rfc2131 According to IETF RFC 2131 you should be able to assume clients will accept a datagram of up to 576 octets in length (which implies your entire options field can use at least 312 octets). Anything over this requires negotiation of a maximum DHCP message size with the client. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From openstack at nemebean.com Wed Dec 11 15:58:27 2019 From: openstack at nemebean.com (Ben Nemec) Date: Wed, 11 Dec 2019 09:58:27 -0600 Subject: [oslo] Schedule for the rest of 2019 Message-ID: Hi, I said in the meeting I would send this out Monday, but then I didn't. Sorry. :-) I mentioned I was probably going to be out some this week and it looks like I'll be taking the afternoons off. How many? Who knows! Planning is really hard right now. Probably at least the rest of this week, and possibly into next week. So TLDR: If you need to get ahold of me, do it in the morning. I may not be available after that. I also expect the 20th to be my last working day for the year, so I'm going to take this opportunity to cancel the Oslo meetings on the 23rd and 30th. We'll resume in the new year on the 6th. Thanks. -Ben From openstack at fried.cc Wed Dec 11 16:12:54 2019 From: openstack at fried.cc (Eric Fried) Date: Wed, 11 Dec 2019 10:12:54 -0600 Subject: [oslo] Schedule for the rest of 2019 In-Reply-To: References: Message-ID: > So TLDR: If you need to get ahold of me, do it in the morning. What's "morning" to you (TZ)? efried . From m.hasanshafiq at gmail.com Wed Dec 11 04:41:09 2019 From: m.hasanshafiq at gmail.com (Muhammad Hasan) Date: Wed, 11 Dec 2019 04:41:09 +0000 Subject: QUERY ONMONASCA Message-ID: Hi, Can you share any hands on exercise in monasca api and monasca agent -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Wed Dec 11 08:39:56 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Wed, 11 Dec 2019 09:39:56 +0100 Subject: [neutron][queens] dhcp issue In-Reply-To: References: Message-ID: Thanks for your help: [root at podto2-osctrl01 ~]# neutron net-list neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ | id | name | tenant_id | subnets | +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ | 00361d96-0ea2-4378-8909-dbf7ef4c883f | NVLP2-Provider-backoffice1 | 85cace94dcc7484c85ff9337eb1d0c4c | 9ba2a03d-3f95-494f-bd38-7fe85471465d 10.138.179.0/24 | | 28ac0a79-17a9-46f5-a418-77f386b821b5 | NVLP2-Provider-Bk | 85cace94dcc7484c85ff9337eb1d0c4c | 0705b470-ce99-46c8-944c-cfbfc6a2a5f7 10.138.177.0/24 | | 2e3edf38-7648-4ac8-a8d7-11448adedcca | NVLP2-Ext-Internet | 85cace94dcc7484c85ff9337eb1d0c4c | ff797a1f-7de4-4998-9bdc-29d1241ffb5b 84.240.191.0/24 | | 34a3e80d-a662-4665-ad6f-a7a83e2be346 | NVLP2-Ext-Rupar | 85cace94dcc7484c85ff9337eb1d0c4c | 1a88e004-7062-4702-8965-b18ea9d6af87 10.138.181.0/24 | | 435e5609-c91a-4e42-bd1a-87dbc24e4afa | HA network tenant 85cace94dcc7484c85ff9337eb1d0c4c | | 5bbb31c3-a2cd-4e30-a194-60cee122d0e9 169.254.192.0/18 | | 7f41aa95-b108-4224-9088-21e32d554646 | admin-self-net | 85cace94dcc7484c85ff9337eb1d0c4c | 272ca9b5-2aa0-45d2-90b4-70abe4feac49 192.168.90.0/24 | | | | | ea1c69b8-ded6-4028-bba9-63db12213e36 192.168.100.0/24 | | 8b876fb6-10a3-4e92-a499-f62a9d3aa32b | net-vx4-private | 4e44aad117b64a34806c4785b31c3f88 | dcd2ad5c-afb5-420b-bfa8-7b89ab511f5b 192.168.213.0/24 | | 993f14c9-1b40-47b7-8ee5-fcb00ef92d48 | HA network tenant 4e44aad117b64a34806c4785b31c3f88 | | d2129401-0349-4da3-87fa-5e521b071377 169.254.192.0/18 | | a91bb860-32d6-4961-a319-bf39a492ff92 | NVL-INTERPOD-VNI-PRIV | 85cace94dcc7484c85ff9337eb1d0c4c | 7b8987f7-da16-48c0-b86c-f3c4fd507f22 10.138.240.48/28 | | | | | a855f85b-46b7-4463-993b-8d867fb38556 10.138.240.0/28 | | b11bd55d-c57b-48d1-9be4-ab690624fb76 | NVLP2-Prov-WEB | 85cace94dcc7484c85ff9337eb1d0c4c | 5f369cf5-8b26-4996-97d5-24434f928a84 10.138.236.32/27 | | | | | adbe60f2-e02b-4294-8eb8-18ffb0b8bcda 10.138.168.0/21 | | | | | ae1386b4-c6bc-4c69-909a-1e902df3bc3f 10.138.232.0/27 | | | | | aefeeb08-7fb8-43c7-adc8-e8b0538b57d9 10.138.236.128/27 | | | | | b22247e8-fa4a-48bd-8a3f-dde5f22ee866 10.138.236.0/27 | | | | | d3e834ef-5f7c-43b7-b52d-fd8e223e6c50 10.138.232.128/27 | | | | | d87341b3-f953-4a93-bcaf-46524fbb3408 10.138.236.96/27 | | | | | de2c0bc9-85d4-4e5b-b1ae-d39ae6bbc6ee 10.138.200.0/21 | | | | | e0a4ce65-9d39-496d-960c-88272cafad55 10.138.236.64/27 | | | | | ee1bfd8b-231d-4e1d-83c8-d8d66346acef 10.138.232.32/27 | | | | | efd1f72c-8bd3-4b25-a3fc-e2698dd734d7 10.138.136.0/21 | | | | | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 10.138.232.64/27 | | | | | f986c4b1-1efc-4973-87a8-74db536ac1c0 10.138.232.96/27 | | b250fb4e-5d36-4aa6-bcfb-c623166312ca | net-vx1-private | 85cace94dcc7484c85ff9337eb1d0c4c | e0e82224-93e1-4369-b9c1-7d0ac1614837 192.168.213.0/24 | | c767ef08-c1fb-45ed-bf1e-a4c464e12b93 | private-net | 85cace94dcc7484c85ff9337eb1d0c4c | fb8ec92c-afa6-48f1-b7f8-25e77462b28b 192.192.100.0/24 | | cd8a928b-42ae-4673-ae19-c687cfd181d4 | NVLP2-Prov-BE | 85cace94dcc7484c85ff9337eb1d0c4c | 37b2ba42-6158-4a5f-96f0-48e476b4cd45 10.138.160.0/21 | | | | | 8502138b-c2c0-4436-bec5-9221a36e0c8c 10.138.192.0/21 | | | | | ceabf390-6489-4b5d-a5b9-305176def3a1 10.138.128.0/21 | | dede00bb-bc23-40a8-a3b9-b273eb7533ff | NVLP2-Prov-WEB2-test-2-network | e28e45dbbb3e4471a8fb6dd6cb99d01d | 92da59f1-df82-470b-a2ea-9b9b6f6c8e5e 10.138.185.112/28 | | | | | b8668626-8c05-4c7d-a70b-a1485f0d666e 10.138.217.112/28 | +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ [root at podto2-osctrl01 ~]# neutron subnet-list neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ | id | name | tenant_id | cidr | allocation_pools | +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ | 0705b470-ce99-46c8-944c-cfbfc6a2a5f7 | NVLP2-Provider-Bk-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.177.0/24 | {"start": "10.138.177.70", "end": "10.138.177.100"} | | 1a88e004-7062-4702-8965-b18ea9d6af87 | NVLP2-Ext-Rupar-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.181.0/24 | {"start": "10.138.181.20", "end": "10.138.181.150"} | | 272ca9b5-2aa0-45d2-90b4-70abe4feac49 | admin-self-subnet2 | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.90.0/24 | {"start": "192.168.90.10", "end": "192.168.90.254"} | | 37b2ba42-6158-4a5f-96f0-48e476b4cd45 | NVLP2-Prov-BE-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.160.0/21 | {"start": "10.138.160.20", "end": "10.138.164.255"} | | 5bbb31c3-a2cd-4e30-a194-60cee122d0e9 | HA subnet tenant 85cace94dcc7484c85ff9337eb1d0c4c | | 169.254.192.0/18 | {"start": "169.254.192.1", "end": "169.254.255.254"} | | 5f369cf5-8b26-4996-97d5-24434f928a84 | NVLP2-Prov-Rupar02-2-0cf8fe671a-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.32/27 | {"start": "10.138.236.35", "end": "10.138.236.47"} | | 7b8987f7-da16-48c0-b86c-f3c4fd507f22 | NVLP2-Interpod-subnet-x4 | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.240.48/28 | {"start": "10.138.240.61", "end": "10.138.240.61"} | | 8502138b-c2c0-4436-bec5-9221a36e0c8c | NVLP2-Prov-BE-subnet-78bc7d0889 | e8c2a22432174638bdee830274845c38 | 10.138.192.0/21 | {"start": "10.138.192.20", "end": "10.138.196.255"} | | 92da59f1-df82-470b-a2ea-9b9b6f6c8e5e | NVLP2-Prov-WEB2-test-2-284f32dbb8-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.185.112/28 | {"start": "10.138.185.114", "end": "10.138.185.119"} | | 9ba2a03d-3f95-494f-bd38-7fe85471465d | NVLP2-Provider-backoffice1-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.179.0/24 | {"start": "10.138.179.20", "end": "10.138.179.150"} | | a855f85b-46b7-4463-993b-8d867fb38556 | NVLP2-Interpod-subnet-x1 | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.240.0/28 | {"start": "10.138.240.13", "end": "10.138.240.14"} | | adbe60f2-e02b-4294-8eb8-18ffb0b8bcda | NVLP2-Prov-WEB-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.168.0/21 | {"start": "10.138.168.20", "end": "10.138.172.255"} | | ae1386b4-c6bc-4c69-909a-1e902df3bc3f | NVLP2-Prov-Rupar01-2-d0f1fa90f7-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.0/27 | {"start": "10.138.232.3", "end": "10.138.232.15"} | | aefeeb08-7fb8-43c7-adc8-e8b0538b57d9 | NVLP2-Prov-Rupar05-2-93db411f87-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.128/27 | {"start": "10.138.236.131", "end": "10.138.236.143"} | | b22247e8-fa4a-48bd-8a3f-dde5f22ee866 | NVLP2-Prov-Rupar01-2-6e53be6e00-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.0/27 | {"start": "10.138.236.3", "end": "10.138.236.15"} | | b8668626-8c05-4c7d-a70b-a1485f0d666e | NVLP2-Prov-WEB2-test-2-dbe7f3376f-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.217.112/28 | {"start": "10.138.217.114", "end": "10.138.217.119"} | | ceabf390-6489-4b5d-a5b9-305176def3a1 | NVLP2-Prov-BE-subnet-42bda158af | e8c2a22432174638bdee830274845c38 | 10.138.128.0/21 | {"start": "10.138.128.20", "end": "10.138.132.255"} | | d2129401-0349-4da3-87fa-5e521b071377 | HA subnet tenant 4e44aad117b64a34806c4785b31c3f88 | | 169.254.192.0/18 | {"start": "169.254.192.1", "end": "169.254.255.254"} | | d3e834ef-5f7c-43b7-b52d-fd8e223e6c50 | NVLP2-Prov-Rupar05-2-97e7a2473d-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.128/27 | {"start": "10.138.232.131", "end": "10.138.232.143"} | | d87341b3-f953-4a93-bcaf-46524fbb3408 | NVLP2-Prov-Rupar04-2-a8223d4e2e-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.96/27 | {"start": "10.138.236.99", "end": "10.138.236.111"} | | dcd2ad5c-afb5-420b-bfa8-7b89ab511f5b | sub-vx4 | 4e44aad117b64a34806c4785b31c3f88 | 192.168.213.0/24 | {"start": "192.168.213.100", "end": "192.168.213.200"} | | de2c0bc9-85d4-4e5b-b1ae-d39ae6bbc6ee | NVLP2-Prov-WEB-subnet-ae476243c2 | e8c2a22432174638bdee830274845c38 | 10.138.200.0/21 | {"start": "10.138.200.20", "end": "10.138.204.255"} | | e0a4ce65-9d39-496d-960c-88272cafad55 | NVLP2-Prov-Rupar03-2-3c7a3850d1-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.64/27 | {"start": "10.138.236.67", "end": "10.138.236.79"} | | e0e82224-93e1-4369-b9c1-7d0ac1614837 | sub-vx1 | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.213.0/24 | {"start": "192.168.213.100", "end": "192.168.213.200"} | | ea1c69b8-ded6-4028-bba9-63db12213e36 | admin-self-subnet1 | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.100.0/24 | {"start": "192.168.100.2", "end": "192.168.100.254"} | | ee1bfd8b-231d-4e1d-83c8-d8d66346acef | NVLP2-Prov-Rupar02-2-ecfee60675-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.32/27 | {"start": "10.138.232.35", "end": "10.138.232.47"} | | efd1f72c-8bd3-4b25-a3fc-e2698dd734d7 | NVLP2-Prov-WEB-subnet-e7e65a3ed6 | e8c2a22432174638bdee830274845c38 | 10.138.136.0/21 | {"start": "10.138.136.20", "end": "10.138.140.255"} | | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 | NVLP2-Prov-Rupar03-2-40976657fe-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.64/27 | {"start": "10.138.232.67", "end": "10.138.232.79"} | | f986c4b1-1efc-4973-87a8-74db536ac1c0 | NVLP2-Prov-Rupar04-2-729355c12e-subnet | e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.96/27 | {"start": "10.138.232.99", "end": "10.138.232.111"} | | fb8ec92c-afa6-48f1-b7f8-25e77462b28b | provate-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 192.192.100.0/24 | {"start": "192.192.100.2", "end": "192.192.100.254"} | | ff797a1f-7de4-4998-9bdc-29d1241ffb5b | NVLP2-Ext-Internet-subnet | 85cace94dcc7484c85ff9337eb1d0c4c | 84.240.191.0/24 | {"start": "84.240.191.2", "end": "84.240.191.170"} | +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ [root at podto2-osctrl01 ~]# neutron subnet-show NVLP2-Prov-Rupar04-2-729355c12e-subnet neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +-------------------+-----------------------------------------------------------------+ | Field | Value | +-------------------+-----------------------------------------------------------------+ | allocation_pools | {"start": "10.138.232.99", "end": "10.138.232.111"} | | cidr | 10.138.232.96/27 | | created_at | 2019-11-29T08:14:32Z | | description | | | dns_nameservers | 10.103.48.1 | | | 10.103.48.2 | | enable_dhcp | True | | gateway_ip | 10.138.232.97 | | host_routes | {"nexthop": "10.138.232.97", "destination": "0.0.0.0/0"} | | | {"nexthop": "10.138.232.97", "destination": " 10.138.236.96/27"} | | id | f986c4b1-1efc-4973-87a8-74db536ac1c0 | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | NVLP2-Prov-Rupar04-2-729355c12e-subnet | | network_id | b11bd55d-c57b-48d1-9be4-ab690624fb76 | | project_id | e28e45dbbb3e4471a8fb6dd6cb99d01d | | revision_number | 4 | | service_types | | | subnetpool_id | | | tags | | | tenant_id | e28e45dbbb3e4471a8fb6dd6cb99d01d | | updated_at | 2019-12-10T13:43:02Z | If I create an instance on this subnet , it gets an address in the allocation-pool correctly, but the default gateway is not 10.138.232.97, but 10.138.232.65. This is a gatway of another subnet in the same vlan: [root at podto2-osctrl01 ~]# neutron subnet-show NVLP2-Prov-Rupar03-2-40976657fe-subnet neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +-------------------+-----------------------------------------------------------------+ | Field | Value | +-------------------+-----------------------------------------------------------------+ | allocation_pools | {"start": "10.138.232.67", "end": "10.138.232.79"} | | cidr | 10.138.232.64/27 | | created_at | 2019-11-29T08:13:14Z | | description | | | dns_nameservers | 10.103.48.1 | | | 10.103.48.2 | | enable_dhcp | True | | gateway_ip | 10.138.232.65 | | host_routes | {"nexthop": "10.138.232.65", "destination": " 10.138.236.64/27"} | | id | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | NVLP2-Prov-Rupar03-2-40976657fe-subnet | | network_id | b11bd55d-c57b-48d1-9be4-ab690624fb76 | | project_id | e28e45dbbb3e4471a8fb6dd6cb99d01d | | revision_number | 4 | | service_types | | | subnetpool_id | | | tags | | | tenant_id | e28e45dbbb3e4471a8fb6dd6cb99d01d | | updated_at | 2019-12-11T08:27:20Z | +-------------------+-----------------------------------------------------------------+ The following is the dnsmasq process: [root at podto2-osctrl01 ~]# ps -afe|grep dns|grep b76 nobody 70093 1 0 dic02 ? 00:00:04 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/host --addn-hosts=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts --dhcp-leasefile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/leases --dhcp-match=set:ipxe,175 --local-service --bind-interfaces --dhcp-range=set:tag1,10.138.168.0,static,255.255.248.0,86400s --dhcp-range=set:tag2,10.138.232.0,static,255.255.255.224,86400s --dhcp-range=set:tag4,10.138.232.128,static,255.255.255.224,86400s --dhcp-range=set:tag8,10.138.232.32,static,255.255.255.224,86400s --dhcp-range=set:tag10,10.138.232.64,static,255.255.255.224,86400s --dhcp-range=set:tag11,10.138.232.96,static,255.255.255.224,86400s --dhcp-option-force=option:mtu,9000 --dhcp-lease-max=2208 --conf-file= --domain=site02.nivolapiemonte.it Ignazio Il giorno mer 11 dic 2019 alle ore 09:19 Slawek Kaplonski < skaplons at redhat.com> ha scritto: > Hi, > > Can You maybe give us some more details? Like how exactly this network and > subnets looks like (neutron net-show and neutron subnet-show), and please > also check dnsmasq config files - those are located on node where DHCP > agent is running. If You will do something like: > > ps aux | grep > > You will find dnsmasq process for this network. > > > On 10 Dec 2019, at 16:52, Ignazio Cassano > wrote: > > > > Hello Everyone, > > I have an openstack queens installation based on centos 7. > > I created a shared vlan network and multiple subnet under it. > > When I start an instance on one of the subnet network it gets the > correct ip based on its subnet but the gateway seems random (one of the > subnet of the vlan). > > > > Any suggestion, please ? > > > > Ignazio > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Wed Dec 11 09:35:16 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Wed, 11 Dec 2019 10:35:16 +0100 Subject: [neutron][queens] dhcp issue In-Reply-To: References: Message-ID: Hi, cat /var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts tag:tag1,option:dns-server,10.103.48.1,10.103.48.2 tag:tag1,option:classless-static-route, 10.138.136.0/21,10.138.168.1,10.138.200.0/21,10.138.168.1,169.254.169.254/32,10.138.168.22,10.138.236.32/27,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.168.1 tag:tag1,249, 10.138.136.0/21,10.138.168.1,10.138.200.0/21,10.138.168.1,169.254.169.254/32,10.138.168.22,10.138.236.32/27,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.168.1 tag:tag1,option:router,10.138.168.1 tag:tag2,option:dns-server,10.103.48.1,10.103.48.2 tag:tag2,option:classless-static-route, 10.138.236.0/27,10.138.232.1,169.254.169.254/32,10.138.232.4,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.1 tag:tag2,249, 10.138.236.0/27,10.138.232.1,169.254.169.254/32,10.138.232.4,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.1 tag:tag2,option:router,10.138.232.1 tag:tag5,option:dns-server,10.103.48.1,10.103.48.2 tag:tag5,option:classless-static-route, 10.138.236.128/27,10.138.232.129,169.254.169.254/32,10.138.232.131,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.129 tag:tag5,249, 10.138.236.128/27,10.138.232.129,169.254.169.254/32,10.138.232.131,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.129 tag:tag5,option:router,10.138.232.129 tag:tag9,option:dns-server,10.103.48.1,10.103.48.2 tag:tag9,option:classless-static-route, 10.138.236.32/27,10.138.232.33,169.254.169.254/32,10.138.232.36,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.33 tag:tag9,249, 10.138.236.32/27,10.138.232.33,169.254.169.254/32,10.138.232.36,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.33 tag:tag9,option:router,10.138.232.33 tag:tag11,option:dns-server,10.103.48.1,10.103.48.2 tag:tag11,option:classless-static-route, 10.138.236.64/27,10.138.232.65,169.254.169.254/32,10.138.232.67,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.65 tag:tag11,249, 10.138.236.64/27,10.138.232.65,169.254.169.254/32,10.138.232.67,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.65 tag:tag11,option:router,10.138.232.65 tag:tag12,option:dns-server,10.103.48.1,10.103.48.2 tag:tag12,option:classless-static-route, 10.138.236.96/27,10.138.232.97,169.254.169.254/32,10.138.232.101,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,0.0.0.0/0,10.138.232.97 tag:tag12,249, 10.138.236.96/27,10.138.232.97,169.254.169.254/32,10.138.232.101,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,0.0.0.0/0,10.138.232.97 Thanks Il giorno mer 11 dic 2019 alle ore 10:29 Slawek Kaplonski < skaplons at redhat.com> ha scritto: > Hi, > > Can You also send me content of the file > /var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts ? > > > On 11 Dec 2019, at 09:39, Ignazio Cassano > wrote: > > > > Thanks for your help: > > [root at podto2-osctrl01 ~]# neutron net-list > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ > > | id | name > | tenant_id | subnets > | > > > +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ > > | 00361d96-0ea2-4378-8909-dbf7ef4c883f | NVLP2-Provider-backoffice1 > | 85cace94dcc7484c85ff9337eb1d0c4c | > 9ba2a03d-3f95-494f-bd38-7fe85471465d 10.138.179.0/24 | > > | 28ac0a79-17a9-46f5-a418-77f386b821b5 | NVLP2-Provider-Bk > | 85cace94dcc7484c85ff9337eb1d0c4c | > 0705b470-ce99-46c8-944c-cfbfc6a2a5f7 10.138.177.0/24 | > > | 2e3edf38-7648-4ac8-a8d7-11448adedcca | NVLP2-Ext-Internet > | 85cace94dcc7484c85ff9337eb1d0c4c | > ff797a1f-7de4-4998-9bdc-29d1241ffb5b 84.240.191.0/24 | > > | 34a3e80d-a662-4665-ad6f-a7a83e2be346 | NVLP2-Ext-Rupar > | 85cace94dcc7484c85ff9337eb1d0c4c | > 1a88e004-7062-4702-8965-b18ea9d6af87 10.138.181.0/24 | > > | 435e5609-c91a-4e42-bd1a-87dbc24e4afa | HA network tenant > 85cace94dcc7484c85ff9337eb1d0c4c | | > 5bbb31c3-a2cd-4e30-a194-60cee122d0e9 169.254.192.0/18 | > > | 7f41aa95-b108-4224-9088-21e32d554646 | admin-self-net > | 85cace94dcc7484c85ff9337eb1d0c4c | > 272ca9b5-2aa0-45d2-90b4-70abe4feac49 192.168.90.0/24 | > > | | > | | > ea1c69b8-ded6-4028-bba9-63db12213e36 192.168.100.0/24 | > > | 8b876fb6-10a3-4e92-a499-f62a9d3aa32b | net-vx4-private > | 4e44aad117b64a34806c4785b31c3f88 | > dcd2ad5c-afb5-420b-bfa8-7b89ab511f5b 192.168.213.0/24 | > > | 993f14c9-1b40-47b7-8ee5-fcb00ef92d48 | HA network tenant > 4e44aad117b64a34806c4785b31c3f88 | | > d2129401-0349-4da3-87fa-5e521b071377 169.254.192.0/18 | > > | a91bb860-32d6-4961-a319-bf39a492ff92 | NVL-INTERPOD-VNI-PRIV > | 85cace94dcc7484c85ff9337eb1d0c4c | > 7b8987f7-da16-48c0-b86c-f3c4fd507f22 10.138.240.48/28 | > > | | > | | > a855f85b-46b7-4463-993b-8d867fb38556 10.138.240.0/28 | > > | b11bd55d-c57b-48d1-9be4-ab690624fb76 | NVLP2-Prov-WEB > | 85cace94dcc7484c85ff9337eb1d0c4c | > 5f369cf5-8b26-4996-97d5-24434f928a84 10.138.236.32/27 | > > | | > | | > adbe60f2-e02b-4294-8eb8-18ffb0b8bcda 10.138.168.0/21 | > > | | > | | > ae1386b4-c6bc-4c69-909a-1e902df3bc3f 10.138.232.0/27 | > > | | > | | > aefeeb08-7fb8-43c7-adc8-e8b0538b57d9 10.138.236.128/27 | > > | | > | | > b22247e8-fa4a-48bd-8a3f-dde5f22ee866 10.138.236.0/27 | > > | | > | | > d3e834ef-5f7c-43b7-b52d-fd8e223e6c50 10.138.232.128/27 | > > | | > | | > d87341b3-f953-4a93-bcaf-46524fbb3408 10.138.236.96/27 | > > | | > | | > de2c0bc9-85d4-4e5b-b1ae-d39ae6bbc6ee 10.138.200.0/21 | > > | | > | | > e0a4ce65-9d39-496d-960c-88272cafad55 10.138.236.64/27 | > > | | > | | > ee1bfd8b-231d-4e1d-83c8-d8d66346acef 10.138.232.32/27 | > > | | > | | > efd1f72c-8bd3-4b25-a3fc-e2698dd734d7 10.138.136.0/21 | > > | | > | | > f4b9f50b-18e9-49b2-b0db-6a51cf968e95 10.138.232.64/27 | > > | | > | | > f986c4b1-1efc-4973-87a8-74db536ac1c0 10.138.232.96/27 | > > | b250fb4e-5d36-4aa6-bcfb-c623166312ca | net-vx1-private > | 85cace94dcc7484c85ff9337eb1d0c4c | > e0e82224-93e1-4369-b9c1-7d0ac1614837 192.168.213.0/24 | > > | c767ef08-c1fb-45ed-bf1e-a4c464e12b93 | private-net > | 85cace94dcc7484c85ff9337eb1d0c4c | > fb8ec92c-afa6-48f1-b7f8-25e77462b28b 192.192.100.0/24 | > > | cd8a928b-42ae-4673-ae19-c687cfd181d4 | NVLP2-Prov-BE > | 85cace94dcc7484c85ff9337eb1d0c4c | > 37b2ba42-6158-4a5f-96f0-48e476b4cd45 10.138.160.0/21 | > > | | > | | > 8502138b-c2c0-4436-bec5-9221a36e0c8c 10.138.192.0/21 | > > | | > | | > ceabf390-6489-4b5d-a5b9-305176def3a1 10.138.128.0/21 | > > | dede00bb-bc23-40a8-a3b9-b273eb7533ff | NVLP2-Prov-WEB2-test-2-network > | e28e45dbbb3e4471a8fb6dd6cb99d01d | > 92da59f1-df82-470b-a2ea-9b9b6f6c8e5e 10.138.185.112/28 | > > | | > | | > b8668626-8c05-4c7d-a70b-a1485f0d666e 10.138.217.112/28 | > > > +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ > > > > [root at podto2-osctrl01 ~]# neutron subnet-list > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ > > | id | name > | tenant_id | cidr | > allocation_pools | > > > +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ > > | 0705b470-ce99-46c8-944c-cfbfc6a2a5f7 | NVLP2-Provider-Bk-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.177.0/24 > | {"start": "10.138.177.70", "end": "10.138.177.100"} | > > | 1a88e004-7062-4702-8965-b18ea9d6af87 | NVLP2-Ext-Rupar-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.181.0/24 > | {"start": "10.138.181.20", "end": "10.138.181.150"} | > > | 272ca9b5-2aa0-45d2-90b4-70abe4feac49 | admin-self-subnet2 > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.90.0/24 > | {"start": "192.168.90.10", "end": "192.168.90.254"} | > > | 37b2ba42-6158-4a5f-96f0-48e476b4cd45 | NVLP2-Prov-BE-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.160.0/21 > | {"start": "10.138.160.20", "end": "10.138.164.255"} | > > | 5bbb31c3-a2cd-4e30-a194-60cee122d0e9 | HA subnet tenant > 85cace94dcc7484c85ff9337eb1d0c4c | | > 169.254.192.0/18 | {"start": "169.254.192.1", "end": "169.254.255.254"} > | > > | 5f369cf5-8b26-4996-97d5-24434f928a84 | > NVLP2-Prov-Rupar02-2-0cf8fe671a-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.32/27 | {"start": > "10.138.236.35", "end": "10.138.236.47"} | > > | 7b8987f7-da16-48c0-b86c-f3c4fd507f22 | NVLP2-Interpod-subnet-x4 > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.240.48/28 > | {"start": "10.138.240.61", "end": "10.138.240.61"} | > > | 8502138b-c2c0-4436-bec5-9221a36e0c8c | > NVLP2-Prov-BE-subnet-78bc7d0889 | > e8c2a22432174638bdee830274845c38 | 10.138.192.0/21 | {"start": > "10.138.192.20", "end": "10.138.196.255"} | > > | 92da59f1-df82-470b-a2ea-9b9b6f6c8e5e | > NVLP2-Prov-WEB2-test-2-284f32dbb8-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.185.112/28 | {"start": > "10.138.185.114", "end": "10.138.185.119"} | > > | 9ba2a03d-3f95-494f-bd38-7fe85471465d | > NVLP2-Provider-backoffice1-subnet | > 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.179.0/24 | {"start": > "10.138.179.20", "end": "10.138.179.150"} | > > | a855f85b-46b7-4463-993b-8d867fb38556 | NVLP2-Interpod-subnet-x1 > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.240.0/28 > | {"start": "10.138.240.13", "end": "10.138.240.14"} | > > | adbe60f2-e02b-4294-8eb8-18ffb0b8bcda | NVLP2-Prov-WEB-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.168.0/21 > | {"start": "10.138.168.20", "end": "10.138.172.255"} | > > | ae1386b4-c6bc-4c69-909a-1e902df3bc3f | > NVLP2-Prov-Rupar01-2-d0f1fa90f7-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.0/27 | {"start": > "10.138.232.3", "end": "10.138.232.15"} | > > | aefeeb08-7fb8-43c7-adc8-e8b0538b57d9 | > NVLP2-Prov-Rupar05-2-93db411f87-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.128/27 | {"start": > "10.138.236.131", "end": "10.138.236.143"} | > > | b22247e8-fa4a-48bd-8a3f-dde5f22ee866 | > NVLP2-Prov-Rupar01-2-6e53be6e00-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.0/27 | {"start": > "10.138.236.3", "end": "10.138.236.15"} | > > | b8668626-8c05-4c7d-a70b-a1485f0d666e | > NVLP2-Prov-WEB2-test-2-dbe7f3376f-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.217.112/28 | {"start": > "10.138.217.114", "end": "10.138.217.119"} | > > | ceabf390-6489-4b5d-a5b9-305176def3a1 | > NVLP2-Prov-BE-subnet-42bda158af | > e8c2a22432174638bdee830274845c38 | 10.138.128.0/21 | {"start": > "10.138.128.20", "end": "10.138.132.255"} | > > | d2129401-0349-4da3-87fa-5e521b071377 | HA subnet tenant > 4e44aad117b64a34806c4785b31c3f88 | | > 169.254.192.0/18 | {"start": "169.254.192.1", "end": "169.254.255.254"} > | > > | d3e834ef-5f7c-43b7-b52d-fd8e223e6c50 | > NVLP2-Prov-Rupar05-2-97e7a2473d-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.128/27 | {"start": > "10.138.232.131", "end": "10.138.232.143"} | > > | d87341b3-f953-4a93-bcaf-46524fbb3408 | > NVLP2-Prov-Rupar04-2-a8223d4e2e-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.96/27 | {"start": > "10.138.236.99", "end": "10.138.236.111"} | > > | dcd2ad5c-afb5-420b-bfa8-7b89ab511f5b | sub-vx4 > | 4e44aad117b64a34806c4785b31c3f88 | 192.168.213.0/24 > | {"start": "192.168.213.100", "end": "192.168.213.200"} | > > | de2c0bc9-85d4-4e5b-b1ae-d39ae6bbc6ee | > NVLP2-Prov-WEB-subnet-ae476243c2 | > e8c2a22432174638bdee830274845c38 | 10.138.200.0/21 | {"start": > "10.138.200.20", "end": "10.138.204.255"} | > > | e0a4ce65-9d39-496d-960c-88272cafad55 | > NVLP2-Prov-Rupar03-2-3c7a3850d1-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.64/27 | {"start": > "10.138.236.67", "end": "10.138.236.79"} | > > | e0e82224-93e1-4369-b9c1-7d0ac1614837 | sub-vx1 > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.213.0/24 > | {"start": "192.168.213.100", "end": "192.168.213.200"} | > > | ea1c69b8-ded6-4028-bba9-63db12213e36 | admin-self-subnet1 > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.100.0/24 > | {"start": "192.168.100.2", "end": "192.168.100.254"} | > > | ee1bfd8b-231d-4e1d-83c8-d8d66346acef | > NVLP2-Prov-Rupar02-2-ecfee60675-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.32/27 | {"start": > "10.138.232.35", "end": "10.138.232.47"} | > > | efd1f72c-8bd3-4b25-a3fc-e2698dd734d7 | > NVLP2-Prov-WEB-subnet-e7e65a3ed6 | > e8c2a22432174638bdee830274845c38 | 10.138.136.0/21 | {"start": > "10.138.136.20", "end": "10.138.140.255"} | > > | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 | > NVLP2-Prov-Rupar03-2-40976657fe-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.64/27 | {"start": > "10.138.232.67", "end": "10.138.232.79"} | > > | f986c4b1-1efc-4973-87a8-74db536ac1c0 | > NVLP2-Prov-Rupar04-2-729355c12e-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.96/27 | {"start": > "10.138.232.99", "end": "10.138.232.111"} | > > | fb8ec92c-afa6-48f1-b7f8-25e77462b28b | provate-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.192.100.0/24 > | {"start": "192.192.100.2", "end": "192.192.100.254"} | > > | ff797a1f-7de4-4998-9bdc-29d1241ffb5b | NVLP2-Ext-Internet-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 84.240.191.0/24 > | {"start": "84.240.191.2", "end": "84.240.191.170"} | > > > +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ > > [root at podto2-osctrl01 ~]# neutron subnet-show > NVLP2-Prov-Rupar04-2-729355c12e-subnet > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > +-------------------+-----------------------------------------------------------------+ > > | Field | Value > | > > > +-------------------+-----------------------------------------------------------------+ > > | allocation_pools | {"start": "10.138.232.99", "end": > "10.138.232.111"} | > > | cidr | 10.138.232.96/27 > | > > | created_at | 2019-11-29T08:14:32Z > | > > | description | > | > > | dns_nameservers | 10.103.48.1 > | > > | | 10.103.48.2 > | > > | enable_dhcp | True > | > > | gateway_ip | 10.138.232.97 > | > > | host_routes | {"nexthop": "10.138.232.97", "destination": " > 0.0.0.0/0"} | > > | | {"nexthop": "10.138.232.97", "destination": " > 10.138.236.96/27"} | > > | id | f986c4b1-1efc-4973-87a8-74db536ac1c0 > | > > | ip_version | 4 > | > > | ipv6_address_mode | > | > > | ipv6_ra_mode | > | > > | name | NVLP2-Prov-Rupar04-2-729355c12e-subnet > | > > | network_id | b11bd55d-c57b-48d1-9be4-ab690624fb76 > | > > | project_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > | revision_number | 4 > | > > | service_types | > | > > | subnetpool_id | > | > > | tags | > | > > | tenant_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > | updated_at | 2019-12-10T13:43:02Z > | > > > > > > If I create an instance on this subnet , it gets an address in the > allocation-pool correctly, but the default gateway is not 10.138.232.97, > but 10.138.232.65. > > > > This is a gatway of another subnet in the same vlan: > > [root at podto2-osctrl01 ~]# neutron subnet-show > NVLP2-Prov-Rupar03-2-40976657fe-subnet > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > +-------------------+-----------------------------------------------------------------+ > > | Field | Value > | > > > +-------------------+-----------------------------------------------------------------+ > > | allocation_pools | {"start": "10.138.232.67", "end": > "10.138.232.79"} | > > | cidr | 10.138.232.64/27 > | > > | created_at | 2019-11-29T08:13:14Z > | > > | description | > | > > | dns_nameservers | 10.103.48.1 > | > > | | 10.103.48.2 > | > > | enable_dhcp | True > | > > | gateway_ip | 10.138.232.65 > | > > | host_routes | {"nexthop": "10.138.232.65", "destination": " > 10.138.236.64/27"} | > > | id | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 > | > > | ip_version | 4 > | > > | ipv6_address_mode | > | > > | ipv6_ra_mode | > | > > | name | NVLP2-Prov-Rupar03-2-40976657fe-subnet > | > > | network_id | b11bd55d-c57b-48d1-9be4-ab690624fb76 > | > > | project_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > | revision_number | 4 > | > > | service_types | > | > > | subnetpool_id | > | > > | tags | > | > > | tenant_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > | updated_at | 2019-12-11T08:27:20Z > | > > > +-------------------+-----------------------------------------------------------------+ > > > > The following is the dnsmasq process: > > > > [root at podto2-osctrl01 ~]# ps -afe|grep dns|grep b76 > > nobody 70093 1 0 dic02 ? 00:00:04 dnsmasq --no-hosts > --no-resolv > --pid-file=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/pid > --dhcp-hostsfile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/host > --addn-hosts=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/addn_hosts > --dhcp-optsfile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts > --dhcp-leasefile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/leases > --dhcp-match=set:ipxe,175 --local-service --bind-interfaces > --dhcp-range=set:tag1,10.138.168.0,static,255.255.248.0,86400s > --dhcp-range=set:tag2,10.138.232.0,static,255.255.255.224,86400s > --dhcp-range=set:tag4,10.138.232.128,static,255.255.255.224,86400s > --dhcp-range=set:tag8,10.138.232.32,static,255.255.255.224,86400s > --dhcp-range=set:tag10,10.138.232.64,static,255.255.255.224,86400s > --dhcp-range=set:tag11,10.138.232.96,static,255.255.255.224,86400s > --dhcp-option-force=option:mtu,9000 --dhcp-lease-max=2208 --conf-file= > --domain=site02.nivolapiemonte.it > > Ignazio > > > > > > Il giorno mer 11 dic 2019 alle ore 09:19 Slawek Kaplonski < > skaplons at redhat.com> ha scritto: > > Hi, > > > > Can You maybe give us some more details? Like how exactly this network > and subnets looks like (neutron net-show and neutron subnet-show), and > please also check dnsmasq config files - those are located on node where > DHCP agent is running. If You will do something like: > > > > ps aux | grep > > > > You will find dnsmasq process for this network. > > > > > On 10 Dec 2019, at 16:52, Ignazio Cassano > wrote: > > > > > > Hello Everyone, > > > I have an openstack queens installation based on centos 7. > > > I created a shared vlan network and multiple subnet under it. > > > When I start an instance on one of the subnet network it gets the > correct ip based on its subnet but the gateway seems random (one of the > subnet of the vlan). > > > > > > Any suggestion, please ? > > > > > > Ignazio > > > > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ignaziocassano at gmail.com Wed Dec 11 10:41:55 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Wed, 11 Dec 2019 11:41:55 +0100 Subject: [neutron][queens] dhcp issue In-Reply-To: References: Message-ID: It is our case: restarting solves. Many thanks Ignazio Il Mer 11 Dic 2019, 11:25 Slawek Kaplonski ha scritto: > Hi, > > I think You could hit bug https://bugs.launchpad.net/neutron/+bug/1848738 > Patch for partial fix for this issue is in master > https://review.opendev.org/#/c/689515/ and it was backported to Queens in > https://review.opendev.org/#/c/690731/ > > But You may also need to backport https://review.opendev.org/#/c/690700/ > to Queens branch to address this issue completly. > > > On 11 Dec 2019, at 10:35, Ignazio Cassano > wrote: > > > > Hi, cat /var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts > > tag:tag1,option:dns-server,10.103.48.1,10.103.48.2 > > tag:tag1,option:classless-static-route, > 10.138.136.0/21,10.138.168.1,10.138.200.0/21,10.138.168.1,169.254.169.254/32,10.138.168.22,10.138.236.32/27,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.168.1 > > tag:tag1,249, > 10.138.136.0/21,10.138.168.1,10.138.200.0/21,10.138.168.1,169.254.169.254/32,10.138.168.22,10.138.236.32/27,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.168.1 > > tag:tag1,option:router,10.138.168.1 > > tag:tag2,option:dns-server,10.103.48.1,10.103.48.2 > > tag:tag2,option:classless-static-route, > 10.138.236.0/27,10.138.232.1,169.254.169.254/32,10.138.232.4,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.1 > > tag:tag2,249, > 10.138.236.0/27,10.138.232.1,169.254.169.254/32,10.138.232.4,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.1 > > tag:tag2,option:router,10.138.232.1 > > tag:tag5,option:dns-server,10.103.48.1,10.103.48.2 > > tag:tag5,option:classless-static-route, > 10.138.236.128/27,10.138.232.129,169.254.169.254/32,10.138.232.131,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.129 > > tag:tag5,249, > 10.138.236.128/27,10.138.232.129,169.254.169.254/32,10.138.232.131,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.129 > > tag:tag5,option:router,10.138.232.129 > > tag:tag9,option:dns-server,10.103.48.1,10.103.48.2 > > tag:tag9,option:classless-static-route, > 10.138.236.32/27,10.138.232.33,169.254.169.254/32,10.138.232.36,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.33 > > tag:tag9,249, > 10.138.236.32/27,10.138.232.33,169.254.169.254/32,10.138.232.36,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.33 > > tag:tag9,option:router,10.138.232.33 > > tag:tag11,option:dns-server,10.103.48.1,10.103.48.2 > > tag:tag11,option:classless-static-route, > 10.138.236.64/27,10.138.232.65,169.254.169.254/32,10.138.232.67,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.65 > > tag:tag11,249, > 10.138.236.64/27,10.138.232.65,169.254.169.254/32,10.138.232.67,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.96/27,0.0.0.0,0.0.0.0/0,10.138.232.65 > > tag:tag11,option:router,10.138.232.65 > > tag:tag12,option:dns-server,10.103.48.1,10.103.48.2 > > tag:tag12,option:classless-static-route, > 10.138.236.96/27,10.138.232.97,169.254.169.254/32,10.138.232.101,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,0.0.0.0/0,10.138.232.97 > > tag:tag12,249, > 10.138.236.96/27,10.138.232.97,169.254.169.254/32,10.138.232.101,10.138.236.32/27,0.0.0.0,10.138.168.0/21,0.0.0.0,10.138.232.0/27,0.0.0.0,10.138.236.128/27,0.0.0.0,10.138.236.0/27,0.0.0.0,10.138.232.128/27,0.0.0.0,10.138.236.96/27,0.0.0.0,10.138.200.0/21,0.0.0.0,10.138.236.64/27,0.0.0.0,10.138.232.32/27,0.0.0.0,10.138.136.0/21,0.0.0.0,10.138.232.64/27,0.0.0.0,0.0.0.0/0,10.138.232.97 > > > > > > > > > > Thanks > > > > Il giorno mer 11 dic 2019 alle ore 10:29 Slawek Kaplonski < > skaplons at redhat.com> ha scritto: > > Hi, > > > > Can You also send me content of the file > /var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts ? > > > > > On 11 Dec 2019, at 09:39, Ignazio Cassano > wrote: > > > > > > Thanks for your help: > > > [root at podto2-osctrl01 ~]# neutron net-list > > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > > +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ > > > | id | name > | tenant_id | subnets > | > > > > +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ > > > | 00361d96-0ea2-4378-8909-dbf7ef4c883f | NVLP2-Provider-backoffice1 > | 85cace94dcc7484c85ff9337eb1d0c4c | > 9ba2a03d-3f95-494f-bd38-7fe85471465d 10.138.179.0/24 | > > > | 28ac0a79-17a9-46f5-a418-77f386b821b5 | NVLP2-Provider-Bk > | 85cace94dcc7484c85ff9337eb1d0c4c | > 0705b470-ce99-46c8-944c-cfbfc6a2a5f7 10.138.177.0/24 | > > > | 2e3edf38-7648-4ac8-a8d7-11448adedcca | NVLP2-Ext-Internet > | 85cace94dcc7484c85ff9337eb1d0c4c | > ff797a1f-7de4-4998-9bdc-29d1241ffb5b 84.240.191.0/24 | > > > | 34a3e80d-a662-4665-ad6f-a7a83e2be346 | NVLP2-Ext-Rupar > | 85cace94dcc7484c85ff9337eb1d0c4c | > 1a88e004-7062-4702-8965-b18ea9d6af87 10.138.181.0/24 | > > > | 435e5609-c91a-4e42-bd1a-87dbc24e4afa | HA network tenant > 85cace94dcc7484c85ff9337eb1d0c4c | | > 5bbb31c3-a2cd-4e30-a194-60cee122d0e9 169.254.192.0/18 | > > > | 7f41aa95-b108-4224-9088-21e32d554646 | admin-self-net > | 85cace94dcc7484c85ff9337eb1d0c4c | > 272ca9b5-2aa0-45d2-90b4-70abe4feac49 192.168.90.0/24 | > > > | | > | | > ea1c69b8-ded6-4028-bba9-63db12213e36 192.168.100.0/24 | > > > | 8b876fb6-10a3-4e92-a499-f62a9d3aa32b | net-vx4-private > | 4e44aad117b64a34806c4785b31c3f88 | > dcd2ad5c-afb5-420b-bfa8-7b89ab511f5b 192.168.213.0/24 | > > > | 993f14c9-1b40-47b7-8ee5-fcb00ef92d48 | HA network tenant > 4e44aad117b64a34806c4785b31c3f88 | | > d2129401-0349-4da3-87fa-5e521b071377 169.254.192.0/18 | > > > | a91bb860-32d6-4961-a319-bf39a492ff92 | NVL-INTERPOD-VNI-PRIV > | 85cace94dcc7484c85ff9337eb1d0c4c | > 7b8987f7-da16-48c0-b86c-f3c4fd507f22 10.138.240.48/28 | > > > | | > | | > a855f85b-46b7-4463-993b-8d867fb38556 10.138.240.0/28 | > > > | b11bd55d-c57b-48d1-9be4-ab690624fb76 | NVLP2-Prov-WEB > | 85cace94dcc7484c85ff9337eb1d0c4c | > 5f369cf5-8b26-4996-97d5-24434f928a84 10.138.236.32/27 | > > > | | > | | > adbe60f2-e02b-4294-8eb8-18ffb0b8bcda 10.138.168.0/21 | > > > | | > | | > ae1386b4-c6bc-4c69-909a-1e902df3bc3f 10.138.232.0/27 | > > > | | > | | > aefeeb08-7fb8-43c7-adc8-e8b0538b57d9 10.138.236.128/27 | > > > | | > | | > b22247e8-fa4a-48bd-8a3f-dde5f22ee866 10.138.236.0/27 | > > > | | > | | > d3e834ef-5f7c-43b7-b52d-fd8e223e6c50 10.138.232.128/27 | > > > | | > | | > d87341b3-f953-4a93-bcaf-46524fbb3408 10.138.236.96/27 | > > > | | > | | > de2c0bc9-85d4-4e5b-b1ae-d39ae6bbc6ee 10.138.200.0/21 | > > > | | > | | > e0a4ce65-9d39-496d-960c-88272cafad55 10.138.236.64/27 | > > > | | > | | > ee1bfd8b-231d-4e1d-83c8-d8d66346acef 10.138.232.32/27 | > > > | | > | | > efd1f72c-8bd3-4b25-a3fc-e2698dd734d7 10.138.136.0/21 | > > > | | > | | > f4b9f50b-18e9-49b2-b0db-6a51cf968e95 10.138.232.64/27 | > > > | | > | | > f986c4b1-1efc-4973-87a8-74db536ac1c0 10.138.232.96/27 | > > > | b250fb4e-5d36-4aa6-bcfb-c623166312ca | net-vx1-private > | 85cace94dcc7484c85ff9337eb1d0c4c | > e0e82224-93e1-4369-b9c1-7d0ac1614837 192.168.213.0/24 | > > > | c767ef08-c1fb-45ed-bf1e-a4c464e12b93 | private-net > | 85cace94dcc7484c85ff9337eb1d0c4c | > fb8ec92c-afa6-48f1-b7f8-25e77462b28b 192.192.100.0/24 | > > > | cd8a928b-42ae-4673-ae19-c687cfd181d4 | NVLP2-Prov-BE > | 85cace94dcc7484c85ff9337eb1d0c4c | > 37b2ba42-6158-4a5f-96f0-48e476b4cd45 10.138.160.0/21 | > > > | | > | | > 8502138b-c2c0-4436-bec5-9221a36e0c8c 10.138.192.0/21 | > > > | | > | | > ceabf390-6489-4b5d-a5b9-305176def3a1 10.138.128.0/21 | > > > | dede00bb-bc23-40a8-a3b9-b273eb7533ff | > NVLP2-Prov-WEB2-test-2-network | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 92da59f1-df82-470b-a2ea-9b9b6f6c8e5e > 10.138.185.112/28 | > > > | | > | | > b8668626-8c05-4c7d-a70b-a1485f0d666e 10.138.217.112/28 | > > > > +--------------------------------------+----------------------------------------------------+----------------------------------+--------------------------------------------------------+ > > > > > > [root at podto2-osctrl01 ~]# neutron subnet-list > > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > > +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ > > > | id | name > | tenant_id | cidr > | allocation_pools | > > > > +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ > > > | 0705b470-ce99-46c8-944c-cfbfc6a2a5f7 | NVLP2-Provider-Bk-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.177.0/24 > | {"start": "10.138.177.70", "end": "10.138.177.100"} | > > > | 1a88e004-7062-4702-8965-b18ea9d6af87 | NVLP2-Ext-Rupar-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.181.0/24 > | {"start": "10.138.181.20", "end": "10.138.181.150"} | > > > | 272ca9b5-2aa0-45d2-90b4-70abe4feac49 | admin-self-subnet2 > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.90.0/24 > | {"start": "192.168.90.10", "end": "192.168.90.254"} | > > > | 37b2ba42-6158-4a5f-96f0-48e476b4cd45 | NVLP2-Prov-BE-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.160.0/21 > | {"start": "10.138.160.20", "end": "10.138.164.255"} | > > > | 5bbb31c3-a2cd-4e30-a194-60cee122d0e9 | HA subnet tenant > 85cace94dcc7484c85ff9337eb1d0c4c | | > 169.254.192.0/18 | {"start": "169.254.192.1", "end": "169.254.255.254"} > | > > > | 5f369cf5-8b26-4996-97d5-24434f928a84 | > NVLP2-Prov-Rupar02-2-0cf8fe671a-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.32/27 | {"start": > "10.138.236.35", "end": "10.138.236.47"} | > > > | 7b8987f7-da16-48c0-b86c-f3c4fd507f22 | NVLP2-Interpod-subnet-x4 > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.240.48/28 > | {"start": "10.138.240.61", "end": "10.138.240.61"} | > > > | 8502138b-c2c0-4436-bec5-9221a36e0c8c | > NVLP2-Prov-BE-subnet-78bc7d0889 | > e8c2a22432174638bdee830274845c38 | 10.138.192.0/21 | {"start": > "10.138.192.20", "end": "10.138.196.255"} | > > > | 92da59f1-df82-470b-a2ea-9b9b6f6c8e5e | > NVLP2-Prov-WEB2-test-2-284f32dbb8-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.185.112/28 | {"start": > "10.138.185.114", "end": "10.138.185.119"} | > > > | 9ba2a03d-3f95-494f-bd38-7fe85471465d | > NVLP2-Provider-backoffice1-subnet | > 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.179.0/24 | {"start": > "10.138.179.20", "end": "10.138.179.150"} | > > > | a855f85b-46b7-4463-993b-8d867fb38556 | NVLP2-Interpod-subnet-x1 > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.240.0/28 > | {"start": "10.138.240.13", "end": "10.138.240.14"} | > > > | adbe60f2-e02b-4294-8eb8-18ffb0b8bcda | NVLP2-Prov-WEB-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 10.138.168.0/21 > | {"start": "10.138.168.20", "end": "10.138.172.255"} | > > > | ae1386b4-c6bc-4c69-909a-1e902df3bc3f | > NVLP2-Prov-Rupar01-2-d0f1fa90f7-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.0/27 | {"start": > "10.138.232.3", "end": "10.138.232.15"} | > > > | aefeeb08-7fb8-43c7-adc8-e8b0538b57d9 | > NVLP2-Prov-Rupar05-2-93db411f87-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.128/27 | {"start": > "10.138.236.131", "end": "10.138.236.143"} | > > > | b22247e8-fa4a-48bd-8a3f-dde5f22ee866 | > NVLP2-Prov-Rupar01-2-6e53be6e00-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.0/27 | {"start": > "10.138.236.3", "end": "10.138.236.15"} | > > > | b8668626-8c05-4c7d-a70b-a1485f0d666e | > NVLP2-Prov-WEB2-test-2-dbe7f3376f-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.217.112/28 | {"start": > "10.138.217.114", "end": "10.138.217.119"} | > > > | ceabf390-6489-4b5d-a5b9-305176def3a1 | > NVLP2-Prov-BE-subnet-42bda158af | > e8c2a22432174638bdee830274845c38 | 10.138.128.0/21 | {"start": > "10.138.128.20", "end": "10.138.132.255"} | > > > | d2129401-0349-4da3-87fa-5e521b071377 | HA subnet tenant > 4e44aad117b64a34806c4785b31c3f88 | | > 169.254.192.0/18 | {"start": "169.254.192.1", "end": "169.254.255.254"} > | > > > | d3e834ef-5f7c-43b7-b52d-fd8e223e6c50 | > NVLP2-Prov-Rupar05-2-97e7a2473d-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.128/27 | {"start": > "10.138.232.131", "end": "10.138.232.143"} | > > > | d87341b3-f953-4a93-bcaf-46524fbb3408 | > NVLP2-Prov-Rupar04-2-a8223d4e2e-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.96/27 | {"start": > "10.138.236.99", "end": "10.138.236.111"} | > > > | dcd2ad5c-afb5-420b-bfa8-7b89ab511f5b | sub-vx4 > | 4e44aad117b64a34806c4785b31c3f88 | 192.168.213.0/24 > | {"start": "192.168.213.100", "end": "192.168.213.200"} | > > > | de2c0bc9-85d4-4e5b-b1ae-d39ae6bbc6ee | > NVLP2-Prov-WEB-subnet-ae476243c2 | > e8c2a22432174638bdee830274845c38 | 10.138.200.0/21 | {"start": > "10.138.200.20", "end": "10.138.204.255"} | > > > | e0a4ce65-9d39-496d-960c-88272cafad55 | > NVLP2-Prov-Rupar03-2-3c7a3850d1-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.236.64/27 | {"start": > "10.138.236.67", "end": "10.138.236.79"} | > > > | e0e82224-93e1-4369-b9c1-7d0ac1614837 | sub-vx1 > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.213.0/24 > | {"start": "192.168.213.100", "end": "192.168.213.200"} | > > > | ea1c69b8-ded6-4028-bba9-63db12213e36 | admin-self-subnet1 > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.168.100.0/24 > | {"start": "192.168.100.2", "end": "192.168.100.254"} | > > > | ee1bfd8b-231d-4e1d-83c8-d8d66346acef | > NVLP2-Prov-Rupar02-2-ecfee60675-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.32/27 | {"start": > "10.138.232.35", "end": "10.138.232.47"} | > > > | efd1f72c-8bd3-4b25-a3fc-e2698dd734d7 | > NVLP2-Prov-WEB-subnet-e7e65a3ed6 | > e8c2a22432174638bdee830274845c38 | 10.138.136.0/21 | {"start": > "10.138.136.20", "end": "10.138.140.255"} | > > > | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 | > NVLP2-Prov-Rupar03-2-40976657fe-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.64/27 | {"start": > "10.138.232.67", "end": "10.138.232.79"} | > > > | f986c4b1-1efc-4973-87a8-74db536ac1c0 | > NVLP2-Prov-Rupar04-2-729355c12e-subnet | > e28e45dbbb3e4471a8fb6dd6cb99d01d | 10.138.232.96/27 | {"start": > "10.138.232.99", "end": "10.138.232.111"} | > > > | fb8ec92c-afa6-48f1-b7f8-25e77462b28b | provate-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 192.192.100.0/24 > | {"start": "192.192.100.2", "end": "192.192.100.254"} | > > > | ff797a1f-7de4-4998-9bdc-29d1241ffb5b | NVLP2-Ext-Internet-subnet > | 85cace94dcc7484c85ff9337eb1d0c4c | 84.240.191.0/24 > | {"start": "84.240.191.2", "end": "84.240.191.170"} | > > > > +--------------------------------------+---------------------------------------------------+----------------------------------+-------------------+--------------------------------------------------------+ > > > [root at podto2-osctrl01 ~]# neutron subnet-show > NVLP2-Prov-Rupar04-2-729355c12e-subnet > > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > > +-------------------+-----------------------------------------------------------------+ > > > | Field | Value > | > > > > +-------------------+-----------------------------------------------------------------+ > > > | allocation_pools | {"start": "10.138.232.99", "end": > "10.138.232.111"} | > > > | cidr | 10.138.232.96/27 > | > > > | created_at | 2019-11-29T08:14:32Z > | > > > | description | > | > > > | dns_nameservers | 10.103.48.1 > | > > > | | 10.103.48.2 > | > > > | enable_dhcp | True > | > > > | gateway_ip | 10.138.232.97 > | > > > | host_routes | {"nexthop": "10.138.232.97", "destination": " > 0.0.0.0/0"} | > > > | | {"nexthop": "10.138.232.97", "destination": " > 10.138.236.96/27"} | > > > | id | f986c4b1-1efc-4973-87a8-74db536ac1c0 > | > > > | ip_version | 4 > | > > > | ipv6_address_mode | > | > > > | ipv6_ra_mode | > | > > > | name | NVLP2-Prov-Rupar04-2-729355c12e-subnet > | > > > | network_id | b11bd55d-c57b-48d1-9be4-ab690624fb76 > | > > > | project_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > > | revision_number | 4 > | > > > | service_types | > | > > > | subnetpool_id | > | > > > | tags | > | > > > | tenant_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > > | updated_at | 2019-12-10T13:43:02Z > | > > > > > > > > > If I create an instance on this subnet , it gets an address in the > allocation-pool correctly, but the default gateway is not 10.138.232.97, > but 10.138.232.65. > > > > > > This is a gatway of another subnet in the same vlan: > > > [root at podto2-osctrl01 ~]# neutron subnet-show > NVLP2-Prov-Rupar03-2-40976657fe-subnet > > > neutron CLI is deprecated and will be removed in the future. Use > openstack CLI instead. > > > > +-------------------+-----------------------------------------------------------------+ > > > | Field | Value > | > > > > +-------------------+-----------------------------------------------------------------+ > > > | allocation_pools | {"start": "10.138.232.67", "end": > "10.138.232.79"} | > > > | cidr | 10.138.232.64/27 > | > > > | created_at | 2019-11-29T08:13:14Z > | > > > | description | > | > > > | dns_nameservers | 10.103.48.1 > | > > > | | 10.103.48.2 > | > > > | enable_dhcp | True > | > > > | gateway_ip | 10.138.232.65 > | > > > | host_routes | {"nexthop": "10.138.232.65", "destination": " > 10.138.236.64/27"} | > > > | id | f4b9f50b-18e9-49b2-b0db-6a51cf968e95 > | > > > | ip_version | 4 > | > > > | ipv6_address_mode | > | > > > | ipv6_ra_mode | > | > > > | name | NVLP2-Prov-Rupar03-2-40976657fe-subnet > | > > > | network_id | b11bd55d-c57b-48d1-9be4-ab690624fb76 > | > > > | project_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > > | revision_number | 4 > | > > > | service_types | > | > > > | subnetpool_id | > | > > > | tags | > | > > > | tenant_id | e28e45dbbb3e4471a8fb6dd6cb99d01d > | > > > | updated_at | 2019-12-11T08:27:20Z > | > > > > +-------------------+-----------------------------------------------------------------+ > > > > > > The following is the dnsmasq process: > > > > > > [root at podto2-osctrl01 ~]# ps -afe|grep dns|grep b76 > > > nobody 70093 1 0 dic02 ? 00:00:04 dnsmasq --no-hosts > --no-resolv > --pid-file=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/pid > --dhcp-hostsfile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/host > --addn-hosts=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/addn_hosts > --dhcp-optsfile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/opts > --dhcp-leasefile=/var/lib/neutron/dhcp/b11bd55d-c57b-48d1-9be4-ab690624fb76/leases > --dhcp-match=set:ipxe,175 --local-service --bind-interfaces > --dhcp-range=set:tag1,10.138.168.0,static,255.255.248.0,86400s > --dhcp-range=set:tag2,10.138.232.0,static,255.255.255.224,86400s > --dhcp-range=set:tag4,10.138.232.128,static,255.255.255.224,86400s > --dhcp-range=set:tag8,10.138.232.32,static,255.255.255.224,86400s > --dhcp-range=set:tag10,10.138.232.64,static,255.255.255.224,86400s > --dhcp-range=set:tag11,10.138.232.96,static,255.255.255.224,86400s > --dhcp-option-force=option:mtu,9000 --dhcp-lease-max=2208 --conf-file= > --domain=site02.nivolapiemonte.it > > > Ignazio > > > > > > > > > Il giorno mer 11 dic 2019 alle ore 09:19 Slawek Kaplonski < > skaplons at redhat.com> ha scritto: > > > Hi, > > > > > > Can You maybe give us some more details? Like how exactly this network > and subnets looks like (neutron net-show and neutron subnet-show), and > please also check dnsmasq config files - those are located on node where > DHCP agent is running. If You will do something like: > > > > > > ps aux | grep > > > > > > You will find dnsmasq process for this network. > > > > > > > On 10 Dec 2019, at 16:52, Ignazio Cassano > wrote: > > > > > > > > Hello Everyone, > > > > I have an openstack queens installation based on centos 7. > > > > I created a shared vlan network and multiple subnet under it. > > > > When I start an instance on one of the subnet network it gets the > correct ip based on its subnet but the gateway seems random (one of the > subnet of the vlan). > > > > > > > > Any suggestion, please ? > > > > > > > > Ignazio > > > > > > > > > > — > > > Slawek Kaplonski > > > Senior software engineer > > > Red Hat > > > > > > > — > > Slawek Kaplonski > > Senior software engineer > > Red Hat > > > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gagehugo at gmail.com Wed Dec 11 16:32:14 2019 From: gagehugo at gmail.com (Gage Hugo) Date: Wed, 11 Dec 2019 10:32:14 -0600 Subject: [OSSA-2019-006] Credentials API allows listing and retrieving of all users credentials (CVE-2019-19687) Message-ID: ===================================================================================== OSSA-2019-006: Credentials API allows listing and retrieving of all users credentials ===================================================================================== :Date: December 09, 2019 :CVE: CVE-2019-19687 Affects ~~~~~~~ - Keystone: ==15.0.0, ==16.0.0 Description ~~~~~~~~~~~ Daniel Preussker reported a vulnerability in Keystone's list credentials API. Any user with a role on a project is able to list any credentials with the /v3/credentials API when [oslo_policy] enforce_scope is false. Users with a role on a project are able to view any other users credentials, which could leak sign-on information for Time-based One Time Passwords (TOTP) or othewise. Deployments running keystone with [oslo_policy] enforce_scope set to false are affected. There will be a slight performance impact for the list credentials API once this issue is fixed. Patches ~~~~~~~ - https://review.opendev.org/697731 (Stein) - https://review.opendev.org/697611 (Train) - https://review.opendev.org/697355 (Ussuri) Credits ~~~~~~~ - Daniel Preussker (CVE-2019-19687) References ~~~~~~~~~~ - https://bugs.launchpad.net/keystone/+bug/1855080 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-19687 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEWa125cLHIuv6ekof56j9K3b+vREFAl3xGeUACgkQ56j9K3b+ vRGIFw/8CNnL2aDqOi+7AcCH5alknVoWJS5Dd/EymQVDI07vbe4yneghwcQwEs6S BUR25+xB5ukjPJdxsILTrm4UbhEMPUzjgT7PQN71Q1/ge8XY1YIRzBwMnjYm0nQr sYRBl8L1OQ3CQsAtnH+hB/qMCkxlQeXtOMiBcz3/1B2qcH/AruXtJxhVD9swJHew fOn5WNdODlDPKCmls0xK6csZX4RBcsXc04aPr3fSuOIzcwNZ260sr+8MfeTr3mi3 SO4t2uYQgQmhgaPnvM3tyMFa+ry+mFvq6MFigPUhp+Akd0eYasHa3rg6/I/LQO0Z ImWCSAnjntWigbyhQhLrxIIEdoxu/vgTVCciPWtgwMfxDOLw1jhj3kBzdeb3S5pA yoyY1+H8nXaze2UQOfbM1FHyO2cTkOjyzVdbrF5bg8vNl4haL5egu6UVD018H5pc voJwcrxgJUobO+kTzuOpjWzlHeqRryLXDhH191cVKoUBIgIpi0KEbyBOIAwMNciU vkvNqRDi9rGfaM+QWUWgf12G2FRnkqmFqbnlLY6hpz5uinueEVn7iTC2LnfBA+Um GX0qsWv4L3dQXf9rtTdCNpGvw7nsBGcqYqUw9C7piAVKfGDZ6LBiuQbseW+ONLCK 7vq7rLyJq9JCjwIWr53d8lLp0hLTrV/bH5hcICusEY3WfH+cJZs= =pUkl -----END PGP SIGNATURE----- From openstack at nemebean.com Wed Dec 11 16:35:19 2019 From: openstack at nemebean.com (Ben Nemec) Date: Wed, 11 Dec 2019 10:35:19 -0600 Subject: [oslo] Schedule for the rest of 2019 In-Reply-To: References: Message-ID: On 12/11/19 10:12 AM, Eric Fried wrote: >> So TLDR: If you need to get ahold of me, do it in the morning. > > What's "morning" to you (TZ)? Shoot, meant to include that. I'm central time U.S. From ignaziocassano at gmail.com Wed Dec 11 17:36:16 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Wed, 11 Dec 2019 18:36:16 +0100 Subject: [neutron][queens] dhcp issue In-Reply-To: <20191211152331.xmre63uzald5t4tg@yuggoth.org> References: <20191211152331.xmre63uzald5t4tg@yuggoth.org> Message-ID: Hello Jeremy, thanks for your help. Indeed when more subnet of a provider vlan are created, instance on one of the subnet receives the static routes of all subnets. Is it correct? Ignazio Il Mer 11 Dic 2019, 16:29 Jeremy Stanley ha scritto: > On 2019-12-11 10:10:17 +0100 (+0100), Ignazio Cassano wrote: > [...] > > Dec 11 09:49:06 podto2-osctrl01 dnsmasq-dhcp[70093]: cannot send > > DHCP/BOOTP option 121: no space left in packet > [...] > > This error is fairly straightforward. DHCP replies are limited to > what you can fit in a single UDP packet and how large of a datagram > the client agrees to accept. The number of routes you're trying to > include in via option 121 could be the problem (is it a bunch?) but > also the other fields and their lengths can also contribute, as can > the MTU on that network segment: > > https://tools.ietf.org/html/rfc2131 > > According to IETF RFC 2131 you should be able to assume clients will > accept a datagram of up to 576 octets in length (which implies your > entire options field can use at least 312 octets). Anything over > this requires negotiation of a maximum DHCP message size with the > client. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Wed Dec 11 18:38:10 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 11 Dec 2019 18:38:10 +0000 Subject: [neutron][queens] dhcp issue In-Reply-To: References: <20191211152331.xmre63uzald5t4tg@yuggoth.org> Message-ID: <20191211183809.pnm4f33me2lpom6q@yuggoth.org> On 2019-12-11 18:36:16 +0100 (+0100), Ignazio Cassano wrote: > Indeed when more subnet of a provider vlan are created, instance > on one of the subnet receives the static routes of all subnets. Is > it correct? [...] I'm afraid my expertise in this matter is limited to directly managing DHCP servers, I'm not personally familiar with running Neutron. Hopefully someone with more architectural background on Neutron itself can answer that part of your question. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From witold.bedyk at suse.com Wed Dec 11 19:23:06 2019 From: witold.bedyk at suse.com (Witek Bedyk) Date: Wed, 11 Dec 2019 20:23:06 +0100 Subject: [monasca] QUERY ON MONASCA In-Reply-To: References: Message-ID: <2abdbf84-257c-c185-341b-0b285aabc8b9@suse.com> Hi Muhammad, try this out: https://github.com/martinchacon/monasca-bootcamp/blob/master/MonascaBootcamp.ipynb You can deploy Monasca with Docker Compose as described or just use DevStack. Cheers Witek On 12/11/19 5:41 AM, Muhammad Hasan wrote: > Hi, > Can you share any hands on exercise in monasca api and monasca agent From juliaashleykreger at gmail.com Wed Dec 11 19:32:05 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 11 Dec 2019 11:32:05 -0800 Subject: [ironic] Last meeting of 2019 will be December 16th. Message-ID: Greetings everyone, During our weekly meeting this week[1], we had a consensus that we would hold the December 16th meeting as our last for the year as we anticipate a number of contributors will be taking time off. This doesn't prevent anyone from jumping in IRC and saying Good Morning! During this time of year, IRC does tend to become very quiet as people take time off from work. Please feel free to update the whiteboard as necessary[2]. If there are any issues that arise that require immediate attention by the community, please feel free to contact me on IRC. Thanks everyone! -Julia [1]: http://eavesdrop.openstack.org/meetings/ironic/2019/ironic.2019-12-09-15.00.log.txt [2]: https://etherpad.openstack.org/p/IronicWhiteBoard From juliaashleykreger at gmail.com Wed Dec 11 19:36:59 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 11 Dec 2019 11:36:59 -0800 Subject: a question about ironic booting from volume In-Reply-To: References: Message-ID: Greetings Guo, The intended design is that by some means, the baremetal node is able to reach the endpoints offered by the storage node. As such, custom security groups, firewall ACLs, and routing would be required to permit iSCSI connectivity to the storage node(s). Since these are deployment specific settings, you would have to put those settings in place to match your environment. -Julia On Mon, Dec 9, 2019 at 12:46 PM 海风吹 <1306890576 at qq.com> wrote: > > hi: > The pike version of ironic support the feature: booting from volume. There is a question: when booting from volume, the baremetal node should be reachable to the storage node,but when deploy done, the baremetal's network completely belong to the vpc. How could the baremetal node booting from volume?? > I should add the extar storage nic to communicate with storage node?? > > > expecting for the answer > > > thanks > guo > > From ignaziocassano at gmail.com Wed Dec 11 19:53:10 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Wed, 11 Dec 2019 20:53:10 +0100 Subject: [neutron][queens] dhcp issue In-Reply-To: <20191211183809.pnm4f33me2lpom6q@yuggoth.org> References: <20191211152331.xmre63uzald5t4tg@yuggoth.org> <20191211183809.pnm4f33me2lpom6q@yuggoth.org> Message-ID: No problem Jeremy. Thanks for information you sent me. Ignazio Il Mer 11 Dic 2019, 19:43 Jeremy Stanley ha scritto: > On 2019-12-11 18:36:16 +0100 (+0100), Ignazio Cassano wrote: > > Indeed when more subnet of a provider vlan are created, instance > > on one of the subnet receives the static routes of all subnets. Is > > it correct? > [...] > > I'm afraid my expertise in this matter is limited to directly > managing DHCP servers, I'm not personally familiar with running > Neutron. Hopefully someone with more architectural background on > Neutron itself can answer that part of your question. > -- > Jeremy Stanley > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haleyb.dev at gmail.com Wed Dec 11 20:13:26 2019 From: haleyb.dev at gmail.com (Brian Haley) Date: Wed, 11 Dec 2019 15:13:26 -0500 Subject: [neutron][queens] dhcp issue In-Reply-To: References: <20191211152331.xmre63uzald5t4tg@yuggoth.org> Message-ID: <87c59311-e3dd-84a0-0344-602e517ea93c@gmail.com> On 12/11/19 12:36 PM, Ignazio Cassano wrote: > Hello Jeremy, thanks for your help. > Indeed when more subnet of a provider vlan are created, instance on one > of the subnet receives the static routes of all subnets. > Is it correct? Yes, this behavior is correct, the dhcp-agent will add host routes for all subnets in a network. I looks like in your case you had a network with 12 subnets? I wonder whether it's worth checking either the number of subnets is below some threshold (10?) and just stop supplying the routes, or if there is some other way to determine the packet size. The only issue with limiting the packet size in the reply is the admin wouldn't know we did it, since adding the subnet is asynchronous from the dhcp-agent adding it's option logic, and things might fail in random ways, but I guess having a running dnsmasq is better than not. I guess at worst we should document that adding too many subnets, or extra dhcp options for that matter, could cause issues. -Brian > > Il Mer 11 Dic 2019, 16:29 Jeremy Stanley > ha scritto: > > On 2019-12-11 10:10:17 +0100 (+0100), Ignazio Cassano wrote: > [...] > > Dec 11 09:49:06 podto2-osctrl01 dnsmasq-dhcp[70093]: cannot send > >     DHCP/BOOTP option 121: no space left in packet > [...] > > This error is fairly straightforward. DHCP replies are limited to > what you can fit in a single UDP packet and how large of a datagram > the client agrees to accept. The number of routes you're trying to > include in via option 121 could be the problem (is it a bunch?) but > also the other fields and their lengths can also contribute, as can > the MTU on that network segment: > > https://tools.ietf.org/html/rfc2131 > > According to IETF RFC 2131 you should be able to assume clients will > accept a datagram of up to 576 octets in length (which implies your > entire options field can use at least 312 octets). Anything over > this requires negotiation of a maximum DHCP message size with the > client. > -- > Jeremy Stanley > From gouthampravi at gmail.com Wed Dec 11 20:53:24 2019 From: gouthampravi at gmail.com (Goutham Pacha Ravi) Date: Thu, 12 Dec 2019 02:23:24 +0530 Subject: [manila] No weekly meeting on 12/12/2019 Message-ID: Hi Zorillas and other interested stackers, Numerous Manila contributors convened today for a virtual PTG and discussed a range of topics [1]. I will compile the concerns, agreements and action items and send out a summary to this list soon. We also decided that we would skip this week's meeting since some of us will be on PTO tomorrow 12/12 and most pressing concerns were addressed during the Virtual PTG. If you have any questions/concerns, please bring them to #openstack-manila on freenode; or post on this mailing list. See you all on 19th Dec 2019 at 15:00 UTC! [2] Cheers! Goutham [1] https://etherpad.openstack.org/p/shanghai-ptg-manila-virtual [2] https://wiki.openstack.org/wiki/Manila/Meetings From brett.milford at canonical.com Thu Dec 12 01:49:57 2019 From: brett.milford at canonical.com (Brett Milford) Date: Thu, 12 Dec 2019 12:49:57 +1100 Subject: [nova][cinder][ops] question/confirmation of legacy vol attachment migration In-Reply-To: <20191210121530.v6mhioyxeumyihpc@localhost> References: <37e953ee-f3c8-9797-446f-f3e3db9dcad6@gmail.com> <20191010100050.hn546tikeihaho7e@localhost> <20191017102419.pa3qqlqgrlp2b7qx@localhost> <20191210121530.v6mhioyxeumyihpc@localhost> Message-ID: On Tue, Dec 10, 2019 at 11:15 PM Gorka Eguileor wrote: > > On 06/12, Brett Milford wrote: > > On Fri, Nov 22, 2019 at 3:54 AM Matt Riedemann wrote: > > > > > > On 10/17/2019 5:24 AM, Gorka Eguileor wrote: > > > > I stand by my initial recommendation, being able to update the existing > > > > attachment to add the connection information from Nova. > > > > > > OK, thanks for the input and thoughtfulness on this. I've abandoned my > > > change since I'm not going to be pushing this boulder anymore but left > > > notes in the change in case someone else wants to pick it up some day. > > > > > > Note to nova cores: this means we'll have legacy volume attachment > > > compat code around forever. > > > > > > -- > > > > > > Thanks, > > > > > > Matt > > > > > > > Hi Groka, Cinder & Nova devs, > > > > Following up this thread from the context of > > https://review.opendev.org/#/c/579004/ > > > > To summarise the discussion: > > - initialize_connection shouldn't be called more than once, as it > > may mess up some drivers. > > - To (safely) refresh connection_info a new api for cinder is required. > > Hi, > > It may be a new API or a modification of one of the existing ones. > > > > - a patch to nova, such as #579004 could make a call to this new api > > to refresh connection info on events such as reboot. > > > > Is there any other context to this issue I've missed or an alternate > > path to solving this? > > > > Does the creation of a new api for cinder have any implications for > > being able to backport this patch for nova? > > > > I don't think backporting it in Nova would do us much good, since the > Cinder code would not be backportable (it's a new feature, not a bug > fix). > > > > What would be the process for me to kick off this work? > > > > You should propose a Cinder spec [1] for your work and you may provide a > WIP patch to Cinder at the same time, though depending on the review you > may have to completely change that code later. > > Cheers, > Gorka. > > [1]: https://opendev.org/openstack/cinder-specs > > > Thanks for your help, > > Brett (bcm) > > -- > > ❯ brett > > > Cheers Matt, Groka, I thought this might be the case. -- ❯ brett From zhangbailin at inspur.com Thu Dec 12 13:24:11 2019 From: zhangbailin at inspur.com (=?gb2312?B?QnJpbiBaaGFuZyjVxbDZwdYp?=) Date: Thu, 12 Dec 2019 13:24:11 +0000 Subject: [nova] live migration with the NUMA topology Message-ID: <5110f073c61e44c391702055743c63ea@inspur.com> Hi folks:: About http://specs.openstack.org/openstack/nova-specs/specs/train/approved/numa-aware-live-migration.html this spec in Train. I have a question, if the destination server's NUMA topology (e.g. nume_node=2) < source server's NUMA topology (e.g. numa_noed=4) in a instance. If I am living migration *this* instance, what will be happened? Rollback and keep the instance to the original status? Or make it to ERROR? In that SPEC I had not find the details about the red description in "Third, information about the instance’s new NUMA characteristics needs to be generated on the destination (an InstanceNUMATopolgy object is not enough, more on that later)", or lack of careful reading :). Anyway, I want to know how to deal with this NUMA topology during live migration? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Thu Dec 12 13:36:22 2019 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 12 Dec 2019 13:36:22 +0000 Subject: [infra][zuul][kolla] ansible_python_interpreter & stable branches Message-ID: Hi, It appears that Zuul is now using python3 as the ansible_python_interpreter for Ubuntu [0]. This seems to also affect stable branches. For kolla ansible this has broken our deploy job which uses the pip module to install our dependencies, and now does so using python3. For Stein and earlier, our scripts use #/usr/bin/env python, which links to python2. I think we can easily fix this by setting ansible_python_interpreter in our zuul vars on stable branches, but perhaps there are some other unintended side-effects? Cheers, Mark [0] https://zuul.opendev.org/t/openstack/build/e0c56c36ef064c9dae00d31ea2979b32/log/primary/logs/facts.json#1308 From thierry at openstack.org Thu Dec 12 13:50:19 2019 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 12 Dec 2019 14:50:19 +0100 Subject: [largescale-sig] Next meeting: Dec 18, 9utc Message-ID: <21d84bcd-1694-c327-0de2-7f7322d6eb7c@openstack.org> Hi everyone, The Large Scale SIG will have a meeting next week on Wednesday, Dec 18 at 9 UTC[1] in #openstack-meeting on IRC. [1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=20191218T09 Since the first meeting[2], the SIG was officially approved[3] and is now listed on the SIG page[4]. [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011206.html [3] https://review.opendev.org/696302 [4] https://governance.openstack.org/sigs/ For our next meeting I'd like us to work on refining the initial goals that we discussed at the first meeting, ideally splitting them into early objectives and initial steps. The two goals are documented in the following etherpads: Scaling within one cluster, and instrumentation of the bottlenecks: https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling Document large scale configuration and tips &tricks: https://etherpad.openstack.org/p/large-scale-sig-documentation In preparation for the meeting, please review those documents and add specific actions, notes and comments to them. Regards, -- Thierry Carrez (ttx) From mriedemos at gmail.com Thu Dec 12 13:59:12 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Thu, 12 Dec 2019 07:59:12 -0600 Subject: [nova] live migration with the NUMA topology In-Reply-To: <5110f073c61e44c391702055743c63ea@inspur.com> References: <5110f073c61e44c391702055743c63ea@inspur.com> Message-ID: On 12/12/2019 7:24 AM, Brin Zhang(张百林) wrote: > I have a question, if the destination server's NUMA topology (e.g. > nume_node=2) < source server's NUMA topology (e.g. numa_noed=4) in a > instance. If I am living migration *this* instance, what will be > happened? Rollback and keep the instance to the original status? Or make > it to ERROR? In that SPEC I had not find the details about the red > description in "Third, information about the instance’s new NUMA > characteristics needs to be generated on the destination (an > InstanceNUMATopolgy object is not enough, more on that later)", or lack > of careful reading J. Anyway, I want to know how to deal with this NUMA > topology during live migration? Artom can answer this in detail but I would expect the claim to fail on the dest host here: https://github.com/openstack/nova/blob/20.0.0/nova/compute/manager.py#L6656 Which will be handled here in conductor: https://github.com/openstack/nova/blob/20.0.0/nova/conductor/tasks/live_migrate.py#L502 And trigger a "reschedule" to an alternate host. If we run out of alternates then MaxRetriesExceeded would be raised: https://github.com/openstack/nova/blob/20.0.0/nova/conductor/tasks/live_migrate.py#L555 And handled here as NoValidHost: https://github.com/openstack/nova/blob/20.0.0/nova/conductor/manager.py#L457 The vm_state should be unchanged (stay ACTIVE) but the migration status will go to "error". Artom has been working on functional tests [1] but I'm not sure if they cover this kind of scenario - I'd hope they would. Of course the simpler answer might be, and it would be cool if it is, the scheduler should not select the dest host that can't fit the instance so we don't even get to the low-level compute resource claim. [1] https://review.opendev.org/#/c/672595/ -- Thanks, Matt From skaplons at redhat.com Thu Dec 12 14:06:37 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 12 Dec 2019 15:06:37 +0100 Subject: [neutron][devstack] Future of lib/neutron and lib/neutron-legacy Message-ID: <48325993-0233-437B-ABC5-D038F0785F04@redhat.com> Hi, Few years ago (I don’t know exactly when it started as I wasn’t very active upstream than for sure) Neutron team started work on new neutron module for Devstack. See [1] for details. But since then this new module is still not ready to use. For example in gate jobs we are still using lib/neutron-legacy module as this new one is always causing some problems. Things in its current shape may be confusing for new Devstack users as in fact to deploy Neutron properly they should use lib/neutron-legacy instead of lib/neutron now. Personally I never really checked exactly what issues are there in fact but I do remember some issues with using lib/neutron e.g. on subnodes in multinode jobs - see [2]. So I would like to ask a question what do You think we should do with it. Is there maybe any volunteer who would like to continue this work on adopting lib/neutron and to finally use it in gate jobs? Or if there is no anyone and we are all fine with using old module, maybe we should remove this lib/neutron and “undeprecate” and rename lib/neutron-legacy? [1] https://github.com/openstack/devstack/commit/2a242519f71e86416e78541826cac2b54fcd04a5 [2] https://review.opendev.org/#/c/662480/ — Slawek Kaplonski Senior software engineer Red Hat From openstack at fried.cc Thu Dec 12 14:23:34 2019 From: openstack at fried.cc (Eric Fried) Date: Thu, 12 Dec 2019 08:23:34 -0600 Subject: [nova] No more meetings til 2020 Message-ID: <171ffacf-6332-c08d-ef0b-7035b340c124@fried.cc> Next nova meeting is January 2nd (2100 UTC), though I don't expect it to be heavily attended. efried . From alifshit at redhat.com Thu Dec 12 16:17:04 2019 From: alifshit at redhat.com (Artom Lifshitz) Date: Thu, 12 Dec 2019 11:17:04 -0500 Subject: [nova] live migration with the NUMA topology In-Reply-To: References: <5110f073c61e44c391702055743c63ea@inspur.com> Message-ID: On Thu, Dec 12, 2019 at 9:01 AM Matt Riedemann wrote: > > On 12/12/2019 7:24 AM, Brin Zhang(张百林) wrote: > > I have a question, if the destination server's NUMA topology (e.g. > > nume_node=2) < source server's NUMA topology (e.g. numa_noed=4) in a > > instance. If I am living migration *this* instance, what will be > > happened? Rollback and keep the instance to the original status? Or make > > it to ERROR? In that SPEC I had not find the details about the red > > description in "Third, information about the instance’s new NUMA > > characteristics needs to be generated on the destination (an > > InstanceNUMATopolgy object is not enough, more on that later)", or lack > > of careful reading J. Anyway, I want to know how to deal with this NUMA > > topology during live migration? > > Artom can answer this in detail but I would expect the claim to fail on > the dest host here: > > https://github.com/openstack/nova/blob/20.0.0/nova/compute/manager.py#L6656 > > Which will be handled here in conductor: > > https://github.com/openstack/nova/blob/20.0.0/nova/conductor/tasks/live_migrate.py#L502 > > And trigger a "reschedule" to an alternate host. If we run out of > alternates then MaxRetriesExceeded would be raised: > > https://github.com/openstack/nova/blob/20.0.0/nova/conductor/tasks/live_migrate.py#L555 > > And handled here as NoValidHost: > > https://github.com/openstack/nova/blob/20.0.0/nova/conductor/manager.py#L457 > > The vm_state should be unchanged (stay ACTIVE) but the migration status > will go to "error". > > Artom has been working on functional tests [1] but I'm not sure if they > cover this kind of scenario - I'd hope they would. > > Of course the simpler answer might be, and it would be cool if it is, > the scheduler should not select the dest host that can't fit the > instance so we don't even get to the low-level compute resource claim. Yeah, the scheduler (unless it's bypassed, obviously) shouldn't pick a host where the instance can't fit. And once we're on the host, if the claim fails (either because the scheduler was bypassed or another instance raced with ours and took our resources), we'll keep rescheduling until we can't, and then the migration fails. So what Matt wrote above is correct as well. > > [1] https://review.opendev.org/#/c/672595/ > > -- > > Thanks, > > Matt > -- Artom Lifshitz Software Engineer, OpenStack Compute DFG From zbitter at redhat.com Thu Dec 12 20:30:17 2019 From: zbitter at redhat.com (Zane Bitter) Date: Thu, 12 Dec 2019 15:30:17 -0500 Subject: [dev] Counting the chaff In-Reply-To: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> References: <20191210173214.lyv3l6avhoj3ednm@yuggoth.org> Message-ID: On 10/12/19 12:32 pm, Jeremy Stanley wrote: > (Note that there*are* plenty of high-profile projects on GitHub > like Kubernetes using more of a rebase model, with PR owners > typically force-pushing changes to address reviewer comments or CI > results, and also projects where it's their custom to squash all > commits in a PR before merging, so in those cases our normal change > counts are more applicable for ~1:1 comparisons.) Force-pushing is one thing, but projects that squash all PRs in an act of truly evil stupidity (looking at you, CPython) will undercount changes, since what would be a series of changes in OpenStack or a PR with multiple commits in another project ends up as a single commit. That GitHub (and GitLab, who came up with it first) forces you to choose between squashing independent commits in one PR or not squashing the 15 "fix tests" commits in another is a prime example of what makes it an awful tool for code review. From juliaashleykreger at gmail.com Thu Dec 12 21:28:31 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Thu, 12 Dec 2019 13:28:31 -0800 Subject: [ironic] ibmc driver Message-ID: Greetings fellow ironicans and stackers! I'd like some thoughts on the `ibmc` driver[1] that was added early this year. Specifically it merged into ironic with an operating Third-Party CI, however soon after a series of unfortunate events occurred where the Third-Party CI stopped reporting sometime in early May, and shortly their after restrictions were put into place[2]. With that being said, I feel like we should go ahead and remove this driver since it does not seem to be maintained nor tested. -Julia [1]: https://review.opendev.org/#/c/639288/ [2]: https://www.federalregister.gov/documents/2019/05/21/2019-10616/addition-of-entities-to-the-entity-list From Albert.Braden at synopsys.com Thu Dec 12 21:34:04 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Thu, 12 Dec 2019 21:34:04 +0000 Subject: Finding configuration values Message-ID: This thread consists of 6 emails, all from me: http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011465.html This illustrates a problem I've seen in Openstack. Many of the configuration settings are undocumented, or the documentation is obscure. The only way for an openstack operator to know the correct setting is to be told by an openstack developer. Sometimes I get lucky and someone tells me the answer on IRC, or here on the list. Sometimes nobody does, and then it seems that I am stuck. I have to keep bugging the developers until they tell me the answer. In this case, after searching for a week, I opened a Neutron bug, and then one of the Neutron developers told me how to fix my config. What am I doing wrong? Is there a document I should be reading, that tells how to configure openstack? Why are there no answers in #openstack? Does everyone else have this same issue, or is it only me? How can Openstack operators find out how to configure Openstack, besides begging the developers for the answer? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ken at jots.org Thu Dec 12 21:50:46 2019 From: ken at jots.org (Ken D'Ambrosio) Date: Thu, 12 Dec 2019 16:50:46 -0500 Subject: Upgrade: Juno -> Pike (or maybe later) Message-ID: Hey -- I've run a number of OpenStack clouds from multiple vendors for several years, now. But I've *never* done an upgrade. I've read some of the docs, but they don't really make it clear (at least, the ones I've thus-far found): * If going from Juno to something pretty recent is even feasible * If I have to do intermediate steps along the way * How likely it is to "just work:" complexity, common issues, etc. The fact that there's a clearly codified fallback procedure, while an inherently good thing to have, does concern me some. Any insights would be much appreciated. Thanks! -Ken From sshnaidm at redhat.com Thu Dec 12 22:00:22 2019 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Fri, 13 Dec 2019 00:00:22 +0200 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - next steps In-Reply-To: References: Message-ID: Hi, all short minutes from the meeting today about moving of Openstack Ansible modules to Openstack. 1. Because of some level of uncertainty and different opinions, the details of treatment of old modules will be under discussion in ML. I'll send a mail about this topic. 2. We agreed to have modules under "openstack." namespace and named "cloud". So regular modules will be named like "openstack.cloud.os_server" for example. 3. We agreed to keep Ansible modules as thin as possible, putting the logic into SDK. 4. Also we will keep compatibility with as much Ansible versions as possible. 5. We agreed to have manual releases of Ansible modules as much as we need. Similarly as it's done with SDK. Logs: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.00.log.html Minutes: http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.00.html Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules Next time: Thursday 19 Dec 2019 4.00 PM UTC. Thanks On Fri, Dec 6, 2019 at 12:03 AM Sagi Shnaidman wrote: > Hi, all > short minutes from the meeting today about Openstack Ansible modules. > > 1. Ansible 2.10 is going to move all modules to collections, so Openstack > modules should find a new home in Openstack repos. > 2. Namespace for openstack modules will be named "openstack.". What is > coming after the dot is still under discussion. > 3. Current modules will be migrated to collections in "openstack." as is > with their names and will be still available for playbooks (via > symlinking). It will avoid breaking people that use in their playbooks os_* > modules now. > 4. Old modules will be frozen after migrations and all development work > will go in the new modules which will live aside. > 5. Critical bugfixes to 2.9 versions will be done via Ansible GitHub repo > as usual and synced manually to "openstack." collection. It must be a very > exceptional case. > 6. Migrations are set for mid of January 2020 approximately. > 7. Modules should stay compatible with last Ansible and collections API > changed. > 8. Because current old modules are licensed with GPL and license of > Openstack is Apache2, we need to figure out if we can either relicense them > or develop new ones with different license or to continue to work on new > ones with GPL in SIG repo. Agreed to ask on legal-discuss ML. > > Long minutes: > http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.00.html > Logs: > http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-05-16.00.log.html > > Etherpad: https://etherpad.openstack.org/p/openstack-ansible-modules > Next time Thursday 12 Dec 2019 4.00 PM UTC. > > Thanks > > On Tue, Dec 3, 2019 at 8:18 PM Sagi Shnaidman wrote: > >> Hi, all >> In the meeting today we agreed to meet every Thursday starting *this >> week* at 4.00 PM UTC on #openstack-sdks channel on Freenode. We'll >> discuss everything related to Openstack Ansible modules. >> Agenda and topics are in the etherpad: >> https://etherpad.openstack.org/p/openstack-ansible-modules >> (I've created a new one, because we don't limit to Ironic modules only, >> it's about all of them in general) >> >> Short minutes from meeting today: >> Organizational: >> 1. We meet every Thursday from this week at 4.00 PM UTC on #openstack-sdks >> 2. Interested parties for now are: Ironic, Tripleo, Openstack-Ansible, >> Kolla-ansible, OpenstackSDK teams. Feel free to join and add yourself in >> the etherpad. [1] >> 3. We'll track our work in Storyboard for ansible-collections-openstack >> (in progress) >> 4. Openstack Ansible modules will live as collections under Ansible SIG >> in repo openstack/ansible-collections-openstack [2] because there are >> issues with different licensing: GPLv3 for Ansible in upstream and >> Openstack license (Apache2). >> 5. Ansible upstream Openstack modules will be merge-frozen when we'll >> have our collections fully working and will be deprecated from Ansible at >> some point in the future. >> 6. Openstack Ansible collections will be published to Galaxy. >> 7. There is a list of people that can be pinged for reviews in >> ansible-collections-openstack project, feel free to join there [1] >> >> Technical: >> 1. We use openstacksdk instead of [project]client modules. >> 2. We will rename modules to be more like os_[service_type] named, >> examples are in Ironic modules etherpad [3] >> >> Logs from meeting today you can find here: >> http://eavesdrop.openstack.org/meetings/ansible_sig/2019/ansible_sig.2019-12-03-15.01.log.html >> Please feel free to participate and add topics to agenda. [1] >> >> [1] https://etherpad.openstack.org/p/openstack-ansible-modules >> [2] https://review.opendev.org/#/c/684740/ >> [3] https://etherpad.openstack.org/p/ironic-ansible-modules >> >> Thanks >> >> On Wed, Nov 27, 2019 at 7:57 PM Sagi Shnaidman >> wrote: >> >>> Hi, all >>> >>> in the light of finding the new home place for openstack related ansible >>> modules [1] I'd like to discuss the best strategy to create Ironic ansible >>> modules. Existing Ironic modules in Ansible repo don't cover even half of >>> Ironic functionality, don't fit current needs and definitely require an >>> additional work. There are a few topics that require attention and better >>> be solved before modules are written to save additional work. We prepared >>> an etherpad [2] with all these questions and if you have ideas or >>> suggestions on how it should look you're welcome to update it. >>> We'd like to decide the final place for them, name conventions (the most >>> complex one!), what they should look like and how better to implement. >>> Anybody interested in Ansible and baremetal management in Openstack, >>> you're more than welcome to contribute. >>> >>> Thanks >>> >>> [1] https://review.opendev.org/#/c/684740/ >>> [2] https://etherpad.openstack.org/p/ironic-ansible-modules >>> >>> -- >>> Best regards >>> Sagi Shnaidman >>> >> >> >> -- >> Best regards >> Sagi Shnaidman >> > > > -- > Best regards > Sagi Shnaidman > -- Best regards Sagi Shnaidman -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Thu Dec 12 22:30:32 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Thu, 12 Dec 2019 16:30:32 -0600 Subject: Upgrade: Juno -> Pike (or maybe later) In-Reply-To: References: Message-ID: <2b7afbf9-9888-0369-ddd0-b24cd99b16a3@gmail.com> On 12/12/2019 3:50 PM, Ken D'Ambrosio wrote: > Hey -- I've run a number of OpenStack clouds from multiple vendors for > several years, now.  But I've *never* done an upgrade.  I've read some > of the docs, but they don't really make it clear (at least, the ones > I've thus-far found): > * If going from Juno to something pretty recent is even feasible > * If I have to do intermediate steps along the way > * How likely it is to "just work:" complexity, common issues, etc. > > The fact that there's a clearly codified fallback procedure, while an > inherently good thing to have, does concern me some. > > Any insights would be much appreciated. Oh man. Well, probably start here [1]. And watch some of these [2]. This thread should probably be tagged with [ops] because it's a long-standing thing that several people have done but I'm not sure how well any of it is documented and that wiki might be old. BTW, if your clouds are from vendors why aren't the vendors doing your upgrade? [1] https://wiki.openstack.org/wiki/Fast_forward_upgrades [2] https://www.openstack.org/videos/search?search=fast%20forward -- Thanks, Matt From mriedemos at gmail.com Thu Dec 12 22:38:10 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Thu, 12 Dec 2019 16:38:10 -0600 Subject: Finding configuration values In-Reply-To: References: Message-ID: <15012b8d-31d3-f856-35de-eed96cb6bf55@gmail.com> On 12/12/2019 3:34 PM, Albert Braden wrote: > This illustrates a problem I’ve seen in Openstack. Many of the > configuration settings are undocumented, or the documentation is > obscure. The only way for an openstack operator to know the correct > setting is to be told by an openstack developer. Since Pike the docs in each project should be standardized and the configuration options should be generated from code. So for example: https://docs.openstack.org/nova/latest/configuration/ https://docs.openstack.org/glance/latest/configuration/ https://docs.openstack.org/cinder/latest/configuration/ https://docs.openstack.org/neutron/latest/configuration/ So that answers your question about where to go for config docs. Now if the docs themselves are no good, which would not be a surprise to me - developers tend to care more about getting code in than making it consumable by others - then you could open a bug against the project. Each docs page should have a little bug icon in the top right for reporting a docs bug against the project for that page. However, reporting the bug doesn't mean it's going to get fixed, so if you have a solution or something to write up by all means contribute it to help the next person that runs into the same issue. Also, as for not getting a lot of help in #openstack, it entirely depends on who is around, how specific the question is and who is actually able to answer it. Sometimes that means going to the per-project channels and asking but again, the more specific you can be the better. Remember that open source doesn't necessarily mean open support so if you go to #openstack-nova and are like, "hey can someone help me debug my scheduler configuration?" you're likely not going to get much help because the channel is full of ( mostly :) ) busy developers working on things for their employer and paying customers. OpenStack is great but let's be honest and remember that its development is primarily driven by huge corporations, not hobbyists. -- Thanks, Matt From mnaser at vexxhost.com Thu Dec 12 22:39:42 2019 From: mnaser at vexxhost.com (Mohammed Naser) Date: Thu, 12 Dec 2019 17:39:42 -0500 Subject: Upgrade: Juno -> Pike (or maybe later) In-Reply-To: References: Message-ID: On Thu, Dec 12, 2019 at 4:53 PM Ken D'Ambrosio wrote: > > Hey -- I've run a number of OpenStack clouds from multiple vendors for > several years, now. But I've *never* done an upgrade. I've read some > of the docs, but they don't really make it clear (at least, the ones > I've thus-far found): > * If going from Juno to something pretty recent is even feasible Stuff will probably break, you're talking about upgrading from 11 releases. There's just SO much that happened in that period of time & transitions. > * If I have to do intermediate steps along the way Many, placement extraction, migration to cells v2 are two big ones that come to mind > * How likely it is to "just work:" complexity, common issues, etc. Quite honestly, the complexity is very high of what you're trying to accomplish and I would seriously consider redeploying your clouds on new hardware and migrating workloads at this point. > The fact that there's a clearly codified fallback procedure, while an > inherently good thing to have, does concern me some. > > Any insights would be much appreciated. > > Thanks! > > -Ken > -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From Albert.Braden at synopsys.com Thu Dec 12 23:09:28 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Thu, 12 Dec 2019 23:09:28 +0000 Subject: Finding configuration values In-Reply-To: <15012b8d-31d3-f856-35de-eed96cb6bf55@gmail.com> References: <15012b8d-31d3-f856-35de-eed96cb6bf55@gmail.com> Message-ID: This is the Rocky doc I was using: https://docs.openstack.org/neutron/rocky/configuration/neutron.html This is what it says about the rpc_workers setting: rpc_workers Type: integer Default: 1 Number of RPC worker processes for service. Is this the correct place to add something like 'If your neutron-metadata-agent.log is full of "error: [Errno 32] Broken pipe" then increase this setting' ? I'd be happy to help update the docs but I'm not sure how to get started. Opening document bugs doesn't seem to result in any change; if I could figure out how to submit changes myself that might be more helpful. How can I register as a document change submitter? -----Original Message----- From: Matt Riedemann Sent: Thursday, December 12, 2019 2:38 PM To: openstack-discuss at lists.openstack.org Subject: Re: Finding configuration values On 12/12/2019 3:34 PM, Albert Braden wrote: > This illustrates a problem I've seen in Openstack. Many of the > configuration settings are undocumented, or the documentation is > obscure. The only way for an openstack operator to know the correct > setting is to be told by an openstack developer. Since Pike the docs in each project should be standardized and the configuration options should be generated from code. So for example: From mriedemos at gmail.com Fri Dec 13 00:41:14 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Thu, 12 Dec 2019 18:41:14 -0600 Subject: Finding configuration values In-Reply-To: References: <15012b8d-31d3-f856-35de-eed96cb6bf55@gmail.com> Message-ID: On 12/12/2019 5:09 PM, Albert Braden wrote: > Is this the correct place to add something like 'If your neutron-metadata-agent.log is full of "error: [Errno 32] Broken pipe" then increase this setting' ? > > I'd be happy to help update the docs but I'm not sure how to get started. Opening document bugs doesn't seem to result in any change; if I could figure out how to submit changes myself that might be more helpful. How can I register as a document change submitter? I'm not sure if that troubleshooting type information should go into the config option help itself, likely a better place is a troubleshooting doc, like what nova has here [1]. I'm not sure if neutron has something similar in their docs. The actual config options in neutron are under neutron/conf/ so in this case [2]. As for how to get started with contributing, this is where you would get started with contributing to OpenStack in general [3]. That has the ICLA and foundation account stuff you'd need for Gerrit. This doc is about contributing to neutron specifically [4] (again each project should have a standard docs url like /latest/contributor/). [1] https://docs.openstack.org/nova/latest/admin/support-compute.html [2] https://github.com/openstack/neutron/blob/7fbba811edec6093df07f615358e90c8acac68a5/neutron/conf/service.py#L31 [3] https://docs.openstack.org/infra/manual/developers.html [4] https://docs.openstack.org/neutron/latest/contributor/ -- Thanks, Matt From zhangbailin at inspur.com Fri Dec 13 01:41:35 2019 From: zhangbailin at inspur.com (=?utf-8?B?QnJpbiBaaGFuZyjlvKDnmb7mnpcp?=) Date: Fri, 13 Dec 2019 01:41:35 +0000 Subject: =?utf-8?B?562U5aSNOiBbbGlzdHMub3BlbnN0YWNrLm9yZ+S7o+WPkV1SZTogW25vdmFd?= =?utf-8?Q?_live_migration_with_the_NUMA_topology?= In-Reply-To: References: <3ab62c25640a1f7e815e5e28e14b4386@sslemail.net> Message-ID: <46aba19ebef4429a90bf75c1615588f9@inspur.com> > -----邮件原件----- > 发件人: Artom Lifshitz [mailto:alifshit at redhat.com] > 发送时间: 2019年12月13日 0:17 > 收件人: Matt Riedemann > 抄送: OpenStack Discuss > 主题: [lists.openstack.org代发]Re: [nova] live migration with the NUMA > topology > > On Thu, Dec 12, 2019 at 9:01 AM Matt Riedemann > wrote: > > > > On 12/12/2019 7:24 AM, Brin Zhang(张百林) wrote: > > > I have a question, if the destination server's NUMA topology (e.g. > > > nume_node=2) < source server's NUMA topology (e.g. numa_noed=4) in a > > > instance. If I am living migration *this* instance, what will be > > > happened? Rollback and keep the instance to the original status? Or > > > make it to ERROR? In that SPEC I had not find the details about the > > > red description in "Third, information about the instance’s new NUMA > > > characteristics needs to be generated on the destination (an > > > InstanceNUMATopolgy object is not enough, more on that later)", or > > > lack of careful reading J. Anyway, I want to know how to deal with > > > this NUMA topology during live migration? > > > > Artom can answer this in detail but I would expect the claim to fail > > on the dest host here: > > > > https://github.com/openstack/nova/blob/20.0.0/nova/compute/manager.py# > > L6656 > > > > Which will be handled here in conductor: > > > > https://github.com/openstack/nova/blob/20.0.0/nova/conductor/tasks/liv > > e_migrate.py#L502 > > > > And trigger a "reschedule" to an alternate host. If we run out of > > alternates then MaxRetriesExceeded would be raised: > > > > https://github.com/openstack/nova/blob/20.0.0/nova/conductor/tasks/liv > > e_migrate.py#L555 > > > > And handled here as NoValidHost: > > > > https://github.com/openstack/nova/blob/20.0.0/nova/conductor/manager.p > > y#L457 > > > > The vm_state should be unchanged (stay ACTIVE) but the migration > > status will go to "error". > > > > Artom has been working on functional tests [1] but I'm not sure if > > they cover this kind of scenario - I'd hope they would. > > > > Of course the simpler answer might be, and it would be cool if it is, > > the scheduler should not select the dest host that can't fit the > > instance so we don't even get to the low-level compute resource claim. > > Yeah, the scheduler (unless it's bypassed, obviously) shouldn't pick a host > where the instance can't fit. And once we're on the host, if the claim fails > (either because the scheduler was bypassed or another instance raced with > ours and took our resources), we'll keep rescheduling until we can't, and then > the migration fails. So what Matt wrote above is correct as well. > Yes, it's better to do this scenario in [1] patch. And I have a suggestion, in https://github.com/openstack/nova-specs/blob/master/specs/train/approved/numa-aware-live-migration.rst#proposed-change to claify this will be better, rather than said " more on that later ", or add a jump link to the "more details info", and this will be good to the reader. > > > > [1] https://review.opendev.org/#/c/672595/ > > > > -- > > > > Thanks, > > > > Matt > > > > > -- > Artom Lifshitz > Software Engineer, OpenStack Compute DFG > brinzhang From melwittt at gmail.com Fri Dec 13 01:47:11 2019 From: melwittt at gmail.com (melanie witt) Date: Thu, 12 Dec 2019 17:47:11 -0800 Subject: [nova][ironic] need help understanding 'cpu_arch' in nova ironic driver Message-ID: Hey all, Recently I'm trying to figure out how to fulfill a use case in the nova ironic driver around treating an ironic node's 'cpu_arch' as optional. This is coming up as a result of a downstream issue [1] and recent change on the ironic side [2] to make cpu_arch optional for iscsi deployments. The change makes it so that ironic will _not_ include a 'cpu_arch' attribute as part of a node's properties if iscsi. On the nova side, we have a filter scheduler ImagePropertiesFilter which will only match a node if the architecture, hypervisor_type, and vm_mode in the glance image properties match the cpu_arch of the node. We have always pulled the cpu_arch of the from the ironic node properties since the original ironic driver code was added to nova. Now, the use case I'm trying to solve today [3] is where an iscsi ironic deployment has no cpu_arch specified on the ironic side and the deployer wants to use glance images with architecture specified in their image properties. Today (and historically always) the request to create an instance in this situation will fail with NoValidHost because the nova ironic driver could not determine a cpu_arch from the ironic node. My questions are: * Is this a valid thing to want to do? * If if is valid, what is the correct way that we should handle missing cpu_arch in the nova ironic driver? Should we guess at a default cpu_arch? If so, how? Or should we add a new config option where a default cpu_arch can be specified? Or, other? Thanks in advance for any input you can offer. Cheers, -melanie [1] https://bugzilla.redhat.com/show_bug.cgi?id=1653788 [2] https://review.opendev.org/620634 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1688838 From florian at datalounges.com Fri Dec 13 06:17:18 2019 From: florian at datalounges.com (Florian Rommel) Date: Fri, 13 Dec 2019 08:17:18 +0200 Subject: Upgrade: Juno -> Pike (or maybe later) In-Reply-To: References: Message-ID: Hi, I agree here, maybe a forklift upgrade might be easier. As Mohammed pointed out the placement and cells are one of the big ones. If anything, you would need to upgrade just pre placement and pre cell and then use the upgrade path there to overcome that hurdle and then move forward. With 11 (at least) releases , it’s gonna be rough. I would argue, carefully with the amount of changes, you are trying to migrate windows 3.11 to windows 10... It probably can be done but (aside the 32 bit etc:)) ... wow If you need to upgrade on existing hardware, why not consolidate some instances and free up some nodes and build the environment then up? Good luck!:) > On 13. Dec 2019, at 0.41, Mohammed Naser wrote: > > On Thu, Dec 12, 2019 at 4:53 PM Ken D'Ambrosio wrote: >> >> Hey -- I've run a number of OpenStack clouds from multiple vendors for >> several years, now. But I've *never* done an upgrade. I've read some >> of the docs, but they don't really make it clear (at least, the ones >> I've thus-far found): >> * If going from Juno to something pretty recent is even feasible > > Stuff will probably break, you're talking about upgrading from 11 releases. > > There's just SO much that happened in that period of time & transitions. > >> * If I have to do intermediate steps along the way > > Many, placement extraction, migration to cells v2 are two big ones > that come to mind > >> * How likely it is to "just work:" complexity, common issues, etc. > > Quite honestly, the complexity is very high of what you're trying to > accomplish and I would seriously consider redeploying your clouds > on new hardware and migrating workloads at this point. > >> The fact that there's a clearly codified fallback procedure, while an >> inherently good thing to have, does concern me some. >> >> Any insights would be much appreciated. >> >> Thanks! >> >> -Ken >> > > > -- > Mohammed Naser — vexxhost > ----------------------------------------------------- > D. 514-316-8872 > D. 800-910-1726 ext. 200 > E. mnaser at vexxhost.com > W. https://vexxhost.com > From merlin.blom at bertelsmann.de Fri Dec 13 07:48:30 2019 From: merlin.blom at bertelsmann.de (Blom, Merlin, NMU-OI) Date: Fri, 13 Dec 2019 07:48:30 +0000 Subject: [openstacksdk][nova] distinguish image and boot from volume instances Message-ID: Hey, I wrote a function for distinguishing an image instance from an boot from volume instance for billing purpose. But it does not work this way, it seems as if BfV Instances also have the Image property set. Is there an elegant solution other than analyzing the volume attachment state? This cloud be a problem, because an image instance cloud also have volumes. def getBootFromVolInstances(serverId): '''Determines wether an instance is a boot frome volume instance''' server = conn.compute.find_server(serverId, ignore_missing=True) if server: if server.image: return False else: return True return False Merlin Blom Cloud System Engineer E-Mail: merlin.blom at bertelsmann.de arvato-systems.de -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5195 bytes Desc: not available URL: From jean-philippe at evrard.me Fri Dec 13 08:56:37 2019 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Fri, 13 Dec 2019 09:56:37 +0100 Subject: [tc] Follow up on action tems Message-ID: <4ca40b2e690eb5f6eb18d88782bb149d3b261b0e.camel@evrard.me> Hello TC members, - There are still business opportunities merging for 2019. The process for 2020 need to start soon. Any takers? - We have pending action items from the last meeting: - ricolin: contact to Infra team to see if we got more ARM server in our Nodepool - ricolin/ttx: track the merger of the SIGs self-healing and auto- scaling - mnaser: static hosting story: Is it restarting? Where are we now? - ttx: Work on technical vision reflection update - ricolin/evrardjp: Expect/Request an update from Telemetry team. As mentioned in our channel [7], Rong Zh (PTL for Telemetry) will provide a Telemetry status report (high chance to send it out next week). - jungleboyj: analysis of the user survey. The patch is out for TCs to review [6] - ttx: make sure to start the discussion about uc/tc merge in January after the ops meetup - mnaser: write documentation in governance to clarify what was discussed about stable policy - We have new action items: - We need to update our charter for the election terms, to be more flexible. Who is okay with starting this? - We have plenty of things which are not reviewed, or WIP. Could you update (y)our patches please with either changes or votes? - aprice has asked me (JP) to write a report for the TC and SIG activities of 2019. If you want to know what I am cooking, or want to help on the text, please join here [5] - In the next elections, we'll be reducing our (=TC) size temporarily due to the current elections timing proposal. In order to not have this happen again, we should propose a charter change as soon as possible. Can someone tackle this? Please see the conversations here [1] and here [2] - We still have long action items from PTG: - For community goal `Drop Python 2.7 Support`, you can find the status update on the ML [9]. React there if you need. - The community goal `project specific PTL and contributor guides` proposal patch is under review [8] and will be merged soon. - gmann is sending an official goal schedule plan [3] . We need TC members to review this as well, as soon as possible. We also need to discuss when should we start the first step for next cycle in schedule during next meeting (if/when patch landed). - Please also review other patches, like [4] . And finally, some things to keep in mind: - We need to promote the first contact sig to organisations that might need help on getting things achieved in OpenStack. Regards, JP & Rico [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011396.html [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011155.html [3] https://review.opendev.org/#/c/698299 [4] https://review.opendev.org/#/c/691737 [5] https://etherpad.openstack.org/p/2019-openstack-foundation-annual-report-draft [6] https://review.opendev.org/#/c/698582/ [7] http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-12-10.log.html#t2019-12-10T08:54:08 [8] https://review.opendev.org/#/c/691737 [9] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011432.html From jean-philippe at evrard.me Fri Dec 13 09:00:11 2019 From: jean-philippe at evrard.me (Jean-Philippe Evrard) Date: Fri, 13 Dec 2019 10:00:11 +0100 Subject: [all][tc] What happened in OpenStack Governance recently Message-ID: <0ec9382a68a45837a115ad9af773f8c4e888dd11.camel@evrard.me> Hello folks! We've finally made up our mind with the release naming process. We got a majority of the TC voting positively for a simplified release naming process [0]. Talking about naming, V-release naming poll already started [6]. Go pick your favorit now! Voting will officially end December 16, 2019 at 23:59 UTC! So... pick now! Thanks for Colleen Murphy (cmurphy) and Ghanshyam Mann (gmann) starting a policy pop up team [2]. Please raise your hand if you're interested to participate. devstack already migrated to Python3 and most of our non-python2 specific jobs are also migrated. Your test jobs (like grenade jobs) could need migration, like [1]. Also, if you want to know a full status update of the `Drop Python 2.7 Support`, you can find the status update in [9]. The ansible-sig has new repositories! ansible-collections-openstack [4], ansible-plugin-container-connection will be joining soon [5] There is a new charm charm-cinder-backup-swift-proxy [3] SIG guideline [8] is landed, so if you're maintaining or starting a SIG. Please check it out and provide your feedback to make it getting better. Thanks to Graham Hayes, the last OpenStack Foundation board meeting has officially approved to the savedotorg campaign [7]. Have a good week-end, enjoy your end of year holidays (assuming you have some soon), and merry Christmas/happy new year for those who celebrate them! :) Regards, JP & Rico [0] https://review.opendev.org/#/c/695071/3 [1] https://review.opendev.org/#/c/695088/ [2] https://governance.openstack.org/tc/reference/popup-teams.html#secure-default-policies [3] https://review.opendev.org/696723 [4] https://review.opendev.org/684740 [5] https://review.opendev.org/696737 [6] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011477.html [7] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011155.html [8] https://governance.openstack.org/sigs/reference/sig-guideline.html [9] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011432.html From thierry at openstack.org Fri Dec 13 09:11:21 2019 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 13 Dec 2019 10:11:21 +0100 Subject: Upgrade: Juno -> Pike (or maybe later) In-Reply-To: References: Message-ID: <519cf624-06d2-1cd3-7051-83f29795dc62@openstack.org> Ken D'Ambrosio wrote: > Hey -- I've run a number of OpenStack clouds from multiple vendors for > several years, now.  But I've *never* done an upgrade.  I've read some > of the docs, but they don't really make it clear (at least, the ones > I've thus-far found): > * If going from Juno to something pretty recent is even feasible > * If I have to do intermediate steps along the way > * How likely it is to "just work:" complexity, common issues, etc. > > The fact that there's a clearly codified fallback procedure, while an > inherently good thing to have, does concern me some. > > Any insights would be much appreciated. OVH did migrate all its regions from Juno to Newton, you should be able to find presentations on that from past summits videos. Similarly, Verizon media / Oath did migrate from Juno to Ocata. James Penick did presentations and some blog articles about it. But yes, as others said, that would certainly not be the first upgrade I ever attempted. Juno pre-dates most of the work on simplifying upgrades, so it's a painful starting point. -- Thierry Carrez (ttx) From dtantsur at redhat.com Fri Dec 13 12:22:37 2019 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Fri, 13 Dec 2019 13:22:37 +0100 Subject: [ironic] ibmc driver In-Reply-To: References: Message-ID: Hi, On Thu, Dec 12, 2019 at 10:30 PM Julia Kreger wrote: > Greetings fellow ironicans and stackers! > > I'd like some thoughts on the `ibmc` driver[1] that was added early > this year. Specifically it merged into ironic with an operating > Third-Party CI, however soon after a series of unfortunate events > occurred where the Third-Party CI stopped reporting sometime in early > May, and shortly their after restrictions were put into place[2]. With > that being said, I feel like we should go ahead and remove this driver > since it does not seem to be maintained nor tested. > Do you mean deprecate and remove in V or just straight remove? I'd follow the regular procedure. Otherwise, it seems that we have no other options :( An interesting topic to discuss is how to avoid such situations when a driver only lives 1-2 cycles.. Dmitry > > -Julia > > > [1]: https://review.opendev.org/#/c/639288/ > [2]: > https://www.federalregister.gov/documents/2019/05/21/2019-10616/addition-of-entities-to-the-entity-list > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dtantsur at redhat.com Fri Dec 13 12:31:23 2019 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Fri, 13 Dec 2019 13:31:23 +0100 Subject: [nova][ironic] need help understanding 'cpu_arch' in nova ironic driver In-Reply-To: References: Message-ID: Thank you for starting this! On Fri, Dec 13, 2019 at 2:48 AM melanie witt wrote: > Hey all, > > Recently I'm trying to figure out how to fulfill a use case in the nova > ironic driver around treating an ironic node's 'cpu_arch' as optional. > > This is coming up as a result of a downstream issue [1] and recent > change on the ironic side [2] to make cpu_arch optional for iscsi > deployments. The change makes it so that ironic will _not_ include a > 'cpu_arch' attribute as part of a node's properties if iscsi. > > On the nova side, we have a filter scheduler ImagePropertiesFilter which > will only match a node if the architecture, hypervisor_type, and vm_mode > in the glance image properties match the cpu_arch of the node. We have > always pulled the cpu_arch of the from the ironic node properties since > the original ironic driver code was added to nova. > > Now, the use case I'm trying to solve today [3] is where an iscsi ironic > deployment has no cpu_arch specified on the ironic side and the deployer > wants to use glance images with architecture specified in their image > properties. Today (and historically always) the request to create an > instance in this situation will fail with NoValidHost because the nova > ironic driver could not determine a cpu_arch from the ironic node. > I tend to think that if the image requires an explicit architecture, we should keep failing. However, hypervisor_type==baremetal should probably be a valid case (and I don't really know what vm_mode is, so cannot comment here). > > My questions are: > > * Is this a valid thing to want to do? > > * If if is valid, what is the correct way that we should handle missing > cpu_arch in the nova ironic driver? Should we guess at a default > cpu_arch? If so, how? Or should we add a new config option where a > default cpu_arch can be specified? Or, other? > Worst case, we can document cpu_arch as being required for nova. A bit inconvenient, but certainly not the end of the world. Dmitry > > Thanks in advance for any input you can offer. > > Cheers, > -melanie > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1653788 > [2] https://review.opendev.org/620634 > [3] https://bugzilla.redhat.com/show_bug.cgi?id=1688838 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Fri Dec 13 12:54:46 2019 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 13 Dec 2019 13:54:46 +0100 Subject: [tc] Concluding the "2019 TC vision" exercise Message-ID: <0e9f934f-80e9-4204-1722-4fb963bec412@openstack.org> Hi everyone, Back in 2016-2017, TC members participated in stewardship training at Zingerman's. Part of their framework relies on the concept of "visioning": how describing in details a desirable future helps you get there. As a result of that training, TC members in 2017 created such a "vision"[1], set in 2019. As the sole TC survivor of these ancient times and as we close the 2019 year, it sounds appropriate to reflect back on this exercise and discuss if anything more should be done in that area. [1] https://governance.openstack.org/tc/resolutions/20170404-vision-2019.html The first part of the vision, "Navigating with Constellations", was describing efforts to make OpenStack components combinations more easy to understand for specific use cases, and individual components more easily reusable in other types of deployments. The "constellation" concept never really took off, and product management is still very much done downstream of OpenStack. The "vision for OpenStack Clouds" living document[2] helped us refine our scope and better answer the "What is OpenStack" question. The OpenStack Map[3] helps navigate the ecosystem, and content on the software pages on openstack.org is now more directly driven[4] from the project teams, but overall making OpenStack easier to navigate is still very much a work in progress. That said, individual components are now generally easier to reuse in other contexts, as seen with Ironic reuse in Metal³. The second part of the vision, "Working with Adjacent Communities", was describing how we should interact and combine our work with components produced by adjacent communities, rather than beginning and ending with OpenStack. While the specific outcomes mentioned in the vision never happened, several efforts resulted in OpenStack having a more definite scope and place in the general technology landscape. Success of Kubernetes as an interoperable application deployment API helped position OpenStack more as an infrastructure provider. Our technical and non-technical experience was shared with adjacent communities like the cloud-native communities, but also with new open infrastructure projects under the OSF. The third part, "Embracing Community Diversity", is probably where we've seen the less progress in the last 3 years. It describes efforts to get a more diverse contributor base and TC representation. I feel like we still have a long way to go in that area, although things are gradually improving: we have seen more and more users of OpenStack step up and lead project teams, we are also seeing people with more of an operator background be elected to the TC, and this year the TC saw its first APAC-based member. The last part of the vision, "Growing New Leaders" describes facilitating inter-project work. Several initiatives have been put in place in that area since 2017. Community goals[5], SIGs[6], and more recently pop-up teams[7] (a concept which was described in the vision) all try to facilitate forming cross-disciplinary groups to advance complex inter-project issues in OpenStack. In summary, little of the specific plans described in the vision have actually been put in place, and the topics mentioned in there are all still work in progress. That said, the vision was a useful exercise in that it defined areas of focus for the TC stewardship of the community, and it did definitely inspire several of the plans we put in place those last 3 years. It was a time-consuming exercise though, so I'm not sure we should go through that exercise again and painting a vision for 2023. Comments, thoughts? [2] https://governance.openstack.org/tc/reference/technical-vision.html [3] http://www.openstack.org/openstack-map [4] https://opendev.org/osf/openstack-map/ [5] https://governance.openstack.org/tc/goals/index.html [6] https://governance.openstack.org/sigs/ [7] https://governance.openstack.org/tc/reference/popup-teams.html -- Thierry Carrez (ttx) From juliaashleykreger at gmail.com Fri Dec 13 14:14:14 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Fri, 13 Dec 2019 06:14:14 -0800 Subject: [ironic] ibmc driver In-Reply-To: References: Message-ID: On Fri, Dec 13, 2019 at 4:24 AM Dmitry Tantsur wrote: > > Hi, > > On Thu, Dec 12, 2019 at 10:30 PM Julia Kreger wrote: >> >> Greetings fellow ironicans and stackers! >> >> I'd like some thoughts on the `ibmc` driver[1] that was added early >> this year. Specifically it merged into ironic with an operating >> Third-Party CI, however soon after a series of unfortunate events >> occurred where the Third-Party CI stopped reporting sometime in early >> May, and shortly their after restrictions were put into place[2]. With >> that being said, I feel like we should go ahead and remove this driver >> since it does not seem to be maintained nor tested. > > > Do you mean deprecate and remove in V or just straight remove? I'd follow the regular procedure. Otherwise, it seems that we have no other options :( Either... Kind of why I emailed the mailing list. :) I'd lean for deprecated and take that route, but maybe someone has interest in keeping it working > > An interesting topic to discuss is how to avoid such situations when a driver only lives 1-2 cycles.. Knowing a preference may exist for us to avoid the word support, I feel like it was a preview that didn't pan out. I don't feel that fear of this happening again should change any of our patterns moving forward though, since that would be the exact opposite of what is needed to encourage adoption. We also have some drivers that do the minimal, some that are engaged, and some that are kind of like they have a warp drive attached. I'd love for us to merge the coffee driver as a POC interface to help people understand mechanics, fwiw. Not that we can have a coffee machine wired into CI. "launched" "has_impulse" "has_warp_drive"? -Julia > > Dmitry > >> >> >> -Julia >> >> >> [1]: https://review.opendev.org/#/c/639288/ >> [2]: https://www.federalregister.gov/documents/2019/05/21/2019-10616/addition-of-entities-to-the-entity-list >> From juliaashleykreger at gmail.com Fri Dec 13 14:18:55 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Fri, 13 Dec 2019 06:18:55 -0800 Subject: [nova][ironic] need help understanding 'cpu_arch' in nova ironic driver In-Reply-To: References: Message-ID: On Fri, Dec 13, 2019 at 4:34 AM Dmitry Tantsur wrote: > > Thank you for starting this! > > On Fri, Dec 13, 2019 at 2:48 AM melanie witt wrote: >> >> Hey all, >> >> Recently I'm trying to figure out how to fulfill a use case in the nova >> ironic driver around treating an ironic node's 'cpu_arch' as optional. >> >> This is coming up as a result of a downstream issue [1] and recent >> change on the ironic side [2] to make cpu_arch optional for iscsi >> deployments. The change makes it so that ironic will _not_ include a >> 'cpu_arch' attribute as part of a node's properties if iscsi. >> >> On the nova side, we have a filter scheduler ImagePropertiesFilter which >> will only match a node if the architecture, hypervisor_type, and vm_mode >> in the glance image properties match the cpu_arch of the node. We have >> always pulled the cpu_arch of the from the ironic node properties since >> the original ironic driver code was added to nova. >> >> Now, the use case I'm trying to solve today [3] is where an iscsi ironic >> deployment has no cpu_arch specified on the ironic side and the deployer >> wants to use glance images with architecture specified in their image >> properties. Today (and historically always) the request to create an >> instance in this situation will fail with NoValidHost because the nova >> ironic driver could not determine a cpu_arch from the ironic node. > > > I tend to think that if the image requires an explicit architecture, we should keep failing. However, hypervisor_type==baremetal should probably be a valid case (and I don't really know what vm_mode is, so cannot comment here). > >> >> >> My questions are: >> >> * Is this a valid thing to want to do? >> >> * If if is valid, what is the correct way that we should handle missing >> cpu_arch in the nova ironic driver? Should we guess at a default >> cpu_arch? If so, how? Or should we add a new config option where a >> default cpu_arch can be specified? Or, other? > > > Worst case, we can document cpu_arch as being required for nova. A bit inconvenient, but certainly not the end of the world. > Now that I understand where this entire discussion is coming from,.. there seems to be a strong desire among some of the larger operators (and kind of has been) to be able to have one ironic or as few ironics to manage their hardware fleet as possible. So I'm not sure how well it would work if it is always required that they create the cpu arch field. Then again, most of these deployments use inspector to also do wiring and system configuration validation prior to deployment, so they would have cpu_arch then.... Hmmm, this is a conundrum. > Dmitry > >> >> >> Thanks in advance for any input you can offer. >> >> Cheers, >> -melanie >> >> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1653788 >> [2] https://review.opendev.org/620634 >> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1688838 >> From openstack at fried.cc Fri Dec 13 14:22:49 2019 From: openstack at fried.cc (Eric Fried) Date: Fri, 13 Dec 2019 08:22:49 -0600 Subject: Finding configuration values In-Reply-To: References: <15012b8d-31d3-f856-35de-eed96cb6bf55@gmail.com> Message-ID: > As for how to get started with contributing, this is where you would get > started with contributing to OpenStack in general [3]. That has the ICLA > and foundation account stuff you'd need for Gerrit. > [3] https://docs.openstack.org/infra/manual/developers.html Yes, please contribute. If your style is more "interactive", find me in #openstack-mentoring and I'd be delighted to help you get started. efried . From dtantsur at redhat.com Fri Dec 13 14:23:55 2019 From: dtantsur at redhat.com (Dmitry Tantsur) Date: Fri, 13 Dec 2019 15:23:55 +0100 Subject: [ironic] ibmc driver In-Reply-To: References: Message-ID: Hi, On Fri, Dec 13, 2019 at 3:14 PM Julia Kreger wrote: > On Fri, Dec 13, 2019 at 4:24 AM Dmitry Tantsur > wrote: > > > > Hi, > > > > On Thu, Dec 12, 2019 at 10:30 PM Julia Kreger < > juliaashleykreger at gmail.com> wrote: > >> > >> Greetings fellow ironicans and stackers! > >> > >> I'd like some thoughts on the `ibmc` driver[1] that was added early > >> this year. Specifically it merged into ironic with an operating > >> Third-Party CI, however soon after a series of unfortunate events > >> occurred where the Third-Party CI stopped reporting sometime in early > >> May, and shortly their after restrictions were put into place[2]. With > >> that being said, I feel like we should go ahead and remove this driver > >> since it does not seem to be maintained nor tested. > > > > > > Do you mean deprecate and remove in V or just straight remove? I'd > follow the regular procedure. Otherwise, it seems that we have no other > options :( > > Either... Kind of why I emailed the mailing list. :) I'd lean for > deprecated and take that route, but maybe someone has interest in > keeping it working > > > > > An interesting topic to discuss is how to avoid such situations when a > driver only lives 1-2 cycles.. > > Knowing a preference may exist for us to avoid the word support, I > feel like it was a preview that didn't pan out. I don't feel that fear > of this happening again should change any of our patterns moving > forward though, since that would be the exact opposite of what is > needed to encourage adoption. > We could make all new hardware types starting in some sort of "tech preview" state, so enabling them would require something like enable_tech_preview_hardware_types=True in ironic.conf. It would be similar to the staging area in Linux. The reason is my (completely ungrounded) assumption that if a driver has survived 2 cycles, it has more chances of surviving longer. > > We also have some drivers that do the minimal, some that are engaged, > and some that are kind of like they have a warp drive attached. I'd > love for us to merge the coffee driver as a POC interface to help > people understand mechanics, fwiw. Not that we can have a coffee > machine wired into CI. > "launched" "has_impulse" "has_warp_drive"? > We have ironic-staging-drivers specifically for this ;) Dmitry > > -Julia > > > > > Dmitry > > > >> > >> > >> -Julia > >> > >> > >> [1]: https://review.opendev.org/#/c/639288/ > >> [2]: > https://www.federalregister.gov/documents/2019/05/21/2019-10616/addition-of-entities-to-the-entity-list > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Fri Dec 13 15:40:02 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 13 Dec 2019 16:40:02 +0100 Subject: [neutron] Drivers meetings cancelled Message-ID: Hi neutrinos, As there is Christmas holidays season comming, we decided on today’s drivers meeting, we will cancel next meetings this year. We will meet as usual on January 10th. Happy holidays and see You all in January :) — Slawek Kaplonski Senior software engineer Red Hat From haleyb.dev at gmail.com Fri Dec 13 16:02:10 2019 From: haleyb.dev at gmail.com (Brian Haley) Date: Fri, 13 Dec 2019 11:02:10 -0500 Subject: Finding configuration values In-Reply-To: References: <15012b8d-31d3-f856-35de-eed96cb6bf55@gmail.com> Message-ID: <6b4f8ca5-4ac7-c686-2698-931505df42ac@gmail.com> On 12/12/19 6:09 PM, Albert Braden wrote: > This is the Rocky doc I was using: > > https://docs.openstack.org/neutron/rocky/configuration/neutron.html > > This is what it says about the rpc_workers setting: > > rpc_workers > > Type: integer > Default: 1 > Number of RPC worker processes for service. Just an FYI that the default value for rpc_workers has changed, if you look at the Stein or later config it's now: api_workers Type integer Default Number of separate API worker processes for service. If not specified, the default is equal to the number of CPUs available for best performance, capped by potential RAM usage. rpc_workers Type integer Default Number of RPC worker processes for service. If not specified, the default is equal to half the number of API workers. > Is this the correct place to add something like 'If your neutron-metadata-agent.log is full of "error: [Errno 32] Broken pipe" then increase this setting' ? > > I'd be happy to help update the docs but I'm not sure how to get started. Opening document bugs doesn't seem to result in any change; if I could figure out how to submit changes myself that might be more helpful. How can I register as a document change submitter? There is a doc in the neutron repo available via docs.o.o: https://docs.openstack.org/neutron/train/admin/config-wsgi.html The last section of that covers "Neutron Worker Processes", and might be helpful. If you think it would be useful to add a little more info on this failure please either create a patch (file is at doc/source/admin/config-wsgi.rst in the repo), or open a bug - sometimes these low-hanging fruit things are great for someone new to the project. -Brian > -----Original Message----- > From: Matt Riedemann > Sent: Thursday, December 12, 2019 2:38 PM > To: openstack-discuss at lists.openstack.org > Subject: Re: Finding configuration values > > On 12/12/2019 3:34 PM, Albert Braden wrote: >> This illustrates a problem I've seen in Openstack. Many of the >> configuration settings are undocumented, or the documentation is >> obscure. The only way for an openstack operator to know the correct >> setting is to be told by an openstack developer. > > Since Pike the docs in each project should be standardized and the > configuration options should be generated from code. So for example: > > > From mriedemos at gmail.com Fri Dec 13 16:03:47 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 13 Dec 2019 10:03:47 -0600 Subject: [openstacksdk][nova] distinguish image and boot from volume instances In-Reply-To: References: Message-ID: On 12/13/2019 1:48 AM, Blom, Merlin, NMU-OI wrote: > I wrote a *function for distinguishing an image instance from an boot > from volume instance* for billing purpose. > > But it does not work this way, it seems as if BfV Instances also have > the Image property set. > > *Is there an elegant solution other than analyzing the volume attachment > state?* > > This cloud be a problem, because an image instance cloud also have volumes. > > def getBootFromVolInstances(serverId): > >     '''Determines wether an instance is a boot frome volume instance''' > >     server = conn.compute.find_server(serverId, ignore_missing=True) > >     if server: > >         if server.image: > >             return False > >         else: > >             return True > >     return False > I don't know what client-side tooling you're using (looks like the openstacksdk?) but a volume-backed server will have an 'image' parameter in the response body but it should be an empty string. From the API reference [1]: "The UUID and links for the image for your server instance. The image object might be an empty string when you boot the server from a volume." [1] https://docs.openstack.org/api-ref/compute/?expanded=show-server-details-detail#show-server-details -- Thanks, Matt From mriedemos at gmail.com Fri Dec 13 16:08:59 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 13 Dec 2019 10:08:59 -0600 Subject: [openstacksdk][nova] distinguish image and boot from volume instances In-Reply-To: References: Message-ID: <34135243-da5f-bc9b-6355-6979c79a920a@gmail.com> On 12/13/2019 10:03 AM, Matt Riedemann wrote: > I don't know what client-side tooling you're using (looks like the > openstacksdk?) but a volume-backed server will have an 'image' parameter > in the response body but it should be an empty string. The problem is you're getting an image.v2.image.Image object in the SDK so it's not Falsey. I'm not sure how you translate the empty string from the server.image response body parameter to knowing the SDK Image object is essentially empty. -- Thanks, Matt From mnaser at vexxhost.com Fri Dec 13 16:12:05 2019 From: mnaser at vexxhost.com (Mohammed Naser) Date: Fri, 13 Dec 2019 11:12:05 -0500 Subject: [openstacksdk][nova] distinguish image and boot from volume instances In-Reply-To: References: Message-ID: On Fri, Dec 13, 2019 at 2:54 AM Blom, Merlin, NMU-OI wrote: > > Hey, > > I wrote a function for distinguishing an image instance from an boot from volume instance for billing purpose. > > But it does not work this way, it seems as if BfV Instances also have the Image property set. > > Is there an elegant solution other than analyzing the volume attachment state? > > This cloud be a problem, because an image instance cloud also have volumes. Except that would make it not a boot from volume instance, it would be an instances with attached volumes. > > > def getBootFromVolInstances(serverId): > > '''Determines wether an instance is a boot frome volume instance''' > > server = conn.compute.find_server(serverId, ignore_missing=True) > > if server: > > if server.image: > > return False > > else: > > return True > > return False > > > > Merlin Blom > > Cloud System Engineer > > E-Mail: merlin.blom at bertelsmann.de > > arvato-systems.de > > -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser at vexxhost.com W. https://vexxhost.com From gmann at ghanshyammann.com Fri Dec 13 16:19:38 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Fri, 13 Dec 2019 10:19:38 -0600 Subject: [tc] Follow up on action tems In-Reply-To: <4ca40b2e690eb5f6eb18d88782bb149d3b261b0e.camel@evrard.me> References: <4ca40b2e690eb5f6eb18d88782bb149d3b261b0e.camel@evrard.me> Message-ID: <16f000ef6d4.11b851afb38216.2242094661004159886@ghanshyammann.com> ---- On Fri, 13 Dec 2019 02:56:37 -0600 Jean-Philippe Evrard wrote ---- > Hello TC members, > > - There are still business opportunities merging for 2019. The process > for 2020 need to start soon. Any takers? > > - We have pending action items from the last meeting: > - ricolin: contact to Infra team to see if we got more ARM server > in our Nodepool > - ricolin/ttx: track the merger of the SIGs self-healing and auto- > scaling > - mnaser: static hosting story: Is it restarting? Where are we now? > - ttx: Work on technical vision reflection update > - ricolin/evrardjp: Expect/Request an update from Telemetry team. > As mentioned in our channel [7], Rong Zh (PTL for Telemetry) will > provide a Telemetry status report (high chance to send it out next > week). > - jungleboyj: analysis of the user survey. The patch is out for TCs > to review [6] > - ttx: make sure to start the discussion about uc/tc merge in > January after the ops meetup > - mnaser: write documentation in governance to clarify what was > discussed about stable policy > > - We have new action items: > - We need to update our charter for the election terms, to be more > flexible. Who is okay with starting this? I can work on this as discussed on ML and push something up for review today. -gmann > - We have plenty of things which are not reviewed, or WIP. Could you > update (y)our patches please with either changes or votes? > - aprice has asked me (JP) to write a report for the TC and SIG > activities of 2019. If you want to know what I am cooking, or want to > help on the text, please join here [5] > - In the next elections, we'll be reducing our (=TC) size temporarily > due to the current elections timing proposal. In order to not have this > happen again, we should propose a charter change as soon as possible. > Can someone tackle this? Please see the conversations here [1] and here > [2] > > - We still have long action items from PTG: > - For community goal `Drop Python 2.7 Support`, you can find the > status update on the ML [9]. React there if you need. > - The community goal `project specific PTL and contributor guides` > proposal patch is under review [8] and will be merged soon. > - gmann is sending an official goal schedule plan [3] . We need TC > members to review this as well, as soon as possible. We also need to > discuss when should we start the first step for next cycle in schedule > during next meeting (if/when patch landed). > - Please also review other patches, like [4] . > > And finally, some things to keep in mind: > - We need to promote the first contact sig to organisations that > might need help on getting things achieved in OpenStack. > > Regards, > JP & Rico > > [1] > http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011396.html > [2] > http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011155.html > [3] https://review.opendev.org/#/c/698299 > [4] https://review.opendev.org/#/c/691737 > [5] > https://etherpad.openstack.org/p/2019-openstack-foundation-annual-report-draft > [6] https://review.opendev.org/#/c/698582/ > [7] > http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-12-10.log.html#t2019-12-10T08:54:08 > [8] https://review.opendev.org/#/c/691737 > [9] > http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011432.html > > > From gagehugo at gmail.com Fri Dec 13 16:41:02 2019 From: gagehugo at gmail.com (Gage Hugo) Date: Fri, 13 Dec 2019 10:41:02 -0600 Subject: [Security SIG] Security SIG Newsletter - December 2019 Message-ID: Hope everyone is having a good December so far, here's the update for this month from the Security SIG. Starting this month, this newsletter will be released on a monthly basis rather than a semi-weekly due to the sporadic activity and my availability. Also just a heads up, the next two weeks of meetings will be canceled for the Security SIG, we will resume on Jan 02nd. Have a Happy Holidays everyone! #Month Dec 2019 - Security SIG Meeting Info: http://eavesdrop.openstack.org/#Security_SIG_meeting - Weekly on Thursday at 1500 UTC in #openstack-meeting - Agenda: https://etherpad.openstack.org/p/security-agenda - https://security.openstack.org/ - https://wiki.openstack.org/wiki/Security-SIG #Updates - https://bugs.launchpad.net/keystone/+bug/1855080 was reported and a corresponding OSSA was released - This newsletter to be moved to a monthly release, rather than semi-weekly. #VMT Reports - A full list of publicly marked security issues can be found here: https://bugs.launchpad.net/ossa/ - OSSA-2019-005 was released: https://security.openstack.org/ossa/OSSA-2019-006.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From zbitter at redhat.com Fri Dec 13 16:46:05 2019 From: zbitter at redhat.com (Zane Bitter) Date: Fri, 13 Dec 2019 11:46:05 -0500 Subject: [tc] Concluding the "2019 TC vision" exercise In-Reply-To: <0e9f934f-80e9-4204-1722-4fb963bec412@openstack.org> References: <0e9f934f-80e9-4204-1722-4fb963bec412@openstack.org> Message-ID: <0ebe18ee-b945-c959-ba8f-6fbed101940e@redhat.com> On 13/12/19 7:54 am, Thierry Carrez wrote: > this year the TC saw its first APAC-based member. We've had at least 4 APAC-based members that I can think of, beginning in March 2013. - ZB From balazs.gibizer at est.tech Sat Dec 14 11:25:49 2019 From: balazs.gibizer at est.tech (=?iso-8859-1?Q?Bal=E1zs_Gibizer?=) Date: Sat, 14 Dec 2019 11:25:49 +0000 Subject: [nova][ptg] Continue QoS port support In-Reply-To: <1571998629.31348.8@est.tech> References: <1571998629.31348.8@est.tech> Message-ID: <1576322746.781.0@est.tech> On Fri, Oct 25, 2019 at 10:17, Balázs Gibizer wrote: > Hi Novas! > > Here is my summarized plans for Ussuri about continuing the work on > supporting qos neutron ports (those that has bandwidth resource > request) in nova. As we are at milestone 1 I thought it would be good to summarize the progress so far. > > What is missing: > * Tempest coverage is missing for migrate and resize support that is > merged in Train. This work is already underway and bugs has been > caught > [1][2] These bugs are fixed now. But tempest patches [9][10] are still open and need reviews. > * Support evacuate, live migrate, unshelve. The work is described in > [3][4] and the first set of patches for the evacuation support is up > for review [5] The evacuate support has been merged before M1. I've just finished up the last pieces of the live migration support so that is complete now but code review needs to be continued. My next step (probably after the vacation period) is to look into the unshelve support. > * Support for cross cell resize with qos port needs some work. Matt > prepared the cross cell resize code already in a way that no new RPC > change will be needed [6] and I have a plan what to do [7]. My goal here is to have the cross cell resize support of qos ready and proposed before M2. > * InstancePCIRequest persists parent_ifname during migration but the > such change is not rolled back if the migration fails. This is ugly > but > I think it does not cause any issues [8]. I will look into this to > remove the ugliness. This is still open but so far did not cause any problem. > > The bandwidth support for the nova-manage heal_allocation tool was > merged in Train. Originally I planned to backport that to Stein but > that patch grown so big and incorporated may refactors along the way > that I'm not sure any more that it is reasonable to backport it. I'm > now thinking about keeping it as-is and suggesting operators to > install > Train nova in a virtualenv to run heal allocations for bandwidth aware > servers if needed in Stein. I do have to run some manual tests to see > if it actually works. As we agreed with Matt in October it is reasonable to document my proposal. I would like to run some manual test before I write that doc. In the meantime Eric made a really good progress to use the request group - resource provider mapping from placement instead of re-calculating it in nova. As [11] is merged we only have one single place where the re-calculation is still needed to be able to support revert resize. The next possible step here is transform the resize flow in nova to use the multiple neutron port binding and by that keep the mapping information in the inactive port binding on the source host. This task is on my TODO list but I'm not sure if it will fit the Ussuri timeline. Thanks everybody who helped making this nice progress! Cheers, gibi [9] https://review.opendev.org/#/c/690934 [10] https://review.opendev.org/#/c/694539 [11] https://review.opendev.org/#/c/696992 > > Any feedback is welcome! > > cheers, > gibi > > > [1] https://bugs.launchpad.net/nova/+bug/1849695 > [2] https://bugs.launchpad.net/nova/+bug/1849657 > [3] > https://blueprints.launchpad.net/nova/+spec/support-move-ops-with-qos-ports-ussuri > [4] > https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/support-move-ops-with-qos-ports-ussuri.html > [5] > https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/support-move-ops-with-qos-ports-ussuri > [6] > https://review.opendev.org/#/c/635080/43/nova/compute/manager.py at 5375 > [7] > https://review.opendev.org/#/c/633293/49/nova/compute/manager.py at 4742 > [8] > https://review.opendev.org/#/c/688387/6/nova/compute/manager.py at 3404 > > From gmann at ghanshyammann.com Sat Dec 14 16:29:39 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Sat, 14 Dec 2019 10:29:39 -0600 Subject: [goals][Drop Python 2.7 Support] Week R-22 Update Message-ID: <16f053e80fe.10f635d3855454.1090802515964741682@ghanshyammann.com> Hello Everyone, Below is the progress on "Drop Python 2.7 Support" for R-22 week. Schedule: https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html#schedule * I am adding this in Ussuri schedule page[1] Highlights: ======== * We crossed the Milestone-1 and with that, we are in phase-2 of py2 drop which targets all the common libraries and testing tools can start dropping the py2 support. * All the services should drop the py2 but this time. Patches are already up, review and merge them before gate starts failing due to cross projects py2 drop. * I pushed the patches on specs repo to cleanup py2 specific requirement or tox or zuul changes. many projects were running 'openstack-tox-py27' which I have changed to 'openstack-tox-pep8' instead of moving them to py3. pep8 job is enough for Specs repo. * I saw many spec repo failing their docs job due to oslosphinx and incompatible version of yasfb[2]. We need to fix this along with py2 drop patches if existing spec repo running py2 job. NOTE: Phase-2 is for Library (including client library), QA tooling. Phase-3 is mainly for openstack/requirement and audit to all repo. (we can always adjust the schedule to make it smooth migration. For example, dropping a few testing jobs from phase-2/3 candidates if any cross-testing jobs are failing due to phase-1 candidates dropping the support[3].) Summary: ======= The OpenStack services have not merged the py2 drop patches: * Adjutant * Barbican * Designate * ec2-api * Glance * Neutron (stadium projects) * Freezer * Kolla * Karbor * Congress * Storlets * Sahara * Blazar * Masakari * Tacker * Trove (trove-dashbaord patch is not mergd) * Zaqar * Qinling * Tricircle * openstack-helm * python-openstackclient How you can help: ============== - Review the patches. Push the patches if I missed any repo targetted for phase-1. [1] https://review.opendev.org/#/c/699008/ [2] https://review.opendev.org/#/c/698998/ [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011392.html [4] https://review.opendev.org/#/q/topic:drop-py27-support From mriedemos at gmail.com Sat Dec 14 21:41:18 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Sat, 14 Dec 2019 15:41:18 -0600 Subject: [nova][ptg] Continue QoS port support In-Reply-To: <1576322746.781.0@est.tech> References: <1571998629.31348.8@est.tech> <1576322746.781.0@est.tech> Message-ID: On 12/14/2019 5:25 AM, Balázs Gibizer wrote: > The next possible step here is transform the resize flow > in nova to use the multiple neutron port binding and by that keep the > mapping information in the inactive port binding on the source host. > This task is on my TODO list but I'm not sure if it will fit the Ussuri > timeline. Cross-cell resize uses multiple port bindings so maybe you can borrow from some of that code. -- Thanks, Matt From fungi at yuggoth.org Sun Dec 15 17:05:13 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Sun, 15 Dec 2019 17:05:13 +0000 Subject: [all] Volunteer mailing list administrators Message-ID: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> This is a long-overdue request. It's been more than a year since we combined openstack, openstack-dev, openstack-ops, openstack-sigs, openstack-tc and a number of other lower-traffic mailing lists into this new openstack-discuss mailing list. In that time, I've been serving as the sole administrator and moderator for the combined list. As I'm about to take an extended vacation away from regular Internet access for the first time since the list migration, I will be unable to check the moderation queue on a daily basis, and so would appreciate a bit of help from some of you. If there's anyone interested in helping out as additional administrators for the openstack-discuss mailing list, please reply to this message on-list (so other subscribers know who is volunteering) and I'll help bring you up to speed. In the immediate term I need at least one person willing to check the moderation queue a few times in the second half of December while I'm unavailable (daily would be awesome but a couple times a week is probably fine really). If you're expecting to be around and able to help with that, please respond on Monday or Tuesday of this week and I'll sync up to get you a copy of the Mailman administrator password for it and a quick walk-through of the moderator interface (if you already moderate other Mailman lists, all the better!). Moderation queue volume is typically on the order of twenty to fifty messages a day, most of which are spam of course, but one or two a day on average are legitimate messages either from non-subscribers with a question or sometimes slightly over the size limit because of attached logs. The spam and non-spam are pretty easy to tell apart if you've been around our community for a while: if the subject of the message has a recognizable topic tag in it or mentions the name of some OpenStack project then it's quite likely legitimate and can be approved. Spam on the other had usually either has a subject which is not in English (the required language for messages to this list) or is clearly someone selling something or running a scam. When in doubt though, you can quickly check the message body to be sure. I tend to spend at most 5 minutes a day looking through the moderation queue (and usually far less). Longer term, I'd like to see anywhere between 3-5 admins for the openstack-discuss list. No need to coordinate workload or vacations that way, we can just each check the queue when we think about it and that should be plenty often. Also while keeping on top of the moderation queue is the primary need, it would be great for these folks to know/learn a bit about how Mailman mailing lists are configured and how to troubleshoot them from the WebUI and message headers. Better still, if anyone wants to get even more involved, the OpenDev sysadmins would always welcome new recruits willing to help manage and run our mailing list servers and our infrastructure overall, I'm happy to make introductions! -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From radoslaw.piliszek at gmail.com Sun Dec 15 17:52:24 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Sun, 15 Dec 2019 18:52:24 +0100 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: Count me in, fungi. Let's start with the moderation and mailman. :-) -yoctozepto niedz., 15 gru 2019 o 18:09 Jeremy Stanley napisał(a): > > This is a long-overdue request. It's been more than a year since we > combined openstack, openstack-dev, openstack-ops, openstack-sigs, > openstack-tc and a number of other lower-traffic mailing lists into > this new openstack-discuss mailing list. In that time, I've been > serving as the sole administrator and moderator for the combined > list. As I'm about to take an extended vacation away from regular > Internet access for the first time since the list migration, I will > be unable to check the moderation queue on a daily basis, and so > would appreciate a bit of help from some of you. > > If there's anyone interested in helping out as additional > administrators for the openstack-discuss mailing list, please reply > to this message on-list (so other subscribers know who is > volunteering) and I'll help bring you up to speed. In the immediate > term I need at least one person willing to check the moderation > queue a few times in the second half of December while I'm > unavailable (daily would be awesome but a couple times a week is > probably fine really). If you're expecting to be around and able to > help with that, please respond on Monday or Tuesday of this week and > I'll sync up to get you a copy of the Mailman administrator password > for it and a quick walk-through of the moderator interface (if you > already moderate other Mailman lists, all the better!). > > Moderation queue volume is typically on the order of twenty to fifty > messages a day, most of which are spam of course, but one or two a > day on average are legitimate messages either from non-subscribers > with a question or sometimes slightly over the size limit because of > attached logs. The spam and non-spam are pretty easy to tell apart > if you've been around our community for a while: if the subject of > the message has a recognizable topic tag in it or mentions the name > of some OpenStack project then it's quite likely legitimate and can > be approved. Spam on the other had usually either has a subject > which is not in English (the required language for messages to this > list) or is clearly someone selling something or running a scam. > When in doubt though, you can quickly check the message body to be > sure. I tend to spend at most 5 minutes a day looking through the > moderation queue (and usually far less). > > Longer term, I'd like to see anywhere between 3-5 admins for the > openstack-discuss list. No need to coordinate workload or vacations > that way, we can just each check the queue when we think about it > and that should be plenty often. Also while keeping on top of the > moderation queue is the primary need, it would be great for these > folks to know/learn a bit about how Mailman mailing lists are > configured and how to troubleshoot them from the WebUI and message > headers. Better still, if anyone wants to get even more involved, > the OpenDev sysadmins would always welcome new recruits willing to > help manage and run our mailing list servers and our infrastructure > overall, I'm happy to make introductions! > -- > Jeremy Stanley From florian at datalounges.com Sun Dec 15 17:59:54 2019 From: florian at datalounges.com (Florian Rommel) Date: Sun, 15 Dec 2019 19:59:54 +0200 Subject: [all] Volunteer mailing list administrators In-Reply-To: References: Message-ID: Yep count me in too, if volunteers still needed. //Florian > On 15. Dec 2019, at 19.54, Radosław Piliszek wrote: > > Count me in, fungi. > > Let's start with the moderation and mailman. :-) > > -yoctozepto > > niedz., 15 gru 2019 o 18:09 Jeremy Stanley napisał(a): >> >> This is a long-overdue request. It's been more than a year since we >> combined openstack, openstack-dev, openstack-ops, openstack-sigs, >> openstack-tc and a number of other lower-traffic mailing lists into >> this new openstack-discuss mailing list. In that time, I've been >> serving as the sole administrator and moderator for the combined >> list. As I'm about to take an extended vacation away from regular >> Internet access for the first time since the list migration, I will >> be unable to check the moderation queue on a daily basis, and so >> would appreciate a bit of help from some of you. >> >> If there's anyone interested in helping out as additional >> administrators for the openstack-discuss mailing list, please reply >> to this message on-list (so other subscribers know who is >> volunteering) and I'll help bring you up to speed. In the immediate >> term I need at least one person willing to check the moderation >> queue a few times in the second half of December while I'm >> unavailable (daily would be awesome but a couple times a week is >> probably fine really). If you're expecting to be around and able to >> help with that, please respond on Monday or Tuesday of this week and >> I'll sync up to get you a copy of the Mailman administrator password >> for it and a quick walk-through of the moderator interface (if you >> already moderate other Mailman lists, all the better!). >> >> Moderation queue volume is typically on the order of twenty to fifty >> messages a day, most of which are spam of course, but one or two a >> day on average are legitimate messages either from non-subscribers >> with a question or sometimes slightly over the size limit because of >> attached logs. The spam and non-spam are pretty easy to tell apart >> if you've been around our community for a while: if the subject of >> the message has a recognizable topic tag in it or mentions the name >> of some OpenStack project then it's quite likely legitimate and can >> be approved. Spam on the other had usually either has a subject >> which is not in English (the required language for messages to this >> list) or is clearly someone selling something or running a scam. >> When in doubt though, you can quickly check the message body to be >> sure. I tend to spend at most 5 minutes a day looking through the >> moderation queue (and usually far less). >> >> Longer term, I'd like to see anywhere between 3-5 admins for the >> openstack-discuss list. No need to coordinate workload or vacations >> that way, we can just each check the queue when we think about it >> and that should be plenty often. Also while keeping on top of the >> moderation queue is the primary need, it would be great for these >> folks to know/learn a bit about how Mailman mailing lists are >> configured and how to troubleshoot them from the WebUI and message >> headers. Better still, if anyone wants to get even more involved, >> the OpenDev sysadmins would always welcome new recruits willing to >> help manage and run our mailing list servers and our infrastructure >> overall, I'm happy to make introductions! >> -- >> Jeremy Stanley > From skaplons at redhat.com Sun Dec 15 21:58:35 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Sun, 15 Dec 2019 22:58:35 +0100 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: Hi, I can help as well with this moderation so count me in too :) > On 15 Dec 2019, at 18:05, Jeremy Stanley wrote: > > This is a long-overdue request. It's been more than a year since we > combined openstack, openstack-dev, openstack-ops, openstack-sigs, > openstack-tc and a number of other lower-traffic mailing lists into > this new openstack-discuss mailing list. In that time, I've been > serving as the sole administrator and moderator for the combined > list. As I'm about to take an extended vacation away from regular > Internet access for the first time since the list migration, I will > be unable to check the moderation queue on a daily basis, and so > would appreciate a bit of help from some of you. > > If there's anyone interested in helping out as additional > administrators for the openstack-discuss mailing list, please reply > to this message on-list (so other subscribers know who is > volunteering) and I'll help bring you up to speed. In the immediate > term I need at least one person willing to check the moderation > queue a few times in the second half of December while I'm > unavailable (daily would be awesome but a couple times a week is > probably fine really). If you're expecting to be around and able to > help with that, please respond on Monday or Tuesday of this week and > I'll sync up to get you a copy of the Mailman administrator password > for it and a quick walk-through of the moderator interface (if you > already moderate other Mailman lists, all the better!). > > Moderation queue volume is typically on the order of twenty to fifty > messages a day, most of which are spam of course, but one or two a > day on average are legitimate messages either from non-subscribers > with a question or sometimes slightly over the size limit because of > attached logs. The spam and non-spam are pretty easy to tell apart > if you've been around our community for a while: if the subject of > the message has a recognizable topic tag in it or mentions the name > of some OpenStack project then it's quite likely legitimate and can > be approved. Spam on the other had usually either has a subject > which is not in English (the required language for messages to this > list) or is clearly someone selling something or running a scam. > When in doubt though, you can quickly check the message body to be > sure. I tend to spend at most 5 minutes a day looking through the > moderation queue (and usually far less). > > Longer term, I'd like to see anywhere between 3-5 admins for the > openstack-discuss list. No need to coordinate workload or vacations > that way, we can just each check the queue when we think about it > and that should be plenty often. Also while keeping on top of the > moderation queue is the primary need, it would be great for these > folks to know/learn a bit about how Mailman mailing lists are > configured and how to troubleshoot them from the WebUI and message > headers. Better still, if anyone wants to get even more involved, > the OpenDev sysadmins would always welcome new recruits willing to > help manage and run our mailing list servers and our infrastructure > overall, I'm happy to make introductions! > -- > Jeremy Stanley — Slawek Kaplonski Senior software engineer Red Hat From aaronzhu1121 at gmail.com Mon Dec 16 03:20:11 2019 From: aaronzhu1121 at gmail.com (Rong Zhu) Date: Mon, 16 Dec 2019 11:20:11 +0800 Subject: [Telemetry][TC]Telemetry status Message-ID: Hi, As discussed with Rico I send a mail to report the telemetry project status. We had already finished some work in Ussuri: 1.Support dynamic pollster feature Thanks Rafael Weingärtner hard working https://review.opendev.org/#/c/677031/ https://review.opendev.org/#/c/691659/ https://review.opendev.org/#/c/691944/ https://review.opendev.org/#/c/694345/ https://review.opendev.org/#/c/693088/ https://review.opendev.org/#/c/694519/ 2.Support aodh-evaluator built-in active/active deployment mode 3.Support threshold type alarm again Thanks, Lingxian https://review.opendev.org/#/c/697566/ https://review.opendev.org/#/c/695997/ Also, We plan to do some work in Ussuri: 1.Add Ceilometer API back Since Gnocchi is out of OpenStack and is unmaintained, we need to add Ceilometer API back again. 2.Mongodb database backend support As this thread [0] discuss, there are still some users still use mongodb as the database backend, we decide to add this support again. License issue should consider and fix by user. The OVH and Catalyst Cloud had fix the mongodb performance issue with some mongodb changes. Gnoochi will still support as the backed. Other database (like influxdb, ES....), we would happy everyone to submit patches to support as the database backed in ceilometer. 3.cpu_utils support This is mentioned many many times, this is a useful metric, so we decide support this again. 4.Support container meters with Zun 5. Acknowledgment of OpenStack-wide Goals I created a storyboard to track Ceilometer Ussuri release to do things in [1]. Free free to add things you want to do in Ussuri release. [0]: http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010922.html [1]: https://storyboard.openstack.org/#!/board/205 -- Thanks, Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From yamamoto at midokura.com Mon Dec 16 05:39:52 2019 From: yamamoto at midokura.com (Takashi Yamamoto) Date: Mon, 16 Dec 2019 14:39:52 +0900 Subject: [neutron] bug deputy report for the week of Dec 9th Message-ID: hi, here's the list of the bugs reported in the week. RFE: https://bugs.launchpad.net/neutron/+bug/1855123 Neutron ovs agent still fails vlan transparency check https://bugs.launchpad.net/neutron/+bug/1855854 [RFE] Dynamic DHCP allocation pool Critical: https://bugs.launchpad.net/neutron/+bug/1856156 Constraints file is not considered during doc build * backports probably need some investigations. https://review.opendev.org/#/q/Iea61238f37fdf24c0264f96d104ee0b3b6aec8e2 High: https://bugs.launchpad.net/neutron/+bug/1855759 neutron.plugins.ml2.drivers.agent._common_agent KeyError: 'gateway' . ERROR * a proposed patch: https://review.opendev.org/#/c/698287/ https://bugs.launchpad.net/neutron/+bug/1855888 ovs-offload with vxlan is broken due to adding skb mark * a proposed patch, blocked by gate: https://review.opendev.org/#/c/698336/ https://bugs.launchpad.net/neutron/+bug/1855912 MariaDB 10.1 fails during alembic migration https://bugs.launchpad.net/neutron/+bug/1856319 [SRIOV] Agent is reporting "resources_synced=True" when restarted and placement is down Low: https://bugs.launchpad.net/neutron/+bug/1855919 broken pipe errors cause neutron metadata agent to fail https://bugs.launchpad.net/neutron/+bug/1856152 Vm`s tap bound to br-int，When cfg.CONF.OVS.integration_bridge is configured as another bridge From chris.macnaughton at canonical.com Mon Dec 16 07:36:08 2019 From: chris.macnaughton at canonical.com (Chris MacNaughton) Date: Mon, 16 Dec 2019 08:36:08 +0100 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: I'm also happy to help! Chris On 12/15/19 6:05 PM, Jeremy Stanley wrote: > This is a long-overdue request. It's been more than a year since we > combined openstack, openstack-dev, openstack-ops, openstack-sigs, > openstack-tc and a number of other lower-traffic mailing lists into > this new openstack-discuss mailing list. In that time, I've been > serving as the sole administrator and moderator for the combined > list. As I'm about to take an extended vacation away from regular > Internet access for the first time since the list migration, I will > be unable to check the moderation queue on a daily basis, and so > would appreciate a bit of help from some of you. > > If there's anyone interested in helping out as additional > administrators for the openstack-discuss mailing list, please reply > to this message on-list (so other subscribers know who is > volunteering) and I'll help bring you up to speed. In the immediate > term I need at least one person willing to check the moderation > queue a few times in the second half of December while I'm > unavailable (daily would be awesome but a couple times a week is > probably fine really). If you're expecting to be around and able to > help with that, please respond on Monday or Tuesday of this week and > I'll sync up to get you a copy of the Mailman administrator password > for it and a quick walk-through of the moderator interface (if you > already moderate other Mailman lists, all the better!). > > Moderation queue volume is typically on the order of twenty to fifty > messages a day, most of which are spam of course, but one or two a > day on average are legitimate messages either from non-subscribers > with a question or sometimes slightly over the size limit because of > attached logs. The spam and non-spam are pretty easy to tell apart > if you've been around our community for a while: if the subject of > the message has a recognizable topic tag in it or mentions the name > of some OpenStack project then it's quite likely legitimate and can > be approved. Spam on the other had usually either has a subject > which is not in English (the required language for messages to this > list) or is clearly someone selling something or running a scam. > When in doubt though, you can quickly check the message body to be > sure. I tend to spend at most 5 minutes a day looking through the > moderation queue (and usually far less). > > Longer term, I'd like to see anywhere between 3-5 admins for the > openstack-discuss list. No need to coordinate workload or vacations > that way, we can just each check the queue when we think about it > and that should be plenty often. Also while keeping on top of the > moderation queue is the primary need, it would be great for these > folks to know/learn a bit about how Mailman mailing lists are > configured and how to troubleshoot them from the WebUI and message > headers. Better still, if anyone wants to get even more involved, > the OpenDev sysadmins would always welcome new recruits willing to > help manage and run our mailing list servers and our infrastructure > overall, I'm happy to make introductions! From gr at ham.ie Mon Dec 16 12:10:37 2019 From: gr at ham.ie (Graham Hayes) Date: Mon, 16 Dec 2019 12:10:37 +0000 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: On 15/12/2019 17:05, Jeremy Stanley wrote: > This is a long-overdue request. It's been more than a year since we > combined openstack, openstack-dev, openstack-ops, openstack-sigs, > openstack-tc and a number of other lower-traffic mailing lists into > this new openstack-discuss mailing list. In that time, I've been > serving as the sole administrator and moderator for the combined > list. As I'm about to take an extended vacation away from regular > Internet access for the first time since the list migration, I will > be unable to check the moderation queue on a daily basis, and so > would appreciate a bit of help from some of you. > > If there's anyone interested in helping out as additional > administrators for the openstack-discuss mailing list, please reply > to this message on-list (so other subscribers know who is > volunteering) and I'll help bring you up to speed. In the immediate > term I need at least one person willing to check the moderation > queue a few times in the second half of December while I'm > unavailable (daily would be awesome but a couple times a week is > probably fine really). If you're expecting to be around and able to > help with that, please respond on Monday or Tuesday of this week and > I'll sync up to get you a copy of the Mailman administrator password > for it and a quick walk-through of the moderator interface (if you > already moderate other Mailman lists, all the better!). > > Moderation queue volume is typically on the order of twenty to fifty > messages a day, most of which are spam of course, but one or two a > day on average are legitimate messages either from non-subscribers > with a question or sometimes slightly over the size limit because of > attached logs. The spam and non-spam are pretty easy to tell apart > if you've been around our community for a while: if the subject of > the message has a recognizable topic tag in it or mentions the name > of some OpenStack project then it's quite likely legitimate and can > be approved. Spam on the other had usually either has a subject > which is not in English (the required language for messages to this > list) or is clearly someone selling something or running a scam. > When in doubt though, you can quickly check the message body to be > sure. I tend to spend at most 5 minutes a day looking through the > moderation queue (and usually far less). > > Longer term, I'd like to see anywhere between 3-5 admins for the > openstack-discuss list. No need to coordinate workload or vacations > that way, we can just each check the queue when we think about it > and that should be plenty often. Also while keeping on top of the > moderation queue is the primary need, it would be great for these > folks to know/learn a bit about how Mailman mailing lists are > configured and how to troubleshoot them from the WebUI and message > headers. Better still, if anyone wants to get even more involved, > the OpenDev sysadmins would always welcome new recruits willing to > help manage and run our mailing list servers and our infrastructure > overall, I'm happy to make introductions! > Definitely something I would be able to add to my morning routine - if we still need people sign me up :) - Graham From gr at ham.ie Mon Dec 16 12:15:58 2019 From: gr at ham.ie (Graham Hayes) Date: Mon, 16 Dec 2019 12:15:58 +0000 Subject: [Foundation Board] [OpenStack Foundation] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign In-Reply-To: References: <057BD0AC-451C-4E64-9D30-6CC6A20DFA0F@demarco.com> <1595EA09-2B8C-45ED-B836-5751FC79DBA1@openstack.org> <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> Message-ID: On 10/12/2019 16:01, Sean McGinnis wrote: > Thanks Allison. And thanks Graham for raising this in the community. > > On Tue, Dec 10, 2019, 10:53 Allison Randal > wrote: > > A few more news posts about the topic: > > https://www.theregister.co.uk/2019/11/20/org_registry_sale_shambles/ > > > https://gizmodo.com/private-equity-ghouls-buy-non-profit-that-handles-org-1839860118 > > > http://blogs.harvard.edu/sj/2019/11/23/a-tale-of-icann-and-regulatory-capture-the-dot-org-heist/ > > > https://arstechnica.com/tech-policy/2019/11/private-equity-firm-buys-org-domain-months-after-icann-lifted-price-caps/ > > > https://medium.com/@jacobmalthouse/in-which-i-explain-why-all-of-andrew-sullivans-reasons-for-selling-org-are-wrong-449997d42cac > > > https://www.bbc.com/news/technology-50515786 > > > HTH, > Allison It seems the pressure on ICANN has had an effect: https://www.icann.org/news/blog/org-update From sean.mcginnis at gmail.com Mon Dec 16 13:39:20 2019 From: sean.mcginnis at gmail.com (Sean McGinnis) Date: Mon, 16 Dec 2019 07:39:20 -0600 Subject: [OpenStack Foundation] [Foundation Board] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign In-Reply-To: References: <057BD0AC-451C-4E64-9D30-6CC6A20DFA0F@demarco.com> <1595EA09-2B8C-45ED-B836-5751FC79DBA1@openstack.org> <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> Message-ID: > > > It seems the pressure on ICANN has had an effect: > > https://www.icann.org/news/blog/org-update > > Great to see something happening. Thanks for the update Graham! From johnsomor at gmail.com Mon Dec 16 14:18:46 2019 From: johnsomor at gmail.com (Michael Johnson) Date: Mon, 16 Dec 2019 06:18:46 -0800 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: Hi Jeremy, I can also help out. Michael On Sun, Dec 15, 2019 at 9:09 AM Jeremy Stanley wrote: > > This is a long-overdue request. It's been more than a year since we > combined openstack, openstack-dev, openstack-ops, openstack-sigs, > openstack-tc and a number of other lower-traffic mailing lists into > this new openstack-discuss mailing list. In that time, I've been > serving as the sole administrator and moderator for the combined > list. As I'm about to take an extended vacation away from regular > Internet access for the first time since the list migration, I will > be unable to check the moderation queue on a daily basis, and so > would appreciate a bit of help from some of you. > > If there's anyone interested in helping out as additional > administrators for the openstack-discuss mailing list, please reply > to this message on-list (so other subscribers know who is > volunteering) and I'll help bring you up to speed. In the immediate > term I need at least one person willing to check the moderation > queue a few times in the second half of December while I'm > unavailable (daily would be awesome but a couple times a week is > probably fine really). If you're expecting to be around and able to > help with that, please respond on Monday or Tuesday of this week and > I'll sync up to get you a copy of the Mailman administrator password > for it and a quick walk-through of the moderator interface (if you > already moderate other Mailman lists, all the better!). > > Moderation queue volume is typically on the order of twenty to fifty > messages a day, most of which are spam of course, but one or two a > day on average are legitimate messages either from non-subscribers > with a question or sometimes slightly over the size limit because of > attached logs. The spam and non-spam are pretty easy to tell apart > if you've been around our community for a while: if the subject of > the message has a recognizable topic tag in it or mentions the name > of some OpenStack project then it's quite likely legitimate and can > be approved. Spam on the other had usually either has a subject > which is not in English (the required language for messages to this > list) or is clearly someone selling something or running a scam. > When in doubt though, you can quickly check the message body to be > sure. I tend to spend at most 5 minutes a day looking through the > moderation queue (and usually far less). > > Longer term, I'd like to see anywhere between 3-5 admins for the > openstack-discuss list. No need to coordinate workload or vacations > that way, we can just each check the queue when we think about it > and that should be plenty often. Also while keeping on top of the > moderation queue is the primary need, it would be great for these > folks to know/learn a bit about how Mailman mailing lists are > configured and how to troubleshoot them from the WebUI and message > headers. Better still, if anyone wants to get even more involved, > the OpenDev sysadmins would always welcome new recruits willing to > help manage and run our mailing list servers and our infrastructure > overall, I'm happy to make introductions! > -- > Jeremy Stanley From sean.mcginnis at gmx.com Mon Dec 16 15:13:07 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 16 Dec 2019 09:13:07 -0600 Subject: [all] Volunteer mailing list administrators In-Reply-To: References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: <20191216151307.GA1176874@sm-workstation> > > > > Longer term, I'd like to see anywhere between 3-5 admins for the > > openstack-discuss list. No need to coordinate workload or vacations > > that way, we can just each check the queue when we think about it > > and that should be plenty often. Also while keeping on top of the > > moderation queue is the primary need, it would be great for these > > folks to know/learn a bit about how Mailman mailing lists are > > configured and how to troubleshoot them from the WebUI and message > > headers. Better still, if anyone wants to get even more involved, > > the OpenDev sysadmins would always welcome new recruits willing to > > help manage and run our mailing list servers and our infrastructure > > overall, I'm happy to make introductions! > > > > > Definitely something I would be able to add to my morning routine - if > we still need people sign me up :) > > - Graham > Same for me. Sean From neil at tigera.io Mon Dec 16 15:42:29 2019 From: neil at tigera.io (Neil Jerram) Date: Mon, 16 Dec 2019 15:42:29 +0000 Subject: [all] Which will be the first OpenStack release to require Python 3 ? Message-ID: I think the answer is Ussuri (and hence that Train can still be run with Python 2), but have not been able to find a clear statement of this, hence asking here. (Suggest also updating https://wiki.openstack.org/wiki/Python3 with the answer.) Thanks, Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: From gr at ham.ie Mon Dec 16 15:51:34 2019 From: gr at ham.ie (Graham Hayes) Date: Mon, 16 Dec 2019 15:51:34 +0000 Subject: [all] Which will be the first OpenStack release to require Python 3 ? In-Reply-To: References: Message-ID: <72a413f6-07a6-5266-47b5-8e89ac9ac38c@ham.ie> On 16/12/2019 15:42, Neil Jerram wrote: > I think the answer is Ussuri (and hence that Train can still be run with > Python 2), but have not been able to find a clear statement of this, > hence asking here. > > (Suggest also updating https://wiki.openstack.org/wiki/Python3 > with the answer.) > > Thanks, >    Neil > Yeap, we should update the wiki page, I will do that today. We have a resolution [1] and the Project Testing Interface pages that indicate what we test for each release [2] that indicates that from Ussuri projects are not required to test python2, but I know at least Swift will continue to support python2.7 post Ussuri. - Graham 1 - https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html 2- https://governance.openstack.org/tc/reference/runtimes/ussuri.html#python-runtime-for-ussuri From neil at tigera.io Mon Dec 16 16:10:01 2019 From: neil at tigera.io (Neil Jerram) Date: Mon, 16 Dec 2019 16:10:01 +0000 Subject: [all] Which will be the first OpenStack release to require Python 3 ? In-Reply-To: <72a413f6-07a6-5266-47b5-8e89ac9ac38c@ham.ie> References: <72a413f6-07a6-5266-47b5-8e89ac9ac38c@ham.ie> Message-ID: On Mon, Dec 16, 2019 at 3:52 PM Graham Hayes wrote: > On 16/12/2019 15:42, Neil Jerram wrote: > > I think the answer is Ussuri (and hence that Train can still be run with > > Python 2), but have not been able to find a clear statement of this, > > hence asking here. > > > > (Suggest also updating https://wiki.openstack.org/wiki/Python3 > > with the answer.) > > > > Thanks, > > Neil > > > > Yeap, we should update the wiki page, I will do that today. > Thanks Graham. > We have a resolution [1] and the Project Testing Interface pages that > indicate what we test for each release [2] that indicates that from > Ussuri projects are not required to test python2, but I know at least > Swift will continue to support python2.7 post Ussuri. > Neutron master definitely requires Python 3 now. (But yes, I'm aware that there are Swift-only use cases, so it can be useful for Swift to be different.) > > - Graham > > 1 - > > https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html > 2- > > https://governance.openstack.org/tc/reference/runtimes/ussuri.html#python-runtime-for-ussuri > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sshnaidm at redhat.com Mon Dec 16 16:46:41 2019 From: sshnaidm at redhat.com (Sagi Shnaidman) Date: Mon, 16 Dec 2019 18:46:41 +0200 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - how to move the current modules? In-Reply-To: References: Message-ID: Hi, all recently we had an Openstack Ansible modules meeting, when were discussing how we will manage current modules and new planned modules. Because it was a hot topic which took most of the time and no agreement was reached, I think it's worth to discuss it here in ML. Options that have been raised in the discussion[1]: 1) To freeze current modules and start writing new modules from scratch 2) To freeze current modules and based on them develop new modules 3) To continue to work on current modules and change them step by step 4) In case of freezing current modules, deprecate them later Things to consider: 1) People are using current modules in playbooks and we don't want to break them, so current modules interfaces should stay available and not change for not breaking backward compatibility 2) We might redesign some of modules, including big changes in common modules parts 3) We might redistribute current module functionality over other and new modules I think it can be a start for the discussion on how to move further, please comment. Thanks [1] http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.00.log.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From colleen at gazlene.net Mon Dec 16 17:05:19 2019 From: colleen at gazlene.net (Colleen Murphy) Date: Mon, 16 Dec 2019 09:05:19 -0800 Subject: [keystone] Meeting time change In-Reply-To: <5308b221-3ba1-4598-9bab-f177fb95a7d9@www.fastmail.com> References: <5308b221-3ba1-4598-9bab-f177fb95a7d9@www.fastmail.com> Message-ID: <5dfbfa69-b74c-4bc2-9807-7889153b3353@www.fastmail.com> On Tue, Dec 10, 2019, at 09:03, Colleen Murphy wrote: > As discussed in today's meeting[1], I would like to propose we move the > team meeting to one hour later, 17:00 UTC, same day same channel. > Everyone in attendance today was okay with the change but I wanted to > give anyone who couldn't attend today the chance to object. Let me know > if this change would prevent you from attending the meeting. If there > are no objections this week, I'll propose a change to the eavesdrop > calendar. For the moment, assume the meeting next week (December 17) > will be at 17:00. > > Colleen > > [1] > http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-12-10-16.00.log.html#l-18 > > Now proposed: https://review.opendev.org/699246 From mark at stackhpc.com Mon Dec 16 17:27:15 2019 From: mark at stackhpc.com (Mark Goddard) Date: Mon, 16 Dec 2019 17:27:15 +0000 Subject: [kolla] No more meetings in 2019 Message-ID: Hi, We'll cancel the next 3 weeks meetings, and meet again on the 8th January. I'll be away from tomorrow until 2nd January. Cheers, Mark From cboylan at sapwetik.org Mon Dec 16 18:20:02 2019 From: cboylan at sapwetik.org (Clark Boylan) Date: Mon, 16 Dec 2019 10:20:02 -0800 Subject: =?UTF-8?Q?[Horizon][Designate][Tacker][Tobiko][OSH][Ironic]_Flaky_jobs_a?= =?UTF-8?Q?nd_retries?= Message-ID: <85cde474-cc15-420a-b228-e7191ad8e494@www.fastmail.com> Hello, Zuulv3 has a feature where it will rerun jobs that it thinks failed due to infrastructure problems up to some configured limit. For us the limit is 3 job attempts. The circumstances where Zuul thinks it has infrastructure issues are when pre-run playbooks fail (because these are expected to be stable), and when Ansible returns with an exit code signifying network connectivity issues. Recently I updated Zuul to report the job attempt in job inventories and am using that to report this data into logstash and elasticsearch. Now I can use queries like: 'zuul_attempts:"3" AND filename:"job-output.txt" AND message:"Job console starting"' to see which jobs are flaky and reattempting. I'm writing this email to report on some of what I've found in the hopes that the problems we have can be fixed and hopefully result in better jobs across the board. Designate Horizon Dashboard The nodejs jobs for stable/rocky expect to run against nodejs4 but run on Bionic which has no nodejs 4 packaging. This causes the pre-run to fail as it cannot install the requested package, https://zuul.opendev.org/t/openstack/build/498ebb052e2b4a3393b0939820ee8927/log/job-output.txt#391. Fix in progress here, https://review.opendev.org/#/c/699241/1, to pin to Xenial. Thank you! Horizon projects and dashboards may want to double check they don't have similar problems with nodejs version and node mismatches. Tacker tacker-functional-devstack-multinode-python3 (and possibly other tacker devstack jobs) attempt to download an openwrt image during devstack runs and the server isn't responding. This fails in pre-run, http://zuul.openstack.org/build/27ff514c77724250968d60469923f613/log/controller/logs/devstacklog.txt.gz#45426, and results in the job being retried. I am not aware of a fix for this, but you'll likely need to find a new image host. Let the infra team know if we can help. Tobiko tobiko-devstack-faults-centos-7 is failing because this job runs on CentOS 7 using python2 but Nova needs python3.6 now. This fails in pre-run, https://zuul.opendev.org/t/openstack/build/8071780e7ba748169b447f1d42e069fc/log/controller/logs/devstacklog.txt.gz#11799, and forces the job to be rerun. I'm not sure what the best fix is here. Maybe run your jobs on Ubuntu until CentOS 8 is supported by devstack? OpenStack Helm Ansible has decided that become_user is not valid on include_role tasks. https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/deploy-docker/tasks/deploy-ansible-docker-support.yaml#L26-L49 seems to be the issue. This causes Airship's deckhand functional docker jobs to fail in pre-run, https://zuul.opendev.org/t/openstack/build/7493038ee1744465b2387b44e067d029/log/job-output.txt#396, and the jobs are retried. It is possible this is fallout from Zuul's default ansible version being bumped to 2.8 from 2.7. This also seems to affect OSH's kubernetes keystone auth job, https://zuul.opendev.org/t/openstack/build/68d937b3e5e3449cb6fe2e6947bbf0db/log/job-output.txt#397 Ironic The Ironic example is a bit different and runs into some fun Ansible behaviors. Ironic's IPA multinode jobs (possibly others too) are filling the root disk of the test nodes on some clouds, http://paste.openstack.org/show/787641/. Ansible uses /tmp to bootstrap its ssh connectivity and when /tmp is on / and / is full that results in Ansible returning a network failure error code. This then causes Zuul to rerun the job. Unfortunately because Ansible thinks networking is broken we have to do special things in a Zuul cleanup playbook to double check disk usage and that info is only available on the Zuul executors. But I've double checked for you and can confirm this is what is happening. The fix here is to use the ephemeral disk which is mounted on /opt and contains much more disk space. Another option would be to reduce the size of Ironic's fake baremetal images. In any case they are aware of this and we should expect a fix soon. There are likely more cases I haven't found yet, but I wanted to point out this Zuul behavior to people. The intent is that it handle exceptional cases and not paper over failures that happen often or even 100% of the time. We should do our best to fix these problems when they pop up. Thank you to everyone that has already stepped up to help fix some of these! Clark From gmann at ghanshyammann.com Mon Dec 16 18:59:50 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 16 Dec 2019 12:59:50 -0600 Subject: [tc] Follow up on action tems In-Reply-To: <16f000ef6d4.11b851afb38216.2242094661004159886@ghanshyammann.com> References: <4ca40b2e690eb5f6eb18d88782bb149d3b261b0e.camel@evrard.me> <16f000ef6d4.11b851afb38216.2242094661004159886@ghanshyammann.com> Message-ID: <16f1014b621.da2ced54104177.1926872061448502473@ghanshyammann.com> ---- On Fri, 13 Dec 2019 10:19:38 -0600 Ghanshyam Mann wrote ---- > ---- On Fri, 13 Dec 2019 02:56:37 -0600 Jean-Philippe Evrard wrote ---- > > Hello TC members, > > > > - There are still business opportunities merging for 2019. The process > > for 2020 need to start soon. Any takers? > > > > - We have pending action items from the last meeting: > > - ricolin: contact to Infra team to see if we got more ARM server > > in our Nodepool > > - ricolin/ttx: track the merger of the SIGs self-healing and auto- > > scaling > > - mnaser: static hosting story: Is it restarting? Where are we now? > > - ttx: Work on technical vision reflection update > > - ricolin/evrardjp: Expect/Request an update from Telemetry team. > > As mentioned in our channel [7], Rong Zh (PTL for Telemetry) will > > provide a Telemetry status report (high chance to send it out next > > week). > > - jungleboyj: analysis of the user survey. The patch is out for TCs > > to review [6] > > - ttx: make sure to start the discussion about uc/tc merge in > > January after the ops meetup > > - mnaser: write documentation in governance to clarify what was > > discussed about stable policy > > > > - We have new action items: > > - We need to update our charter for the election terms, to be more > > flexible. Who is okay with starting this? > > I can work on this as discussed on ML and push something up for review today. It's up for review now- https://review.opendev.org/#/c/699277/1 -gmann > > -gmann > > > - We have plenty of things which are not reviewed, or WIP. Could you > > update (y)our patches please with either changes or votes? > > - aprice has asked me (JP) to write a report for the TC and SIG > > activities of 2019. If you want to know what I am cooking, or want to > > help on the text, please join here [5] > > - In the next elections, we'll be reducing our (=TC) size temporarily > > due to the current elections timing proposal. In order to not have this > > happen again, we should propose a charter change as soon as possible. > > Can someone tackle this? Please see the conversations here [1] and here > > [2] > > > > - We still have long action items from PTG: > > - For community goal `Drop Python 2.7 Support`, you can find the > > status update on the ML [9]. React there if you need. > > - The community goal `project specific PTL and contributor guides` > > proposal patch is under review [8] and will be merged soon. > > - gmann is sending an official goal schedule plan [3] . We need TC > > members to review this as well, as soon as possible. We also need to > > discuss when should we start the first step for next cycle in schedule > > during next meeting (if/when patch landed). > > - Please also review other patches, like [4] . > > > > And finally, some things to keep in mind: > > - We need to promote the first contact sig to organisations that > > might need help on getting things achieved in OpenStack. > > > > Regards, > > JP & Rico > > > > [1] > > http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011396.html > > [2] > > http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011155.html > > [3] https://review.opendev.org/#/c/698299 > > [4] https://review.opendev.org/#/c/691737 > > [5] > > https://etherpad.openstack.org/p/2019-openstack-foundation-annual-report-draft > > [6] https://review.opendev.org/#/c/698582/ > > [7] > > http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2019-12-10.log.html#t2019-12-10T08:54:08 > > [8] https://review.opendev.org/#/c/691737 > > [9] > > http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011432.html > > > > > > > > > From openstack at nemebean.com Mon Dec 16 19:55:52 2019 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 16 Dec 2019 13:55:52 -0600 Subject: [goals][Drop Python 2.7 Support] Week R-22 Update In-Reply-To: <16f053e80fe.10f635d3855454.1090802515964741682@ghanshyammann.com> References: <16f053e80fe.10f635d3855454.1090802515964741682@ghanshyammann.com> Message-ID: <320763f1-07e7-ca68-bdcd-3650e4ac0877@nemebean.com> On 12/14/19 10:29 AM, Ghanshyam Mann wrote: > * I pushed the patches on specs repo to cleanup py2 specific requirement or tox or zuul changes. > many projects were running 'openstack-tox-py27' which I have changed to 'openstack-tox-pep8' > instead of moving them to py3. pep8 job is enough for Specs repo. Are you sure? I believe some specs repos are using unit tests to validate that all of the sections of the spec are included (among other things)[0]. I don't think the pep8 job will cover those. 0: https://opendev.org/openstack/nova-specs/src/branch/master/tests/test_titles.py From openstack at nemebean.com Mon Dec 16 20:05:13 2019 From: openstack at nemebean.com (Ben Nemec) Date: Mon, 16 Dec 2019 14:05:13 -0600 Subject: [Foundation Board] [OpenStack Foundation] [tc][board][all] - Adding OpenStack community support to the savedotorg campaign In-Reply-To: References: <057BD0AC-451C-4E64-9D30-6CC6A20DFA0F@demarco.com> <1595EA09-2B8C-45ED-B836-5751FC79DBA1@openstack.org> <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> Message-ID: <566048ae-110e-a8b6-6bb0-50dbd65bed9a@nemebean.com> On 12/16/19 6:15 AM, Graham Hayes wrote: > On 10/12/2019 16:01, Sean McGinnis wrote: >> Thanks Allison. And thanks Graham for raising this in the community. >> >> On Tue, Dec 10, 2019, 10:53 Allison Randal > > wrote: >> >>     A few more news posts about the topic: >> >>     https://www.theregister.co.uk/2019/11/20/org_registry_sale_shambles/ >> >> >> >> >> https://gizmodo.com/private-equity-ghouls-buy-non-profit-that-handles-org-1839860118 >> >> >> >> >> >> >> http://blogs.harvard.edu/sj/2019/11/23/a-tale-of-icann-and-regulatory-capture-the-dot-org-heist/ >> >> >> >> >> >> >> https://arstechnica.com/tech-policy/2019/11/private-equity-firm-buys-org-domain-months-after-icann-lifted-price-caps/ >> >> >> >> >> >> >> https://medium.com/@jacobmalthouse/in-which-i-explain-why-all-of-andrew-sullivans-reasons-for-selling-org-are-wrong-449997d42cac >> >> >> >> >> >>     https://www.bbc.com/news/technology-50515786 >>     >> >>     HTH, >>     Allison > > > > It seems the pressure on ICANN has had an effect: > > https://www.icann.org/news/blog/org-update > It's progress anyway. Hopefully this additional inquiry will result in some real changes. Thanks for getting the word out, Graham. I might have missed this completely otherwise. From gmann at ghanshyammann.com Tue Dec 17 02:02:44 2019 From: gmann at ghanshyammann.com (Ghanshyam Mann) Date: Mon, 16 Dec 2019 20:02:44 -0600 Subject: [goals][Drop Python 2.7 Support] Week R-22 Update In-Reply-To: <320763f1-07e7-ca68-bdcd-3650e4ac0877@nemebean.com> References: <16f053e80fe.10f635d3855454.1090802515964741682@ghanshyammann.com> <320763f1-07e7-ca68-bdcd-3650e4ac0877@nemebean.com> Message-ID: <16f1197e3d8.c8fdb8ca110485.6612707106783067998@ghanshyammann.com> ---- On Mon, 16 Dec 2019 13:55:52 -0600 Ben Nemec wrote ---- > > > On 12/14/19 10:29 AM, Ghanshyam Mann wrote: > > * I pushed the patches on specs repo to cleanup py2 specific requirement or tox or zuul changes. > > many projects were running 'openstack-tox-py27' which I have changed to 'openstack-tox-pep8' > > instead of moving them to py3. pep8 job is enough for Specs repo. > > Are you sure? I believe some specs repos are using unit tests to > validate that all of the sections of the spec are included (among other > things)[0]. I don't think the pep8 job will cover those. pep8 job keep running the unit tests for many of them via explicit pep8 tox to run the unit tests[1] or not defining the env itself. I can move them to py3 job to be explicitly readable. But for few spec repo like keystone-specs pep8 would not cover the unit tests as pep8 is defined there as pep8 checks only. I will move py2 jobs to py3 job there. Thanks for monitor those. [1] https://opendev.org/openstack/nova-specs/src/commit/36b1b7fbf52360c5e7e335c78a3a8e0da8d2cf38/tox.ini -gmann > > 0: > https://opendev.org/openstack/nova-specs/src/branch/master/tests/test_titles.py > > From zbitter at redhat.com Tue Dec 17 03:01:45 2019 From: zbitter at redhat.com (Zane Bitter) Date: Mon, 16 Dec 2019 22:01:45 -0500 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: Message-ID: On 15/12/19 10:20 pm, Rong Zhu wrote: > 1.Add Ceilometer API back >     Since Gnocchi is out of OpenStack and is unmaintained, we need to > add Ceilometer API back again. This is concerning because even the people who wrote it don't consider it adequate to the job. That inadequacy has been the source of significant reputational damage to OpenStack in the past, and many folks (me included) are anxious to avoid a repeat. Telemetry is part of the TC "Approved Release" that is eligible for the trademark program; I think at a minimum the TC will want to remove the resurrected Ceilometer API from the "designated sections" that users are required to run to participate in any trademark program that includes the functionality in question. But I think that we should explore other ways of reducing the chance of anyone confusing this for a viable way of building a cloud, including possibly changing the name (Antediluvian API?) and having this part of the stack live outside of the official OpenStack project. I'm not clear on which parts of the API you plan to resurrect, but for example - wearing my Heat hat - after a long process of migrating Ceilometer alarm resources in Heat to Aodh ones (mostly involving breaking changes forced on us by the absence of a migration path in Telemetry), I have zero interest in turning around and going back in the other direction. I would infinitely prefer that we find a path to e.g. migrate to Monasca as the collector so that the community can speak with one voice about how to do metrics and alarming in OpenStack. > 2.Mongodb database backend support >   As this thread [0] discuss, there are still some users still use > mongodb as the database backend, we decide to add this support again. > License issue should consider and fix by user. So long as this is not the only backend available, I agree the licensing of MongoDB itself is not a concern for OpenStack (we have plenty of drivers that depend on even proprietary software in various places). The issue that Matthias alluded to in the thread is that Linux distributions are dropping MongoDB due to its non-free license, so it is not nearly as widely and easily available as it was in the past. > The OVH and Catalyst > Cloud had fix the mongodb performance issue with some mongodb changes. It's really hard to reconcile this with the information here (and especially in the linked slide): https://github.com/gnocchixyz/gnocchi/issues/1049#issuecomment-555768072 > Gnoochi will still support as the backed. Other database (like influxdb, > ES....), we would happy everyone to submit patches to support as the > database backed in ceilometer. This is good to hear :) cheers, Zane. From haleyb.dev at gmail.com Tue Dec 17 03:32:11 2019 From: haleyb.dev at gmail.com (Brian Haley) Date: Mon, 16 Dec 2019 22:32:11 -0500 Subject: [neutron][devstack] Future of lib/neutron and lib/neutron-legacy In-Reply-To: <48325993-0233-437B-ABC5-D038F0785F04@redhat.com> References: <48325993-0233-437B-ABC5-D038F0785F04@redhat.com> Message-ID: <755fb9bf-799f-1164-1184-96b146222235@gmail.com> Hi Slawek, On 12/12/19 9:06 AM, Slawek Kaplonski wrote: > Hi, > > Few years ago (I don’t know exactly when it started as I wasn’t very active upstream than for sure) Neutron team started work on new neutron module for Devstack. See [1] for details. > But since then this new module is still not ready to use. For example in gate jobs we are still using lib/neutron-legacy module as this new one is always causing some problems. > Things in its current shape may be confusing for new Devstack users as in fact to deploy Neutron properly they should use lib/neutron-legacy instead of lib/neutron now. > Personally I never really checked exactly what issues are there in fact but I do remember some issues with using lib/neutron e.g. on subnodes in multinode jobs - see [2]. > > So I would like to ask a question what do You think we should do with it. > Is there maybe any volunteer who would like to continue this work on adopting lib/neutron and to finally use it in gate jobs? > Or if there is no anyone and we are all fine with using old module, maybe we should remove this lib/neutron and “undeprecate” and rename lib/neutron-legacy? > > [1] https://github.com/openstack/devstack/commit/2a242519f71e86416e78541826cac2b54fcd04a5 > [2] https://review.opendev.org/#/c/662480/ Thanks for looking at this again. My initial thought is to complete the work as it's so close, this topic has the remaining abandoned reviews, https://review.opendev.org/#/q/topic:new-neutron-devstack-in-gate I wish I remembered more about why we never finished, I only remember bgpvpn needed some work, just don't remember much else. Could definitely do some of the reviews. -Brian From iwienand at redhat.com Tue Dec 17 04:35:12 2019 From: iwienand at redhat.com (Ian Wienand) Date: Tue, 17 Dec 2019 15:35:12 +1100 Subject: [meta-sig][multi-arch] propose forming a Multi-arch SIG In-Reply-To: References: <20191121001509.GB976114@fedora19.localdomain> Message-ID: <20191217043512.GA2367741@fedora19.localdomain> On Tue, Nov 26, 2019 at 11:33:16AM +0000, Jonathan Rosser wrote: > openstack-ansible is ready to go on arm CI but in order to make the jobs run > in a reasonable time and not simply timeout a source of pre-built arm python > wheels is needed. It would be a shame to let the work that got contributed > to OSA for arm just rot. So ARM64 wheels are still a work-in-progress, but in the mean time we have merged a change to install a separate queue for ARM64 jobs [1]. Jobs in the "check-arm64" queue will be implicitly non-voting (Zuul isn't configured to add +-1 votes for this queue) but importantly will run asynchronously to the regular queue. Thus if there's very high demand, or any intermittent instability your gates won't be held up. [2] is an example of using this in diskimage-builder. Of course you *can* put ARM64 jobs in your gate queues as voting jobs, but just be aware with only 8 nodes available at this time, it could easily become a bottle-neck to merging code. The "check-arm64" queue is designed to be an automatically-running half-way point as we (hopefully) scale up support (like wheel builds and mirrors) and resources further. Thanks, -i [1] https://review.opendev.org/#/c/698606/ [2] https://review.opendev.org/#/c/676111/ From juliaashleykreger at gmail.com Tue Dec 17 04:42:04 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Mon, 16 Dec 2019 20:42:04 -0800 Subject: [neutron][devstack] Future of lib/neutron and lib/neutron-legacy In-Reply-To: <755fb9bf-799f-1164-1184-96b146222235@gmail.com> References: <48325993-0233-437B-ABC5-D038F0785F04@redhat.com> <755fb9bf-799f-1164-1184-96b146222235@gmail.com> Message-ID: On Mon, Dec 16, 2019 at 7:38 PM Brian Haley wrote: > > Hi Slawek, > > On 12/12/19 9:06 AM, Slawek Kaplonski wrote: > > Hi, > > > > Few years ago (I don’t know exactly when it started as I wasn’t very active upstream than for sure) Neutron team started work on new neutron module for Devstack. See [1] for details. > > But since then this new module is still not ready to use. For example in gate jobs we are still using lib/neutron-legacy module as this new one is always causing some problems. > > Things in its current shape may be confusing for new Devstack users as in fact to deploy Neutron properly they should use lib/neutron-legacy instead of lib/neutron now. Great... I guess that explains why Ironic has never really been able to completely get the "newer" plugin to work as of recent. Well, that and changes to devstack seem to tend to take a very long time to get reviews when we have encountered things that can be fixed in the case of use in the Bare Metal case. Is there an up to date central running list of issues anywhere? I know at some point support to choose vlan finally merged with-in the last few months, so that is at least one major item that has blocked ironic from adopting... > > Personally I never really checked exactly what issues are there in fact but I do remember some issues with using lib/neutron e.g. on subnodes in multinode jobs - see [2]. > > > > So I would like to ask a question what do You think we should do with it. > > Is there maybe any volunteer who would like to continue this work on adopting lib/neutron and to finally use it in gate jobs? > > Or if there is no anyone and we are all fine with using old module, maybe we should remove this lib/neutron and “undeprecate” and rename lib/neutron-legacy? > > > > [1] https://github.com/openstack/devstack/commit/2a242519f71e86416e78541826cac2b54fcd04a5 > > [2] https://review.opendev.org/#/c/662480/ > > Thanks for looking at this again. My initial thought is to complete the > work as it's so close, this topic has the remaining abandoned reviews, > https://review.opendev.org/#/q/topic:new-neutron-devstack-in-gate > > I wish I remembered more about why we never finished, I only remember > bgpvpn needed some work, just don't remember much else. Could > definitely do some of the reviews. > > -Brian > I also think it would be good, but I'm curious why we can't break the plugin out of devstack itself like a number of other projects have plugins which allow changes to be reviewed and merged rapidly by the team working closest to the service code base? From rico.lin.guanyu at gmail.com Tue Dec 17 06:11:11 2019 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Tue, 17 Dec 2019 14:11:11 +0800 Subject: [meta-sig][multi-arch] propose forming a Multi-arch SIG In-Reply-To: <20191217043512.GA2367741@fedora19.localdomain> References: <20191121001509.GB976114@fedora19.localdomain> <20191217043512.GA2367741@fedora19.localdomain> Message-ID: >From this ML, and some IRC and Wechat discussions. I put most of the information I collected in [1]. At this point, we can tell there's a lot of works already in progress in this community. So I think we can definitely get benefits from this SIG. Here are things we need to settle at this point: - *SIG chairs*: We need multiple SIG chairs who can help to drive SIG goals and host meetings/events. *Put your name under `SIG chairs:` if you're interested*. I will propose my name on the create SIG patch since I'm interested in helping set this SIG up and we need to fillup something there. But that won't block you from signing up. And I'm more than happy if we can have more people rush in for the chair role:). - *First meeting schedule*: I create polling for meeting time [2]. *Please pick your favorite for our first meeting time* (And potentially our long term meeting schedule, but let's discuss that in the meeting). I pick the second week of Jan. because some might be on their vacation in the following two weeks. As for the location, I do like to suggest we use #openstack-meeting, so we might be able to get more people's attention. From the experience of other SIGs, to run a meeting on your own IRC channel, make it harder for new community members to join. - *Resources*: We need to find out who or which organization is also interested in this. Right now, I believe we need more servers to run tests, and people to help on making test jobs, feedbacks, or any other tasks. So please help to forward the etherpad([1]) and add on more information that I fail to mention:) If you can find organizations that might be interested in donating servers, I can help to reach out too. *So sign up and provide any information that you think will helps:)* - *Build and trace*: We definitely need to target all the above works(from the previous replies) in this SIG, and (like Ian mentioned) to work on the test infrastructure. And these make great first step tasks for SIG. And to track all jobs, I think it will be reasonable to create a Storyboard for this SIG and document those tasks in one Storyboard. All the above tasks IMO don't need to wait for the first meeting to happen before them, so If anyone likes to put their effort on any of them or like to suggest more initial tasks, you're the most welcome here! [1] https://etherpad.openstack.org/p/Multi-arch [2] https://doodle.com/poll/8znyzc57skqkryv8 On Tue, Dec 17, 2019 at 12:45 PM Ian Wienand wrote: > On Tue, Nov 26, 2019 at 11:33:16AM +0000, Jonathan Rosser wrote: > > openstack-ansible is ready to go on arm CI but in order to make the jobs > run > > in a reasonable time and not simply timeout a source of pre-built arm > python > > wheels is needed. It would be a shame to let the work that got > contributed > > to OSA for arm just rot. > > So ARM64 wheels are still a work-in-progress, but in the mean time we > have merged a change to install a separate queue for ARM64 jobs [1]. > Jobs in the "check-arm64" queue will be implicitly non-voting (Zuul > isn't configured to add +-1 votes for this queue) but importantly will > run asynchronously to the regular queue. Thus if there's very high > demand, or any intermittent instability your gates won't be held up. > > [2] is an example of using this in diskimage-builder. > > Of course you *can* put ARM64 jobs in your gate queues as voting jobs, > but just be aware with only 8 nodes available at this time, it could > easily become a bottle-neck to merging code. > > The "check-arm64" queue is designed to be an automatically-running > half-way point as we (hopefully) scale up support (like wheel builds > and mirrors) and resources further. > > Thanks, > > -i > > [1] https://review.opendev.org/#/c/698606/ > [2] https://review.opendev.org/#/c/676111/ > > > -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcin.juszkiewicz at linaro.org Tue Dec 17 08:44:04 2019 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Tue, 17 Dec 2019 09:44:04 +0100 Subject: [goals][Drop Python 2.7 Support] Week R-22 Update In-Reply-To: <16f053e80fe.10f635d3855454.1090802515964741682@ghanshyammann.com> References: <16f053e80fe.10f635d3855454.1090802515964741682@ghanshyammann.com> Message-ID: W dniu 14.12.2019 o 17:29, Ghanshyam Mann pisze: > The OpenStack services have not merged the py2 drop patches: > * Kolla We are waiting for TripleO team to move to Python 3 in their CentOS 7 images. All our CI jobs use Python 3. From thierry at openstack.org Tue Dec 17 11:14:03 2019 From: thierry at openstack.org (Thierry Carrez) Date: Tue, 17 Dec 2019 12:14:03 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: Message-ID: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Zane Bitter wrote: > On 15/12/19 10:20 pm, Rong Zhu wrote: >> 1.Add Ceilometer API back >>      Since Gnocchi is out of OpenStack and is unmaintained, we need to >> add Ceilometer API back again. > > This is concerning because even the people who wrote it don't consider > it adequate to the job. That inadequacy has been the source of > significant reputational damage to OpenStack in the past, and many folks > (me included) are anxious to avoid a repeat. Yes this concerns me too, and even if we workaround the issue by adding Ceilo API back, I'd like to have a long-term plan to solve this issue. It seems there are several options on the table (including integrating Monasca and Ceilometer into a single stack, and other big-bang replacements) but it's never a one-for-one comparison as the solutions seem to address slightly disjoint problem spaces. I'd like to hear from more Ceilometer users. What are they using Ceilometer for, and what long-term plans would be acceptable. There is a trade-off between adopting short-term workarounds that reintroduce performance issues vs. undergoing a complex migration to the "right" way of fixing this. Like for example there is little point in pushing Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud seems to say, that they would rather have a slow performing Ceilometer API back. > Telemetry is part of the TC "Approved Release" that is eligible for the > trademark program; I think at a minimum the TC will want to remove the > resurrected Ceilometer API from the "designated sections" that users are > required to run to participate in any trademark program that includes > the functionality in question. But I think that we should explore other > ways of reducing the chance of anyone confusing this for a viable way of > building a cloud, including possibly changing the name (Antediluvian > API?) and having this part of the stack live outside of the official > OpenStack project. Legacy API? -- Thierry Carrez (ttx) From fungi at yuggoth.org Tue Dec 17 13:40:54 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 17 Dec 2019 13:40:54 +0000 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> Message-ID: <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> On 2019-12-15 17:05:13 +0000 (+0000), Jeremy Stanley wrote: [...] > If there's anyone interested in helping out as additional > administrators for the openstack-discuss mailing list, please reply > to this message on-list (so other subscribers know who is > volunteering) and I'll help bring you up to speed. [...] The outpouring of volunteers took me by (pleasant!) surprise. I'm going to follow up privately with Radosław Piliszek, Florian Rommel, and Chris MacNaughton to provide them with further details and prepare them for the coming two weeks that I'll be away from my keyboard. A huge thanks as well to Slawek Kaplonski, Graham Hayes, Sean McGinnis, and Michael Johnson. In an effort not to further overload our existing community leaders, do you mind if I keep you on the reserve bench in case we need additional openstack-discuss list admins in the future? -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From sean.mcginnis at gmx.com Tue Dec 17 14:27:20 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Tue, 17 Dec 2019 08:27:20 -0600 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> Message-ID: <20191217142720.GA1238521@sm-workstation> > The outpouring of volunteers took me by (pleasant!) surprise. I'm > going to follow up privately with Radosław Piliszek, Florian Rommel, > and Chris MacNaughton to provide them with further details and > prepare them for the coming two weeks that I'll be away from my > keyboard. > > A huge thanks as well to Slawek Kaplonski, Graham Hayes, Sean > McGinnis, and Michael Johnson. In an effort not to further overload > our existing community leaders, do you mind if I keep you on the > reserve bench in case we need additional openstack-discuss list > admins in the future? > -- > Jeremy Stanley That works for me. Just let me know if any help is needed down the line. Sean From mfedosin at redhat.com Tue Dec 17 14:45:58 2019 From: mfedosin at redhat.com (Mikhail Fedosin) Date: Tue, 17 Dec 2019 15:45:58 +0100 Subject: [nova][metadata] EC2 compatible metadata deprecation Message-ID: Hello! In the documentation I noticed that EC2 metadata may soon be removed from Nova [1]. Currently, we use EC2 metadata in our product to obtain public and private IP addresses, as well as the instance-type (flavor). Therefore, I would like to ask you a couple of questions. 1. Do you plan to ensure compatibility before removing EC2 metadata from the system, i.e. to add these fields to the OpenStack metadata, which is not yet available? 2. When is it expected that EC2 metadata will be removed from the system? Best regards, Mikhail Fedosin [1] https://docs.openstack.org/nova/latest/user/metadata.html#general-guidelines -------------- next part -------------- An HTML attachment was scrubbed... URL: From alifshit at redhat.com Tue Dec 17 15:10:48 2019 From: alifshit at redhat.com (Artom Lifshitz) Date: Tue, 17 Dec 2019 10:10:48 -0500 Subject: [nova][metadata] EC2 compatible metadata deprecation In-Reply-To: References: Message-ID: On Tue, Dec 17, 2019 at 9:49 AM Mikhail Fedosin wrote: > > Hello! > > In the documentation I noticed that EC2 metadata may soon be removed from Nova [1]. Currently, we use EC2 metadata in our product to obtain public and private IP addresses, as well as the instance-type (flavor). Therefore, I would like to ask you a couple of questions. > 1. Do you plan to ensure compatibility before removing EC2 metadata from the system, i.e. to add these fields to the OpenStack metadata, which is not yet available? > 2. When is it expected that EC2 metadata will be removed from the system? Nova's in-tree ec2-api has already been removed [2] (though I can't find the commit that did it). That being said, the out-of-tree ec2-api project [3] is still around and kicking (just barely, looking at the commit history, but it's not inactive). > > Best regards, > Mikhail Fedosin > > [1] https://docs.openstack.org/nova/latest/user/metadata.html#general-guidelines [2] https://opendev.org/openstack/nova/src/branch/master/nova/api [3] https://opendev.org/openstack/ec2-api From mark at stackhpc.com Tue Dec 17 15:13:18 2019 From: mark at stackhpc.com (Mark Goddard) Date: Tue, 17 Dec 2019 15:13:18 +0000 Subject: [kolla] Kolla and Kolla Ansible 9.0.0 released for Train Message-ID: Hi, I'm pleased to announce the availability of the Kolla and Kolla Ansible 9.0.0 releases for the Train cycle! Train brings a number of interesting new features, including Masakari and Qinling support, the ability to deploy multiple Nova cells and TLS encryption of the internal API network. Full details are in the release notes which are available here: https://docs.openstack.org/releasenotes/kolla/train.html https://docs.openstack.org/releasenotes/kolla-ansible/train.html We anticipate significant changes during the Train release series to support a migration to CentOS 8. We will communicate which releases are affected. The first official release of kolla-cli should be available in the next few days. Release notes will be available here: https://docs.openstack.org/releasenotes/kolla-cli/train.html The first official release of Kayobe is making some finishing touches, and should be released by the end of the year, or early next year. Thanks to everyone who contributed to this release, it adds a lot of useful features. A great way to end the year. Cheers, Mark From skaplons at redhat.com Tue Dec 17 15:42:15 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Tue, 17 Dec 2019 16:42:15 +0100 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> Message-ID: Hi, > On 17 Dec 2019, at 14:40, Jeremy Stanley wrote: > > On 2019-12-15 17:05:13 +0000 (+0000), Jeremy Stanley wrote: > [...] >> If there's anyone interested in helping out as additional >> administrators for the openstack-discuss mailing list, please reply >> to this message on-list (so other subscribers know who is >> volunteering) and I'll help bring you up to speed. > [...] > > The outpouring of volunteers took me by (pleasant!) surprise. I'm > going to follow up privately with Radosław Piliszek, Florian Rommel, > and Chris MacNaughton to provide them with further details and > prepare them for the coming two weeks that I'll be away from my > keyboard. > > A huge thanks as well to Slawek Kaplonski, Graham Hayes, Sean > McGinnis, and Michael Johnson. In an effort not to further overload > our existing community leaders, do you mind if I keep you on the > reserve bench in case we need additional openstack-discuss list > admins in the future? Sure. It’s fine for me :) > -- > Jeremy Stanley — Slawek Kaplonski Senior software engineer Red Hat From mriedemos at gmail.com Tue Dec 17 15:58:15 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 17 Dec 2019 09:58:15 -0600 Subject: [nova][metadata] EC2 compatible metadata deprecation In-Reply-To: References: Message-ID: <5c9ade13-9b1b-a07b-699f-cdda8bc9cc46@gmail.com> On 12/17/2019 9:10 AM, Artom Lifshitz wrote: >> Currently, we use EC2 metadata in our product to obtain public and private IP addresses, as well as the instance-type (flavor). Therefore, I would like to ask you a couple of questions. >> 1. Do you plan to ensure compatibility before removing EC2 metadata from the system, i.e. to add these fields to the OpenStack metadata, which is not yet available? I would think adding flavor info to meta_data.json should be trivial. It's an API change so it requires a spec though [1]. As for the network addresses, those aren't in network_data.json? Are you using neutron? >> 2. When is it expected that EC2 metadata will be removed from the system? I wouldn't expect it anytime soon. The documentation that mentions this is a warning to not use something that is no longer maintained in nova (anything related to ec2), like a deprecation warning of sorts. If you have identified feature compatibility gaps to close in the openstack metadata API, please open a spec for Ussuri detailing what you need. Flavor should be pretty easy and the network addresses I would expect are already available in network_data.json but if something is missing there let's get it documented in the spec. > Nova's in-tree ec2-api has already been removed [2] (though I can't > find the commit that did it). That being said, the out-of-tree ec2-api > project [3] is still around and kicking (just barely, looking at the > commit history, but it's not inactive). > Mikhail isn't talking about the user-facing EC2 API shim, he's talking about the metadata API code [2]. [1] https://specs.openstack.org/openstack/nova-specs/readme.html [2] https://github.com/openstack/nova/blob/20.0.0/nova/api/metadata/base.py#L236 -- Thanks, Matt From johnsomor at gmail.com Tue Dec 17 16:27:08 2019 From: johnsomor at gmail.com (Michael Johnson) Date: Tue, 17 Dec 2019 08:27:08 -0800 Subject: [all] Volunteer mailing list administrators In-Reply-To: References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> Message-ID: Sounds good. Michael On Tue, Dec 17, 2019 at 7:45 AM Slawek Kaplonski wrote: > > Hi, > > > On 17 Dec 2019, at 14:40, Jeremy Stanley wrote: > > > > On 2019-12-15 17:05:13 +0000 (+0000), Jeremy Stanley wrote: > > [...] > >> If there's anyone interested in helping out as additional > >> administrators for the openstack-discuss mailing list, please reply > >> to this message on-list (so other subscribers know who is > >> volunteering) and I'll help bring you up to speed. > > [...] > > > > The outpouring of volunteers took me by (pleasant!) surprise. I'm > > going to follow up privately with Radosław Piliszek, Florian Rommel, > > and Chris MacNaughton to provide them with further details and > > prepare them for the coming two weeks that I'll be away from my > > keyboard. > > > > A huge thanks as well to Slawek Kaplonski, Graham Hayes, Sean > > McGinnis, and Michael Johnson. In an effort not to further overload > > our existing community leaders, do you mind if I keep you on the > > reserve bench in case we need additional openstack-discuss list > > admins in the future? > > Sure. It’s fine for me :) > > > -- > > Jeremy Stanley > > — > Slawek Kaplonski > Senior software engineer > Red Hat > > From gr at ham.ie Tue Dec 17 16:39:42 2019 From: gr at ham.ie (Graham Hayes) Date: Tue, 17 Dec 2019 16:39:42 +0000 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> Message-ID: <6057ff2d-1ad1-7372-b85d-71857bfbc860@ham.ie> On 17/12/2019 13:40, Jeremy Stanley wrote: > On 2019-12-15 17:05:13 +0000 (+0000), Jeremy Stanley wrote: > [...] >> If there's anyone interested in helping out as additional >> administrators for the openstack-discuss mailing list, please reply >> to this message on-list (so other subscribers know who is >> volunteering) and I'll help bring you up to speed. > [...] > > The outpouring of volunteers took me by (pleasant!) surprise. I'm > going to follow up privately with Radosław Piliszek, Florian Rommel, > and Chris MacNaughton to provide them with further details and > prepare them for the coming two weeks that I'll be away from my > keyboard. > > A huge thanks as well to Slawek Kaplonski, Graham Hayes, Sean > McGinnis, and Michael Johnson. In an effort not to further overload > our existing community leaders, do you mind if I keep you on the > reserve bench in case we need additional openstack-discuss list > admins in the future? > Please do! - Graham From ralonsoh at redhat.com Tue Dec 17 16:46:10 2019 From: ralonsoh at redhat.com (Rodolfo Alonso) Date: Tue, 17 Dec 2019 16:46:10 +0000 Subject: [Neutron][QoS] Next IRC QoS meeting 14/1/2020 Message-ID: <3eca13c17fa72a28183508abd47bde0ec173d1f0.camel@redhat.com> Hello Neutrinos: We'll cancel next QoS meeting and meet again on the 14th of January. Happy holidays to all! Regards. From mihalis68 at gmail.com Tue Dec 17 17:11:35 2019 From: mihalis68 at gmail.com (Chris Morgan) Date: Tue, 17 Dec 2019 12:11:35 -0500 Subject: [ops] openstack ops meetup team meeting 2019-12-17 Message-ID: Meeting minutes from today's meeting of the openstack ops meetups team on #openstack-operators : Meeting ended Tue Dec 17 15:35:22 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) 10:35 AM Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_team.2019-12-17-15.00.html 10:35 AM Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_team.2019-12-17-15.00.txt 10:35 AM Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_team.2019-12-17-15.00.log.html the next meetup is in London on Jan 7,8 and registration is still open ( https://go.bloomberg.com/attend/invite/openstack-operators-meetup-london2020/ ) Regular updates on meetups are posted to the notification twitter account (see https://twitter.com/osopsmeetup) There will be an openstack ops meetup at the next summit (rumored to be in vancouver) and then later in 2020 in south korea. See the above-linked minutes for more details. Chris Morgan (on behalf of the openstack ops meetups team) -- Chris Morgan -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfedosin at redhat.com Tue Dec 17 18:14:37 2019 From: mfedosin at redhat.com (Mikhail Fedosin) Date: Tue, 17 Dec 2019 19:14:37 +0100 Subject: [nova][metadata] EC2 compatible metadata deprecation In-Reply-To: <5c9ade13-9b1b-a07b-699f-cdda8bc9cc46@gmail.com> References: <5c9ade13-9b1b-a07b-699f-cdda8bc9cc46@gmail.com> Message-ID: Thank you for the explanation! Currently network_data.json doesn't contain ip addresses: $ cat openstack/latest/network_data.json {"services": [], "networks": [{"network_id": "39376584-af8a-4307-8cc3-dbf3474f0d52", "link": "tapde154f64-0c", "type": "ipv4_dhcp", "id": "network0"}], "links": [{"ethernet_mac_address": "fa:16:3e:9b:55:1d", "mtu": 1450, "type": "ovs", "id": "tapde154f64-0c", "vif_id": "de154f64-0cb3-4c71-b449-f3f67646eb2f"}]} but they are available with ec2 metadata $ cat ec2/latest/meta-data.json {"reservation-id": "r-l1rethmm", "security-groups": ["c8xbg-master"], "public-ipv4": "10.0.79.125", "ami-manifest-path": "FIXME", "instance-type": "m1.xlarge", "instance-id": "i-017ae668", "local-ipv4": "10.0.128.15", "local-hostname": "mfedosin-c8xbg-bootstrap", "placement": {"availability-zone": "nova"}, "ami-launch-index": 0, "public-hostname": "mfedosin-c8xbg-bootstrap", "hostname": "mfedosin-c8xbg-bootstrap", "ami-id": "ami-00005ee4", "instance-action": "none", "block-device-mapping": {"ami": "vda", "root": "/dev/vda"}} So far, in OpenStack metadata we don't have "security-groups", "public-ipv4", "local-ipv4", "instance-type", and I hope we can port them easily. I'm not sure about "instance-id" and "reservation-id", and how we can possibly use them in OpenStack, so I think we don't need them at all. Well, my plan is to create a spec for Ussuri then :) On Tue, Dec 17, 2019 at 5:04 PM Matt Riedemann wrote: > On 12/17/2019 9:10 AM, Artom Lifshitz wrote: > >> Currently, we use EC2 metadata in our product to obtain public and > private IP addresses, as well as the instance-type (flavor). Therefore, I > would like to ask you a couple of questions. > >> 1. Do you plan to ensure compatibility before removing EC2 metadata > from the system, i.e. to add these fields to the OpenStack metadata, which > is not yet available? > > I would think adding flavor info to meta_data.json should be trivial. > It's an API change so it requires a spec though [1]. > > As for the network addresses, those aren't in network_data.json? Are you > using neutron? > > >> 2. When is it expected that EC2 metadata will be removed from the > system? > > I wouldn't expect it anytime soon. The documentation that mentions this > is a warning to not use something that is no longer maintained in nova > (anything related to ec2), like a deprecation warning of sorts. > > If you have identified feature compatibility gaps to close in the > openstack metadata API, please open a spec for Ussuri detailing what you > need. Flavor should be pretty easy and the network addresses I would > expect are already available in network_data.json but if something is > missing there let's get it documented in the spec. > > > Nova's in-tree ec2-api has already been removed [2] (though I can't > > find the commit that did it). That being said, the out-of-tree ec2-api > > project [3] is still around and kicking (just barely, looking at the > > commit history, but it's not inactive). > > > > Mikhail isn't talking about the user-facing EC2 API shim, he's talking > about the metadata API code [2]. > > [1] https://specs.openstack.org/openstack/nova-specs/readme.html > [2] > > https://github.com/openstack/nova/blob/20.0.0/nova/api/metadata/base.py#L236 > > -- > > Thanks, > > Matt > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seanroberts66 at gmail.com Mon Dec 16 22:25:49 2019 From: seanroberts66 at gmail.com (Sean) Date: Mon, 16 Dec 2019 14:25:49 -0800 Subject: [OpenStack Foundation] [Foundation Board] [tc][board][all] -Adding OpenStack community support to the savedotorg campaign In-Reply-To: References: <057BD0AC-451C-4E64-9D30-6CC6A20DFA0F@demarco.com> <1595EA09-2B8C-45ED-B836-5751FC79DBA1@openstack.org> <2a19b6c1-c769-7725-7bb9-3998cdfe34c5@ham.ie> <308dbe44-df65-8ae9-af4f-7c6e2878e4a0@lohutok.net> Message-ID: <5df8048f.1c69fb81.6360b.4568@mx.google.com> Thanks to the OpenStack community highlighting this important issue. I have added my comments to icann for what they are worth. ~ sean From: Sean McGinnis Sent: Monday, December 16, 2019 5:42 AM To: Graham Hayes Cc: openstack-discuss at lists.openstack.org; foundation at lists.openstack.org; foundation-board at lists.openstack.org Subject: Re: [OpenStack Foundation] [Foundation Board] [tc][board][all] -Adding OpenStack community support to the savedotorg campaign > > > It seems the pressure on ICANN has had an effect: > > https://www.icann.org/news/blog/org-update > > Great to see something happening. Thanks for the update Graham! _______________________________________________ Foundation mailing list Foundation at lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/foundation -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas at goirand.fr Tue Dec 17 15:05:49 2019 From: thomas at goirand.fr (Thomas Goirand) Date: Tue, 17 Dec 2019 16:05:49 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: Message-ID: <6fac10fd-894b-93f2-f461-6c0c726390e3@goirand.fr> On 12/17/19 4:01 AM, Zane Bitter wrote: > On 15/12/19 10:20 pm, Rong Zhu wrote: >> 1.Add Ceilometer API back >>      Since Gnocchi is out of OpenStack and is unmaintained, we need to >> add Ceilometer API back again. > > This is concerning because even the people who wrote it don't consider > it adequate to the job. That inadequacy has been the source of > significant reputational damage to OpenStack in the past, and many folks > (me included) are anxious to avoid a repeat. > > Telemetry is part of the TC "Approved Release" that is eligible for the > trademark program; I think at a minimum the TC will want to remove the > resurrected Ceilometer API from the "designated sections" that users are > required to run to participate in any trademark program that includes > the functionality in question. But I think that we should explore other > ways of reducing the chance of anyone confusing this for a viable way of > building a cloud, including possibly changing the name (Antediluvian > API?) and having this part of the stack live outside of the official > OpenStack project. > > I'm not clear on which parts of the API you plan to resurrect, but for > example - wearing my Heat hat - after a long process of migrating > Ceilometer alarm resources in Heat to Aodh ones (mostly involving > breaking changes forced on us by the absence of a migration path in > Telemetry), I have zero interest in turning around and going back in the > other direction. I would infinitely prefer that we find a path to e.g. > migrate to Monasca as the collector so that the community can speak with > one voice about how to do metrics and alarming in OpenStack. > >> 2.Mongodb database backend support >>    As this thread [0] discuss, there are still some users still use >> mongodb as the database backend, we decide to add this support again. >> License issue should consider and fix by user. > > So long as this is not the only backend available, I agree the licensing > of MongoDB itself is not a concern for OpenStack (we have plenty of > drivers that depend on even proprietary software in various places). > > The issue that Matthias alluded to in the thread is that Linux > distributions are dropping MongoDB due to its non-free license, so it is > not nearly as widely and easily available as it was in the past. > >> The OVH and Catalyst Cloud had fix the mongodb performance issue with >> some mongodb changes. > > It's really hard to reconcile this with the information here (and > especially in the linked slide): > > https://github.com/gnocchixyz/gnocchi/issues/1049#issuecomment-555768072 > >> Gnoochi will still support as the backed. Other database (like >> influxdb, ES....), we would happy everyone to submit patches to >> support as the database backed in ceilometer. > > This is good to hear :) > > cheers, > Zane. Hi, I very much share Zane's concerns. As a distribution package maintainer, I will *not* re-add a ceilometer-api package. What I don't understand is why you guys aren't taking over Gnocchi, or attempt to use a better time series for Telemetry. I also feel like you're trying to scratch your own itch with MongoDB and Ceilometer API, rather than addressing the issues of the project for the wider community, dismissing both what has been said the the previous thread and what the previous maintainers are telling you (ie: that Ceilometer-api + MongoDB is a very bad idea). Why not instead creating a migration path from MongoDB to something more viable/reliable? Cheers, Thomas Goirand (zigo) From Tim.Bell at cern.ch Tue Dec 17 18:52:56 2019 From: Tim.Bell at cern.ch (Tim Bell) Date: Tue, 17 Dec 2019 18:52:56 +0000 Subject: [nova][metadata] EC2 compatible metadata deprecation In-Reply-To: References: <5c9ade13-9b1b-a07b-699f-cdda8bc9cc46@gmail.com> Message-ID: <04360B45-B323-4C14-9233-57A65D0E91B0@cern.ch> On 17 Dec 2019, at 19:14, Mikhail Fedosin > wrote: Thank you for the explanation! Currently network_data.json doesn't contain ip addresses: $ cat openstack/latest/network_data.json {"services": [], "networks": [{"network_id": "39376584-af8a-4307-8cc3-dbf3474f0d52", "link": "tapde154f64-0c", "type": "ipv4_dhcp", "id": "network0"}], "links": [{"ethernet_mac_address": "fa:16:3e:9b:55:1d", "mtu": 1450, "type": "ovs", "id": "tapde154f64-0c", "vif_id": "de154f64-0cb3-4c71-b449-f3f67646eb2f"}]} but they are available with ec2 metadata $ cat ec2/latest/meta-data.json {"reservation-id": "r-l1rethmm", "security-groups": ["c8xbg-master"], "public-ipv4": "10.0.79.125", "ami-manifest-path": "FIXME", "instance-type": "m1.xlarge", "instance-id": "i-017ae668", "local-ipv4": "10.0.128.15", "local-hostname": "mfedosin-c8xbg-bootstrap", "placement": {"availability-zone": "nova"}, "ami-launch-index": 0, "public-hostname": "mfedosin-c8xbg-bootstrap", "hostname": "mfedosin-c8xbg-bootstrap", "ami-id": "ami-00005ee4", "instance-action": "none", "block-device-mapping": {"ami": "vda", "root": "/dev/vda"}} So far, in OpenStack metadata we don't have "security-groups", "public-ipv4", "local-ipv4", "instance-type", and I hope we can port them easily. I'm not sure about "instance-id" and "reservation-id", and how we can possibly use them in OpenStack, so I think we don't need them at all. Well, my plan is to create a spec for Ussuri then :) Can we also have the ipv6 info in the Nova spec too ? Tim On Tue, Dec 17, 2019 at 5:04 PM Matt Riedemann > wrote: On 12/17/2019 9:10 AM, Artom Lifshitz wrote: >> Currently, we use EC2 metadata in our product to obtain public and private IP addresses, as well as the instance-type (flavor). Therefore, I would like to ask you a couple of questions. >> 1. Do you plan to ensure compatibility before removing EC2 metadata from the system, i.e. to add these fields to the OpenStack metadata, which is not yet available? I would think adding flavor info to meta_data.json should be trivial. It's an API change so it requires a spec though [1]. As for the network addresses, those aren't in network_data.json? Are you using neutron? >> 2. When is it expected that EC2 metadata will be removed from the system? I wouldn't expect it anytime soon. The documentation that mentions this is a warning to not use something that is no longer maintained in nova (anything related to ec2), like a deprecation warning of sorts. If you have identified feature compatibility gaps to close in the openstack metadata API, please open a spec for Ussuri detailing what you need. Flavor should be pretty easy and the network addresses I would expect are already available in network_data.json but if something is missing there let's get it documented in the spec. > Nova's in-tree ec2-api has already been removed [2] (though I can't > find the commit that did it). That being said, the out-of-tree ec2-api > project [3] is still around and kicking (just barely, looking at the > commit history, but it's not inactive). > Mikhail isn't talking about the user-facing EC2 API shim, he's talking about the metadata API code [2]. [1] https://specs.openstack.org/openstack/nova-specs/readme.html [2] https://github.com/openstack/nova/blob/20.0.0/nova/api/metadata/base.py#L236 -- Thanks, Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Tue Dec 17 18:59:37 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 17 Dec 2019 12:59:37 -0600 Subject: [nova][metadata] EC2 compatible metadata deprecation In-Reply-To: References: <5c9ade13-9b1b-a07b-699f-cdda8bc9cc46@gmail.com> Message-ID: <795b25ad-f394-b781-7b23-9be0bafb2f1a@gmail.com> On 12/17/2019 12:14 PM, Mikhail Fedosin wrote: > Currently network_data.json doesn't contain ip addresses: > $ cat openstack/latest/network_data.json > {"services": [], "networks": [{"network_id": > "39376584-af8a-4307-8cc3-dbf3474f0d52", "link": "tapde154f64-0c", > "type": "ipv4_dhcp", "id": "network0"}], "links": > [{"ethernet_mac_address": "fa:16:3e:9b:55:1d", "mtu": 1450, "type": > "ovs", "id": "tapde154f64-0c", "vif_id": > "de154f64-0cb3-4c71-b449-f3f67646eb2f"}]} This seems to be something in the setup of the network(s) attached to the server. Looking at the code that builds the network_data.json payload, it's returning here for you [1]. But if you look further down the network info dict would contain an IP address [2]. Maybe this is a limitation of the subnet that is chosen if there are multiple per network [3]. I want to say I've seen a bug about that recently but can't track it down. [1] https://github.com/openstack/nova/blob/f236c62d2eabc39b9f3301eea3be19389a930d7c/nova/virt/netutils.py#L292 [2] https://github.com/openstack/nova/blob/f236c62d2eabc39b9f3301eea3be19389a930d7c/nova/virt/netutils.py#L305 [3] https://github.com/openstack/nova/blob/f236c62d2eabc39b9f3301eea3be19389a930d7c/nova/virt/netutils.py#L194 -- Thanks, Matt From radoslaw.piliszek at gmail.com Tue Dec 17 19:17:04 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 17 Dec 2019 20:17:04 +0100 Subject: [all] Volunteer mailing list administrators In-Reply-To: <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> Message-ID: While I understand the reasoning behind the decision, me being me I must throw in my 2 cents: I think you have chosen 3 folks in much the same timezone. I might be mistaken but based on knowing myself and mail domains of others I expect we are all in timezones GMT+1/+2 It might be handy to have someone in timezones more suited for China and USA as well. :-) -yoctozepto wt., 17 gru 2019 o 14:49 Jeremy Stanley napisał(a): > > On 2019-12-15 17:05:13 +0000 (+0000), Jeremy Stanley wrote: > [...] > > If there's anyone interested in helping out as additional > > administrators for the openstack-discuss mailing list, please reply > > to this message on-list (so other subscribers know who is > > volunteering) and I'll help bring you up to speed. > [...] > > The outpouring of volunteers took me by (pleasant!) surprise. I'm > going to follow up privately with Radosław Piliszek, Florian Rommel, > and Chris MacNaughton to provide them with further details and > prepare them for the coming two weeks that I'll be away from my > keyboard. > > A huge thanks as well to Slawek Kaplonski, Graham Hayes, Sean > McGinnis, and Michael Johnson. In an effort not to further overload > our existing community leaders, do you mind if I keep you on the > reserve bench in case we need additional openstack-discuss list > admins in the future? > -- > Jeremy Stanley From mgagne at calavera.ca Tue Dec 17 19:36:06 2019 From: mgagne at calavera.ca (=?UTF-8?Q?Mathieu_Gagn=C3=A9?=) Date: Tue, 17 Dec 2019 14:36:06 -0500 Subject: [nova][metadata] EC2 compatible metadata deprecation In-Reply-To: <795b25ad-f394-b781-7b23-9be0bafb2f1a@gmail.com> References: <5c9ade13-9b1b-a07b-699f-cdda8bc9cc46@gmail.com> <795b25ad-f394-b781-7b23-9be0bafb2f1a@gmail.com> Message-ID: On Tue, Dec 17, 2019 at 1:59 PM Matt Riedemann wrote: > > On 12/17/2019 12:14 PM, Mikhail Fedosin wrote: > > Currently network_data.json doesn't contain ip addresses: > > $ cat openstack/latest/network_data.json > > {"services": [], "networks": [{"network_id": > > "39376584-af8a-4307-8cc3-dbf3474f0d52", "link": "tapde154f64-0c", > > "type": "ipv4_dhcp", "id": "network0"}], "links": > > [{"ethernet_mac_address": "fa:16:3e:9b:55:1d", "mtu": 1450, "type": > > "ovs", "id": "tapde154f64-0c", "vif_id": > > "de154f64-0cb3-4c71-b449-f3f67646eb2f"}]} > > This seems to be something in the setup of the network(s) attached to > the server. > > Looking at the code that builds the network_data.json payload, it's > returning here for you [1]. But if you look further down the network > info dict would contain an IP address [2]. Maybe this is a limitation of > the subnet that is chosen if there are multiple per network [3]. I want > to say I've seen a bug about that recently but can't track it down. There was a spec to add all subnets to network_data.json: https://review.opendev.org/#/c/580742/ I unfortunately don't have time anymore to work on it. I don't know the impact on installation using DHCP instead of static IP. > [1] > https://github.com/openstack/nova/blob/f236c62d2eabc39b9f3301eea3be19389a930d7c/nova/virt/netutils.py#L292 > [2] > https://github.com/openstack/nova/blob/f236c62d2eabc39b9f3301eea3be19389a930d7c/nova/virt/netutils.py#L305 > [3] > https://github.com/openstack/nova/blob/f236c62d2eabc39b9f3301eea3be19389a930d7c/nova/virt/netutils.py#L194 From fungi at yuggoth.org Tue Dec 17 19:43:52 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 17 Dec 2019 19:43:52 +0000 Subject: [all] Volunteer mailing list administrators In-Reply-To: References: <20191215170513.6pcwtlj2gce4xj3c@yuggoth.org> <20191217134053.oh2xyin52y7kv2vq@yuggoth.org> Message-ID: <20191217194352.ib5ygt5wvjfn3bwg@yuggoth.org> On 2019-12-17 20:17:04 +0100 (+0100), Radosław Piliszek wrote: > While I understand the reasoning behind the decision, me being me > I must throw in my 2 cents: I think you have chosen 3 folks in > much the same timezone. I might be mistaken but based on knowing > myself and mail domains of others I expect we are all in timezones > GMT+1/+2 It might be handy to have someone in timezones more > suited for China and USA as well. :-) [...] It can't hurt to add more folks, but so far we've gotten by for over a year with one moderator so having several in similar timezones to start out shouldn't be much of a regression. I tend to only look at the queue once or maybe twice a day anyway, at fairly random times, so timezone coverage hasn't seemed crucial for this as I've received no complaints yet about anything awaiting moderation. The greater concern is making sure there's someone around to look at it occasionally when others are on extended absence. The notifications senders get back when a message lands in moderation tends to be self-explanatory, and tells them that they can subscribe or trim the size of their message and then try again (and hopefully also cancel the earlier one) if it's urgent and they don't want to wait for a moderator to approve the previous attempt. In a couple of weeks I'll be back on duty during Americas daylight hours too, so maybe looking for a volunteer in an APAC timezone makes sense after the first of the year? It doesn't seem particularly urgent, but it's a fine suggestion. Thanks again! -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From Albert.Braden at synopsys.com Tue Dec 17 19:59:28 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Tue, 17 Dec 2019 19:59:28 +0000 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: My present employer is just starting with Openstack and we aren't using Ceilometer yet, but when I was at eBay we had a whole team trying to make it work and it was still terrible. I was very happy when I heard that the shitty old ceilometer was going away and would never be heard from again. As an operator, I want to see a clear path forward; a path that does not include going back to something that never worked properly. If there's a choice between doing it the "right" way, and doing something easier, we all know that doing it the "right" way will be easier in the long run. -----Original Message----- From: Thierry Carrez Sent: Tuesday, December 17, 2019 3:14 AM To: openstack-discuss at lists.openstack.org Subject: Re: [Telemetry][TC]Telemetry status I'd like to hear from more Ceilometer users. What are they using Ceilometer for, and what long-term plans would be acceptable. There is a trade-off between adopting short-term workarounds that reintroduce performance issues vs. undergoing a complex migration to the "right" way of fixing this. Like for example there is little point in pushing Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud seems to say, that they would rather have a slow performing Ceilometer API back. From mriedemos at gmail.com Tue Dec 17 20:51:08 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Tue, 17 Dec 2019 14:51:08 -0600 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? Message-ID: The ceph job on stable/pike has been broken for awhile [1]. The fix to make devstack-plugin-ceph (which is branchless) work on stable/pike depends on this devstack change on stable/pike to introduce the TARGET_BRANCH variable [2]. Question is, can we move [2] forward? If not, I make a motion that we drop ceph jobs from stable/pike (and likely stable/ocata if we are still running them there) because they won't work. The ceph job is non-voting anyway so that's why it goes unnoticed for so long when it's broken. If we aren't going to fix this stuff we shouldn't be wasting CI resources running broken jobs. Did I mention that you should all get off my lawn? :) [1] https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1835627 [2] https://review.opendev.org/#/c/684756/ -- Thanks, Matt From mgagne at calavera.ca Tue Dec 17 21:38:27 2019 From: mgagne at calavera.ca (=?UTF-8?Q?Mathieu_Gagn=C3=A9?=) Date: Tue, 17 Dec 2019 16:38:27 -0500 Subject: [searchlight] Elasticsearch 5 and latest support In-Reply-To: References: Message-ID: What would be the plan? 5.x is EOL. And 6.x introduces new concepts and remove some deprecated things from 5.x. I managed to make it (kind of) work with 5.x but then, I found out about some removed features in 6.x. (parent/child are now joins, one document type per index, etc.) There is also the migration path to consider. Should we tell people to start from scratch? I'm not sure how it would be possible to migrate existing data without reindexing everything. But I'm not an ES expert either. Mathieu On Tue, Dec 10, 2019 at 8:32 PM Trinh Nguyen wrote: > > Hi Mathieu, > > We just found a bug and I'm trying to fix it. We do have a plan to support a later version like 6.x or 7.x but It's become a little bit complicated when I just changed job a couple of months ago. If you or someone can give me a hand, it would be much appreciated. > > Bests, > > On Wed, Dec 11, 2019 at 3:35 AM Mathieu Gagné wrote: >> >> Hi, >> >> I'm reading the documentation for Searchlight and it's recommending >> ElasticSearch 5.x: >> https://docs.openstack.org/searchlight/latest/install/elasticsearch.html >> >> I installed and tried to use Elasticsearch 5 and it doesn't work. The >> queries sent by Searchlight aren't accepted by Elasticsearch. For >> examples: >> >> When listing facets: >> TransportError(400, u'parsing_exception', u'[terms] failed to parse >> field [size]'): RequestError: TransportError(400, >> u'parsing_exception', u'[terms] failed to parse field [size]') >> >> When searching for images: >> No registered plugin for type owner: Exception: No registered plugin >> for type owner >> >> Is there any work in progress to add support for Elasticsearch 5? >> In fact, Elasticsearch 5 is EOL. Are there plans to support later versions? >> >> Thanks >> >> -- >> Mathieu >> > > > -- > Trinh Nguyen > www.edlab.xyz > From cboylan at sapwetik.org Tue Dec 17 22:36:33 2019 From: cboylan at sapwetik.org (Clark Boylan) Date: Tue, 17 Dec 2019 14:36:33 -0800 Subject: [OSH][Infra] Open UDP port 111 on test nodes Message-ID: <50a97172-e1ba-479e-9e14-70897e51e5ae@www.fastmail.com> Hello, One of our contributing test node clouds has discovered that occasionally one of our test nodes will have udp port 111 open. The concern with this is that the RPC portmap service can be used in reflection DDoS attacks. Upon further investigation we've discovered that this seems to happen in OSH jobs (like openstack-helm-multinode-temp-ubuntu) [0] that run OSH's setup-firewall role [1]. These jobs do indeed disable the host firewall which would leave any running port mapper service exposed. It looks like these jobs run with multiple nodes in their nodeset, but do not use the multinode base job. I point this out because the multinode base job aims to set up networking and firewalls such that the nodes can talk freely among themselves while still blocking out the outside world. If we need to enable network communicate between hosts in these jobs this seems like a good place to start. That said there is a good chance that kubernetes may need additional open traffic. Additionally, I expect those specific depend on the CNI plugin that has been chosen? >From an infrastructure perspective we'd like to be good stewards of the resources donated to us and in this case that means preventing unwanted network traffic. We are more than happy to help set up more appropriate firewall rules if we can get details on what is needed. I expect the Zuul project is also interested and we can bake some of these common network needs for kubernetes into Zuul's zuul-job standard library. Can the OSH project work with us to fix this problem? Perhaps other kubernetes users/devs/operators can chime in on how they have reconciled host firewalls with kubernetes network needs? Any help that can be provided here would be much appreciated. [0] http://zuul.opendev.org/t/openstack/build/6fc5285fdb76484b894f0d288facdbb2/console#2/1/6/primary [1] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/setup-firewall/tasks/main.yaml Thank you, Clark From mvoelker at vmware.com Tue Dec 17 22:45:19 2019 From: mvoelker at vmware.com (Mark Voelker) Date: Tue, 17 Dec 2019 22:45:19 +0000 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: <4A303B46-89B1-40EB-B082-E2EE4F61E88E@vmware.com> On Dec 17, 2019, at 6:14 AM, Thierry Carrez > wrote: Zane Bitter wrote: On 15/12/19 10:20 pm, Rong Zhu wrote: 1.Add Ceilometer API back Since Gnocchi is out of OpenStack and is unmaintained, we need to add Ceilometer API back again. This is concerning because even the people who wrote it don't consider it adequate to the job. That inadequacy has been the source of significant reputational damage to OpenStack in the past, and many folks (me included) are anxious to avoid a repeat. Yes this concerns me too, and even if we workaround the issue by adding Ceilo API back, I'd like to have a long-term plan to solve this issue. It seems there are several options on the table (including integrating Monasca and Ceilometer into a single stack, and other big-bang replacements) but it's never a one-for-one comparison as the solutions seem to address slightly disjoint problem spaces. I'd like to hear from more Ceilometer users. What are they using Ceilometer for, and what long-term plans would be acceptable. There is a trade-off between adopting short-term workarounds that reintroduce performance issues vs. undergoing a complex migration to the "right" way of fixing this. Like for example there is little point in pushing Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud seems to say, that they would rather have a slow performing Ceilometer API back. Telemetry is part of the TC "Approved Release" that is eligible for the trademark program; I think at a minimum the TC will want to remove the resurrected Ceilometer API from the "designated sections" that users are required to run to participate in any trademark program that includes the functionality in question. But I think that we should explore other ways of reducing the chance of anyone confusing this for a viable way of building a cloud, including possibly changing the name (Antediluvian API?) and having this part of the stack live outside of the official OpenStack project. Legacy API? FWIW, telemetry projects are indeed eligible *for consideration*…but Ceilometer has never actually been required by any interop guideline: https://github.com/openstack/interop/blob/master/2019.11.json#L3218-L3222 It’s come up a few times over the years but was never deemed to have met enough of the criteria [1] to warrant inclusion in a guideline. [1] https://github.com/openstack/interop/blob/master/doc/source/process/CoreCriteria.rst At Your Service, Mark T. Voelker -- Thierry Carrez (ttx) -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangtrinhnt at gmail.com Wed Dec 18 01:06:07 2019 From: dangtrinhnt at gmail.com (Trinh Nguyen) Date: Wed, 18 Dec 2019 10:06:07 +0900 Subject: [searchlight] Elasticsearch 5 and latest support In-Reply-To: References: Message-ID: Hi Mathieu. Thanks for the effort. What you're saying is true. 5.x is EOL and may not be worth fixing. Adding support for 6.x could be a better option. Let me do some research on how to migrate data from 5.x to 6.x and let you know. Thanks, On Wed, Dec 18, 2019 at 6:38 AM Mathieu Gagné wrote: > What would be the plan? > > 5.x is EOL. And 6.x introduces new concepts and remove some deprecated > things from 5.x. > I managed to make it (kind of) work with 5.x but then, I found out > about some removed features in 6.x. (parent/child are now joins, one > document type per index, etc.) > > There is also the migration path to consider. Should we tell people to > start from scratch? > I'm not sure how it would be possible to migrate existing data without > reindexing everything. But I'm not an ES expert either. > > Mathieu > > On Tue, Dec 10, 2019 at 8:32 PM Trinh Nguyen > wrote: > > > > Hi Mathieu, > > > > We just found a bug and I'm trying to fix it. We do have a plan to > support a later version like 6.x or 7.x but It's become a little bit > complicated when I just changed job a couple of months ago. If you or > someone can give me a hand, it would be much appreciated. > > > > Bests, > > > > On Wed, Dec 11, 2019 at 3:35 AM Mathieu Gagné > wrote: > >> > >> Hi, > >> > >> I'm reading the documentation for Searchlight and it's recommending > >> ElasticSearch 5.x: > >> > https://docs.openstack.org/searchlight/latest/install/elasticsearch.html > >> > >> I installed and tried to use Elasticsearch 5 and it doesn't work. The > >> queries sent by Searchlight aren't accepted by Elasticsearch. For > >> examples: > >> > >> When listing facets: > >> TransportError(400, u'parsing_exception', u'[terms] failed to parse > >> field [size]'): RequestError: TransportError(400, > >> u'parsing_exception', u'[terms] failed to parse field [size]') > >> > >> When searching for images: > >> No registered plugin for type owner: Exception: No registered plugin > >> for type owner > >> > >> Is there any work in progress to add support for Elasticsearch 5? > >> In fact, Elasticsearch 5 is EOL. Are there plans to support later > versions? > >> > >> Thanks > >> > >> -- > >> Mathieu > >> > > > > > > -- > > Trinh Nguyen > > www.edlab.xyz > > > -- *Trinh Nguyen* *www.edlab.xyz * -------------- next part -------------- An HTML attachment was scrubbed... URL: From mgagne at calavera.ca Wed Dec 18 01:25:05 2019 From: mgagne at calavera.ca (=?UTF-8?Q?Mathieu_Gagn=C3=A9?=) Date: Tue, 17 Dec 2019 20:25:05 -0500 Subject: [searchlight] Elasticsearch 5 and latest support In-Reply-To: References: Message-ID: I don't mind reindexing from scratch. However my concern is the following: If people are already running Searchlight, they will be running ES 1.x or 2.x. (as ES 5.x and latest aren't supported atm by Searchlight) If we plan on updating Searchlight to support the latest supported ES versions, should we be concerned with the data migration for those existing users? If not, this could greatly speed up development as we wouldn't have to consider any form of backward compatibility or migration path. Also which versions should be supported? Would 6.x be enough? Should 7.x be targeted? Mathieu On Tue, Dec 17, 2019 at 8:06 PM Trinh Nguyen wrote: > > Hi Mathieu. > > Thanks for the effort. What you're saying is true. 5.x is EOL and may not be worth fixing. Adding support for 6.x could be a better option. > Let me do some research on how to migrate data from 5.x to 6.x and let you know. > > Thanks, > > > > On Wed, Dec 18, 2019 at 6:38 AM Mathieu Gagné wrote: >> >> What would be the plan? >> >> 5.x is EOL. And 6.x introduces new concepts and remove some deprecated >> things from 5.x. >> I managed to make it (kind of) work with 5.x but then, I found out >> about some removed features in 6.x. (parent/child are now joins, one >> document type per index, etc.) >> >> There is also the migration path to consider. Should we tell people to >> start from scratch? >> I'm not sure how it would be possible to migrate existing data without >> reindexing everything. But I'm not an ES expert either. >> >> Mathieu >> >> On Tue, Dec 10, 2019 at 8:32 PM Trinh Nguyen wrote: >> > >> > Hi Mathieu, >> > >> > We just found a bug and I'm trying to fix it. We do have a plan to support a later version like 6.x or 7.x but It's become a little bit complicated when I just changed job a couple of months ago. If you or someone can give me a hand, it would be much appreciated. >> > >> > Bests, >> > >> > On Wed, Dec 11, 2019 at 3:35 AM Mathieu Gagné wrote: >> >> >> >> Hi, >> >> >> >> I'm reading the documentation for Searchlight and it's recommending >> >> ElasticSearch 5.x: >> >> https://docs.openstack.org/searchlight/latest/install/elasticsearch.html >> >> >> >> I installed and tried to use Elasticsearch 5 and it doesn't work. The >> >> queries sent by Searchlight aren't accepted by Elasticsearch. For >> >> examples: >> >> >> >> When listing facets: >> >> TransportError(400, u'parsing_exception', u'[terms] failed to parse >> >> field [size]'): RequestError: TransportError(400, >> >> u'parsing_exception', u'[terms] failed to parse field [size]') >> >> >> >> When searching for images: >> >> No registered plugin for type owner: Exception: No registered plugin >> >> for type owner >> >> >> >> Is there any work in progress to add support for Elasticsearch 5? >> >> In fact, Elasticsearch 5 is EOL. Are there plans to support later versions? >> >> >> >> Thanks >> >> >> >> -- >> >> Mathieu >> >> >> > >> > >> > -- >> > Trinh Nguyen >> > www.edlab.xyz >> > > > > > -- > Trinh Nguyen > www.edlab.xyz > From tony at bakeyournoodle.com Wed Dec 18 02:03:26 2019 From: tony at bakeyournoodle.com (Tony Breeds) Date: Wed, 18 Dec 2019 13:03:26 +1100 Subject: [TC] 'V' Technical Elections In-Reply-To: References: Message-ID: <20191218020326.GC21598@thor.bakeyournoodle.com> On Wed, Dec 04, 2019 at 03:01:00PM -0800, Kendall Nelson wrote: > Hello! > > /me takes TC hat off and puts election official hat on > > Having run our handy dandy tool to project dates[1] for the elections, we > have some decisions to make again. /me catches up. If whomever wrote the election guessing tool had documented it things would look better Who was it .... and moving right along. [tony at thor election]$ (tox -e venv setup-election-config -- 2020-06-08 v-release TC ; tox -e venv setup-election-config -- 2020-05-13 v-release PTL) | pastebinit http://paste.openstack.org/show/787485 Which looks more like: TC Election from 2020-04-14T23:45 to 2020-04-21T23:45 TC Campaigning from 2020-04-07T23:45 to 2020-04-14T23:45 TC Nominations from 2020-03-31T23:45 to 2020-04-07T23:45 Set email_deadline to 2020-04-07T00:00 PTL Election from 2020-04-14T23:45 to 2020-04-21T23:45 PTL Nominations from 2020-04-07T23:45 to 2020-04-14T23:45 Set email_deadline to 2020-04-07T00:00 So if we Leave the email deadline at: 2020-04-07 00:00 and move the PTL election out to be the same time as the TC election then we trivially have a combined election. We don't need to move the extra-ATC date as that's already prior to the email deadline. Set email_deadline to 2020-04-07T00:00 Nominations from 2020-03-31T23:45 to 2020-04-07T23:45 Campaigning from 2020-04-07T23:45 to 2020-04-14T23:45 Election from 2020-04-14T23:45 to 2020-04-21T23:45 According to: https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_6c71f84caff2b37c The TC election closed on 06/03/2019, 10:45:23, so given a 16month term (as described in https://www.openstack.org/legal/technical-committee-member-policy/) we're okay to maintain the TC as is. The net result is that *current* PTLs have to serve for an "extra" week. /me will probably regret not reading the whole thread before replying Yours Tony. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From rico.lin.guanyu at gmail.com Wed Dec 18 03:46:36 2019 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Wed, 18 Dec 2019 11:46:36 +0800 Subject: [auto-scaling][self-healing] Discussion to merge two SIG to one In-Reply-To: References: <1b39c3fe-22a1-c84c-ed13-05fbd9360d7d@suse.com> Message-ID: To further push this task. I would like to propose we pick a new joint meeting schedule for both SIGs together. The first steps should be we share same meeting time and schedule, also share same event plan (as Witek suggested). And we can go from there to discuss if we need further plans. I also would like to suggest we move our meeting place to #openstack-meeting so we can have chance to have more people to join. Let's have a quick doodle polling for time, https://doodle.com/poll/98nrf8iibr7zv3kt Please join that doodle survey if you're interested in join us:) On Thu, Nov 28, 2019 at 4:57 PM Rico Lin wrote: > > > On Thu, Nov 28, 2019 at 4:37 PM Witek Bedyk wrote: > > > > Hi, > > how about starting with joining the SIGs meeting times and organizing > > the Forum and PTG events together? The repositories and wiki pages could > > stay as they are and refer to each other. > > > I think even if we merged two SIG, repositories should stay separated as > they're now. IMO we can simply rename openstack/auto-scaling-sig > to openstack/auto-scaling and so as to self-healing. > Or just keep it the same will be fine IMO. > We don't need a new repo for the new SIG (at least not for now). > > I do like the idea to start with joining the SIGs meeting times and > organizing the Forum and PTG events together. > One more proposal in my mind will be, join the channel for IRC. > > > > I think merging is good if you have an idea how to better structure the > > content, and time to review the existing one and do all the formal > > stuff. Just gluing the documents won't help. > Totally agree with this point! > > > > Cheers > > Witek > > > > > -- > May The Force of OpenStack Be With You, > Rico Lin > irc: ricolin > -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Wed Dec 18 04:26:18 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Wed, 18 Dec 2019 04:26:18 +0000 Subject: [TC] 'V' Technical Elections In-Reply-To: <20191218020326.GC21598@thor.bakeyournoodle.com> References: <20191218020326.GC21598@thor.bakeyournoodle.com> Message-ID: <20191218042617.wheqqmsmnka7qpt4@yuggoth.org> On 2019-12-18 13:03:26 +1100 (+1100), Tony Breeds wrote: [...] > Election from 2020-04-14T23:45 to 2020-04-21T23:45 > > According to: > https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_6c71f84caff2b37c > > The TC election closed on 06/03/2019, 10:45:23, so given a 16month term > (as described in > https://www.openstack.org/legal/technical-committee-member-policy/) > we're okay to maintain the TC as is. > > The net result is that *current* PTLs have to serve for an "extra" week. > > /me will probably regret not reading the whole thread before replying I'm a tl;dr it all for ya: bylaws say the TC can declare a term that's up to 16 months, but if they don't specify it prior to the election for that term (which they didn't) then it defaults to 12 and what you have there is (accounting for your curious way of writing 2019-03-05T23:45:23UTC) is a ~13.5 month term. In order to avoid needing a special election to fill the expired terms for ~6 weeks until the election, we need to pull it in by at least a couple weeks (because the charter will allow those seats to remain vacant if the open up within a month of a scheduled election). Clear as mud? ;) -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From sorrison at gmail.com Wed Dec 18 04:36:07 2019 From: sorrison at gmail.com (Sam Morrison) Date: Wed, 18 Dec 2019 15:36:07 +1100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: > On 17 Dec 2019, at 10:14 pm, Thierry Carrez wrote: > > Zane Bitter wrote: >> On 15/12/19 10:20 pm, Rong Zhu wrote: >>> 1.Add Ceilometer API back >>> Since Gnocchi is out of OpenStack and is unmaintained, we need to add Ceilometer API back again. >> This is concerning because even the people who wrote it don't consider it adequate to the job. That inadequacy has been the source of significant reputational damage to OpenStack in the past, and many folks (me included) are anxious to avoid a repeat. > > Yes this concerns me too, and even if we workaround the issue by adding Ceilo API back, I'd like to have a long-term plan to solve this issue. It seems there are several options on the table (including integrating Monasca and Ceilometer into a single stack, and other big-bang replacements) but it's never a one-for-one comparison as the solutions seem to address slightly disjoint problem spaces. > > I'd like to hear from more Ceilometer users. What are they using Ceilometer for, and what long-term plans would be acceptable. There is a trade-off between adopting short-term workarounds that reintroduce performance issues vs. undergoing a complex migration to the "right" way of fixing this. Like for example there is little point in pushing Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud seems to say, that they would rather have a slow performing Ceilometer API back. Nectar Cloud has been a ceilometer user from the early days. Well we tried to be and couldn’t use it as ceilometer api and mongo db just didn’t work at our scale. Gnocchi solved all these issues for us and we use ceilometer/aodh/gnocchi happily in production for several years now. If telemetry project is going down the path of the old days it will mean we will either drop ceilometer all together and look at alternative solutions like monasca or Prometheus etc. I just can’t see how the old architecture of ceilometer is ever going to be usable. If there is some confidence that gnocchi publisher will be supported in the future we would keep using gnocchi and just maintain it ourselves. It’s an open source project and I was hoping the openstack community could keep it going. We’d be happy to help maintain it at least. We use ceilometer/gnocchi to collect and store all metrics from openstack services. We have also written some custom pollsters and gnocchi is quite flexible here to allow this. With these metrics we build reports for our users, our operators, our funders (the government) Please reconsider your direction much like adding cpu_util back in (thank you for this!) Cheers, Sam > >> Telemetry is part of the TC "Approved Release" that is eligible for the trademark program; I think at a minimum the TC will want to remove the resurrected Ceilometer API from the "designated sections" that users are required to run to participate in any trademark program that includes the functionality in question. But I think that we should explore other ways of reducing the chance of anyone confusing this for a viable way of building a cloud, including possibly changing the name (Antediluvian API?) and having this part of the stack live outside of the official OpenStack project. > > Legacy API? > > -- > Thierry Carrez (ttx) > From melwittt at gmail.com Wed Dec 18 04:43:05 2019 From: melwittt at gmail.com (melanie witt) Date: Tue, 17 Dec 2019 20:43:05 -0800 Subject: [nova][ironic] need help understanding 'cpu_arch' in nova ironic driver In-Reply-To: References: Message-ID: On 12/13/19 06:18, Julia Kreger wrote: > On Fri, Dec 13, 2019 at 4:34 AM Dmitry Tantsur wrote: >> >> Thank you for starting this! Thank you both for your input on this issue! >> On Fri, Dec 13, 2019 at 2:48 AM melanie witt wrote: >>> >>> Hey all, >>> >>> Recently I'm trying to figure out how to fulfill a use case in the nova >>> ironic driver around treating an ironic node's 'cpu_arch' as optional. >>> >>> This is coming up as a result of a downstream issue [1] and recent >>> change on the ironic side [2] to make cpu_arch optional for iscsi >>> deployments. The change makes it so that ironic will _not_ include a >>> 'cpu_arch' attribute as part of a node's properties if iscsi. >>> >>> On the nova side, we have a filter scheduler ImagePropertiesFilter which >>> will only match a node if the architecture, hypervisor_type, and vm_mode >>> in the glance image properties match the cpu_arch of the node. We have >>> always pulled the cpu_arch of the from the ironic node properties since >>> the original ironic driver code was added to nova. >>> >>> Now, the use case I'm trying to solve today [3] is where an iscsi ironic >>> deployment has no cpu_arch specified on the ironic side and the deployer >>> wants to use glance images with architecture specified in their image >>> properties. Today (and historically always) the request to create an >>> instance in this situation will fail with NoValidHost because the nova >>> ironic driver could not determine a cpu_arch from the ironic node. >> >> >> I tend to think that if the image requires an explicit architecture, we should keep failing. However, hypervisor_type==baremetal should probably be a valid case (and I don't really know what vm_mode is, so cannot comment here). This makes sense to me and I agree. The more I thought about it, the more I was thinking that using an image with a specific architecture property is directly in opposition to the desire to have cpu_arch optional in the environment. If everything is single arch and the deployer will not specify cpu_arch in ironic, it should be consistent that they not specify arch in the glance image either. >>> My questions are: >>> >>> * Is this a valid thing to want to do? >>> >>> * If if is valid, what is the correct way that we should handle missing >>> cpu_arch in the nova ironic driver? Should we guess at a default >>> cpu_arch? If so, how? Or should we add a new config option where a >>> default cpu_arch can be specified? Or, other? >> >> >> Worst case, we can document cpu_arch as being required for nova. A bit inconvenient, but certainly not the end of the world. To be clear, it's only required for nova in the case of a glance image with a specific architecture included in the image properties. It's sounding to me like the proper solution in this use case is to _not_ set the architecture glance image property if the deployment prefers to treat the cpu_arch as optional. > Now that I understand where this entire discussion is coming from,.. > there seems to be a strong desire among some of the larger operators > (and kind of has been) to be able to have one ironic or as few ironics > to manage their hardware fleet as possible. So I'm not sure how well > it would work if it is always required that they create the cpu arch > field. Then again, most of these deployments use inspector to also do > wiring and system configuration validation prior to deployment, so > they would have cpu_arch then.... Hmmm, this is a conundrum. Yeah, I was thinking the same thing. If they want to avoid setting cpu_arch in ironic, but want to use glance images with architecture specified, they end up having to specify the cpu_arch in nova by way of config option (or something) and defeat the point of trying to avoid setting cpu_arch. Yet another reason it's sounding like for this use case, they should not specify 'architecture' in the glance image properties. >>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1653788 >>> [2] https://review.opendev.org/620634 >>> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1688838 >>> > From anlin.kong at gmail.com Wed Dec 18 05:37:35 2019 From: anlin.kong at gmail.com (Lingxian Kong) Date: Wed, 18 Dec 2019 18:37:35 +1300 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: Hi all, 1. Maybe Gnocchi is good, but it's not part of OpenStack, currently there is no one in the Telemetry team shows any interest in maintaining it, I'd be very happy to see if there is someone raises hand and say: I can commit to spend some time on this project, fixing bugs, maintaining document and answering questions either on IRC or via email. If yes, We will see how it goes in this dev cycle. 2. Monasca is also good, although it doesn't support auditing i.e. storing samples which is needed by billing purpose but I didn't see it's mentioned by anyone in this thread. What's more, it introduces kafaka which is not usually used in all other openstack core services. It's up to the cloud providers for the adoption, but at least it's not in our roadmap. 3. The Telemetry team did have discussed to bring the Ceilometer API and MongoDB support back, given neither Gnocchi nor Monasca is able to store the original samples. However, nothing is happening, so don't be panic. the current project core members are also the Telemetry service cloud providers, we know how important it is to not break anything, to not bring any more overhead than before. we are so glad to see this discussion, at least there are still so many people using Ceilometer/Gnocchi and have concerns about the current upstream situation. It would be much better that people involved in this discussion could participate in the design and implementation of the future tasks. Again, thanks for all the feedback and suggestions! - Best regards, Lingxian Kong Catalyst Cloud On Wed, Dec 18, 2019 at 5:41 PM Sam Morrison wrote: > > > > On 17 Dec 2019, at 10:14 pm, Thierry Carrez > wrote: > > > > Zane Bitter wrote: > >> On 15/12/19 10:20 pm, Rong Zhu wrote: > >>> 1.Add Ceilometer API back > >>> Since Gnocchi is out of OpenStack and is unmaintained, we need to > add Ceilometer API back again. > >> This is concerning because even the people who wrote it don't consider > it adequate to the job. That inadequacy has been the source of significant > reputational damage to OpenStack in the past, and many folks (me included) > are anxious to avoid a repeat. > > > > Yes this concerns me too, and even if we workaround the issue by adding > Ceilo API back, I'd like to have a long-term plan to solve this issue. It > seems there are several options on the table (including integrating Monasca > and Ceilometer into a single stack, and other big-bang replacements) but > it's never a one-for-one comparison as the solutions seem to address > slightly disjoint problem spaces. > > > > I'd like to hear from more Ceilometer users. What are they using > Ceilometer for, and what long-term plans would be acceptable. There is a > trade-off between adopting short-term workarounds that reintroduce > performance issues vs. undergoing a complex migration to the "right" way of > fixing this. Like for example there is little point in pushing > Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud > seems to say, that they would rather have a slow performing Ceilometer API > back. > > Nectar Cloud has been a ceilometer user from the early days. Well we tried > to be and couldn’t use it as ceilometer api and mongo db just didn’t work > at our scale. Gnocchi solved all these issues for us and we use > ceilometer/aodh/gnocchi happily in production for several years now. > If telemetry project is going down the path of the old days it will mean > we will either drop ceilometer all together and look at alternative > solutions like monasca or Prometheus etc. I just can’t see how the old > architecture of ceilometer is ever going to be usable. > > If there is some confidence that gnocchi publisher will be supported in > the future we would keep using gnocchi and just maintain it ourselves. It’s > an open source project and I was hoping the openstack community could keep > it going. We’d be happy to help maintain it at least. > > We use ceilometer/gnocchi to collect and store all metrics from openstack > services. We have also written some custom pollsters and gnocchi is quite > flexible here to allow this. With these metrics we build reports for our > users, our operators, our funders (the government) > > > Please reconsider your direction much like adding cpu_util back in (thank > you for this!) > > Cheers, > Sam > > > > > > >> Telemetry is part of the TC "Approved Release" that is eligible for the > trademark program; I think at a minimum the TC will want to remove the > resurrected Ceilometer API from the "designated sections" that users are > required to run to participate in any trademark program that includes the > functionality in question. But I think that we should explore other ways of > reducing the chance of anyone confusing this for a viable way of building a > cloud, including possibly changing the name (Antediluvian API?) and having > this part of the stack live outside of the official OpenStack project. > > > > Legacy API? > > > > -- > > Thierry Carrez (ttx) > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From missile0407 at gmail.com Wed Dec 18 05:43:38 2019 From: missile0407 at gmail.com (Eddie Yen) Date: Wed, 18 Dec 2019 13:43:38 +0800 Subject: [kolla][horizon] Get 503 Service unavailable after set RETRIEVE_PASSWORD. Message-ID: Hi, Below is the information about this environment. Openstack release: Rocky Kolla container source type: binary OS distro: Ubuntu 18.04 We tried to set OPENSTACK_ENABLE_PASSWORD_RETRIEVE to True in local_settings, but the dashboard got "503 Service Unavailable" after restart the Horizon container. And there's no error log or something in horizon.log Is there anyway to see what's going on? - Eddie -------------- next part -------------- An HTML attachment was scrubbed... URL: From licanwei_cn at 163.com Wed Dec 18 06:05:02 2019 From: licanwei_cn at 163.com (licanwei) Date: Wed, 18 Dec 2019 14:05:02 +0800 (GMT+08:00) Subject: [Watcher]IRC meeting at 8:00 UTC today Message-ID: <644af73d.e538.16f179c1550.Coremail.licanwei_cn@163.com> | | licanwei_cn | | 邮箱：licanwei_cn at 163.com | 签名由 网易邮箱大师 定制 -------------- next part -------------- An HTML attachment was scrubbed... URL: From petebirley at googlemail.com Tue Dec 17 23:04:21 2019 From: petebirley at googlemail.com (Pete Birley) Date: Tue, 17 Dec 2019 17:04:21 -0600 Subject: [OSH][Infra] Open UDP port 111 on test nodes In-Reply-To: <50a97172-e1ba-479e-9e14-70897e51e5ae@www.fastmail.com> References: <50a97172-e1ba-479e-9e14-70897e51e5ae@www.fastmail.com> Message-ID: Clark, This was originally meant to be a short term band-aid when we first implemented the gates, I'll look into the ports we actually require open and make adjustments as necessary. Once I've been able to improve things from our end, we can report back and see if we can get these into the zuul-job standard library. Cheers, Pete On Tue, 17 Dec 2019 at 16:38, Clark Boylan wrote: > Hello, > > One of our contributing test node clouds has discovered that occasionally > one of our test nodes will have udp port 111 open. The concern with this is > that the RPC portmap service can be used in reflection DDoS attacks. Upon > further investigation we've discovered that this seems to happen in OSH > jobs (like openstack-helm-multinode-temp-ubuntu) [0] that run OSH's > setup-firewall role [1]. > > These jobs do indeed disable the host firewall which would leave any > running port mapper service exposed. > > It looks like these jobs run with multiple nodes in their nodeset, but do > not use the multinode base job. I point this out because the multinode base > job aims to set up networking and firewalls such that the nodes can talk > freely among themselves while still blocking out the outside world. If we > need to enable network communicate between hosts in these jobs this seems > like a good place to start. > > That said there is a good chance that kubernetes may need additional open > traffic. Additionally, I expect those specific depend on the CNI plugin > that has been chosen? > > From an infrastructure perspective we'd like to be good stewards of the > resources donated to us and in this case that means preventing unwanted > network traffic. We are more than happy to help set up more appropriate > firewall rules if we can get details on what is needed. I expect the Zuul > project is also interested and we can bake some of these common network > needs for kubernetes into Zuul's zuul-job standard library. > > Can the OSH project work with us to fix this problem? Perhaps other > kubernetes users/devs/operators can chime in on how they have reconciled > host firewalls with kubernetes network needs? Any help that can be provided > here would be much appreciated. > > [0] > http://zuul.opendev.org/t/openstack/build/6fc5285fdb76484b894f0d288facdbb2/console#2/1/6/primary > [1] > https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/setup-firewall/tasks/main.yaml > > Thank you, > Clark > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aj at suse.com Wed Dec 18 08:18:30 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 09:18:30 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories Message-ID: The fuel project has been removed from from OpenStack in 2017 and has no action for some time, the fuel-ccp repositories are dead as well. I've talked with Sergey from Mirantis and he confirmed now that those can be retired. I'll take care of retiring these repositories now and will retire all fuel repos in OpenStack namespace as well as the fuel-ccp repositories in the x namespace. I'll use topic retire-fuel for the changes, Andreas P.S. full list of repos: openstack/fuel-agent openstack/fuel-astute openstack/fuel-dev-tools openstack/fuel-devops openstack/fuel-docs openstack/fuel-library openstack/fuel-main openstack/fuel-menu openstack/fuel-mirror openstack/fuel-nailgun-agent openstack/fuel-nailgun-extension-cluster-upgrade openstack/fuel-nailgun-extension-converted-serializers openstack/fuel-nailgun-extension-iac openstack/fuel-octane openstack/fuel-ostf openstack/fuel-plugin-murano openstack/fuel-plugin-murano-tests openstack/fuel-plugins openstack/fuel-qa openstack/fuel-specs openstack/fuel-stats openstack/fuel-ui openstack/fuel-upgrade openstack/fuel-virtualbox openstack/fuel-web openstack/python-fuelclient x/fuel-ccp x/fuel-ccp-ceph x/fuel-ccp-ci-config x/fuel-ccp-cinder x/fuel-ccp-debian-base x/fuel-ccp-designate x/fuel-ccp-elasticsearch x/fuel-ccp-entrypoint x/fuel-ccp-etcd x/fuel-ccp-galera x/fuel-ccp-glance x/fuel-ccp-grafana x/fuel-ccp-heat x/fuel-ccp-horizon x/fuel-ccp-installer x/fuel-ccp-ironic x/fuel-ccp-keystone x/fuel-ccp-mariadb x/fuel-ccp-memcached x/fuel-ccp-murano x/fuel-ccp-neutron x/fuel-ccp-nginx x/fuel-ccp-nova x/fuel-ccp-openstack-base x/fuel-ccp-rabbitmq x/fuel-ccp-rally x/fuel-ccp-sahara x/fuel-ccp-searchlight x/fuel-ccp-specs x/fuel-ccp-stacklight x/fuel-ccp-tests x/fuel-ccp-zmq -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From stig.openstack at telfer.org Wed Dec 18 09:02:22 2019 From: stig.openstack at telfer.org (Stig Telfer) Date: Wed, 18 Dec 2019 09:02:22 +0000 Subject: [scientific-sig] No IRC meeting today Message-ID: Hi All - Apologies, in the rush to end of year we have no agenda items for today. Given the season, I suggest we reconvene with a fresh start on Tuesday 7th January 2020. Cheers, Stig From aj at suse.com Wed Dec 18 09:04:33 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 10:04:33 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories In-Reply-To: References: Message-ID: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> Part of fuel was also openstack/tuning-box, We will retire this one as well, Andreas On 18/12/2019 09.18, Andreas Jaeger wrote: > The fuel project has been removed from from OpenStack in 2017 and has no > action for some time, the fuel-ccp repositories are dead as well. > > I've talked with Sergey from Mirantis and he confirmed now that those > can be retired. > > I'll take care of retiring these repositories now and will retire > all fuel repos in OpenStack namespace as well as the fuel-ccp > repositories in the x namespace. > > I'll use topic retire-fuel for the changes, > > Andreas > > P.S. full list of repos: > > openstack/fuel-agent > openstack/fuel-astute > openstack/fuel-dev-tools > openstack/fuel-devops > openstack/fuel-docs > openstack/fuel-library > openstack/fuel-main > openstack/fuel-menu > openstack/fuel-mirror > openstack/fuel-nailgun-agent > openstack/fuel-nailgun-extension-cluster-upgrade > openstack/fuel-nailgun-extension-converted-serializers > openstack/fuel-nailgun-extension-iac > openstack/fuel-octane > openstack/fuel-ostf > openstack/fuel-plugin-murano > openstack/fuel-plugin-murano-tests > openstack/fuel-plugins > openstack/fuel-qa > openstack/fuel-specs > openstack/fuel-stats > openstack/fuel-ui > openstack/fuel-upgrade > openstack/fuel-virtualbox > openstack/fuel-web > openstack/python-fuelclient > x/fuel-ccp > x/fuel-ccp-ceph > x/fuel-ccp-ci-config > x/fuel-ccp-cinder > x/fuel-ccp-debian-base > x/fuel-ccp-designate > x/fuel-ccp-elasticsearch > x/fuel-ccp-entrypoint > x/fuel-ccp-etcd > x/fuel-ccp-galera > x/fuel-ccp-glance > x/fuel-ccp-grafana > x/fuel-ccp-heat > x/fuel-ccp-horizon > x/fuel-ccp-installer > x/fuel-ccp-ironic > x/fuel-ccp-keystone > x/fuel-ccp-mariadb > x/fuel-ccp-memcached > x/fuel-ccp-murano > x/fuel-ccp-neutron > x/fuel-ccp-nginx > x/fuel-ccp-nova > x/fuel-ccp-openstack-base > x/fuel-ccp-rabbitmq > x/fuel-ccp-rally > x/fuel-ccp-sahara > x/fuel-ccp-searchlight > x/fuel-ccp-specs > x/fuel-ccp-stacklight > x/fuel-ccp-tests > x/fuel-ccp-zmq > -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From tobias.urdin at binero.se Wed Dec 18 09:12:07 2019 From: tobias.urdin at binero.se (Tobias Urdin) Date: Wed, 18 Dec 2019 10:12:07 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> As an operator I second pretty much everything Samsaid, using ceilometer-api never really worked without hand holding all the time. We migrated over to Gnocchi as that was "the way forward" from the Telemetry team and it has worked great. Gnocchi has a learning curve but after that it has been running flawlessly even at a larger scale, just introduced more workers and everything is set. I think the long term plan from the Telemetry team was to move out any storage abstraction and cultivate ceilometer to a single focus area around collecting metrics. Moving any API, transformations, storage etc to another service. I think it's said to see Gnocchi, the actual solutions to the problem, being unmaintained and out of the OpenStack developer ecosystem. I assume there is a cost to bringing it back in after it was moved out but maybe it's something that is needed? While I don't have a deep understand in Gnocchi I would have no choice but to try to spend more time learning it and fixing any issues that we might see since at this point we can't live without it, as our billing providers supports the Gnocchi API, we using Heat with Gnocchi and Aodh to autoscale etc. As a final note; thanks for bringing back the cpu_util metric, means I can drop the ugly customized code that was required to bring that metric back while it was removed :) Best regards Tobias On 12/18/19 5:39 AM, Sam Morrison wrote: > >> On 17 Dec 2019, at 10:14 pm, Thierry Carrez wrote: >> >> Zane Bitter wrote: >>> On 15/12/19 10:20 pm, Rong Zhu wrote: >>>> 1.Add Ceilometer API back >>>> Since Gnocchi is out of OpenStack and is unmaintained, we need to add Ceilometer API back again. >>> This is concerning because even the people who wrote it don't consider it adequate to the job. That inadequacy has been the source of significant reputational damage to OpenStack in the past, and many folks (me included) are anxious to avoid a repeat. >> Yes this concerns me too, and even if we workaround the issue by adding Ceilo API back, I'd like to have a long-term plan to solve this issue. It seems there are several options on the table (including integrating Monasca and Ceilometer into a single stack, and other big-bang replacements) but it's never a one-for-one comparison as the solutions seem to address slightly disjoint problem spaces. >> >> I'd like to hear from more Ceilometer users. What are they using Ceilometer for, and what long-term plans would be acceptable. There is a trade-off between adopting short-term workarounds that reintroduce performance issues vs. undergoing a complex migration to the "right" way of fixing this. Like for example there is little point in pushing Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud seems to say, that they would rather have a slow performing Ceilometer API back. > Nectar Cloud has been a ceilometer user from the early days. Well we tried to be and couldn’t use it as ceilometer api and mongo db just didn’t work at our scale. Gnocchi solved all these issues for us and we use ceilometer/aodh/gnocchi happily in production for several years now. > If telemetry project is going down the path of the old days it will mean we will either drop ceilometer all together and look at alternative solutions like monasca or Prometheus etc. I just can’t see how the old architecture of ceilometer is ever going to be usable. > > If there is some confidence that gnocchi publisher will be supported in the future we would keep using gnocchi and just maintain it ourselves. It’s an open source project and I was hoping the openstack community could keep it going. We’d be happy to help maintain it at least. > > We use ceilometer/gnocchi to collect and store all metrics from openstack services. We have also written some custom pollsters and gnocchi is quite flexible here to allow this. With these metrics we build reports for our users, our operators, our funders (the government) > > > Please reconsider your direction much like adding cpu_util back in (thank you for this!) > > Cheers, > Sam > > > >>> Telemetry is part of the TC "Approved Release" that is eligible for the trademark program; I think at a minimum the TC will want to remove the resurrected Ceilometer API from the "designated sections" that users are required to run to participate in any trademark program that includes the functionality in question. But I think that we should explore other ways of reducing the chance of anyone confusing this for a viable way of building a cloud, including possibly changing the name (Antediluvian API?) and having this part of the stack live outside of the official OpenStack project. >> Legacy API? >> >> -- >> Thierry Carrez (ttx) >> > > From veeraready at yahoo.co.in Wed Dec 18 09:12:47 2019 From: veeraready at yahoo.co.in (VeeraReddy) Date: Wed, 18 Dec 2019 09:12:47 +0000 (UTC) Subject: [openstack-dev][kuryr] Unable to connect to kube-apiserver References: <681164962.683857.1576660367576.ref@mail.yahoo.com> Message-ID: <681164962.683857.1576660367576@mail.yahoo.com> Hi , I install Kubernetes using “kubeadm init”.  Followed below link to install “kuryr-kubernetes”. https://docs.openstack.org/kuryr-kubernetes/latest/installation/manual.html#configure-kuryr-k8s-controller   Kube api configuration : /etc/kubernetes/manifests/kube-apiserver.yaml http://paste.openstack.org/show/787704/   kuryr config file : /etc/kuryr/kuryr.conf http://paste.openstack.org/show/787705/   Error while starting kuryr : #> kuryr-k8s-controller--config-file /etc/kuryr/kuryr.conf http://paste.openstack.org/show/787706/   Let me what I am missing in “/etc/kuryr/kuryr.conf”   Thanks, Veera. -------------- next part -------------- An HTML attachment was scrubbed... URL: From veera.b at nxp.com Wed Dec 18 09:10:15 2019 From: veera.b at nxp.com (Veera.reddy B) Date: Wed, 18 Dec 2019 09:10:15 +0000 Subject: [openstack-dev][kuryr] Unable to connect to kube-apiserver Message-ID: Hi , I install Kubernetes using "kubeadm init". Followed below link to install "kuryr-kubernetes". https://docs.openstack.org/kuryr-kubernetes/latest/installation/manual.html#configure-kuryr-k8s-controller Kube api configuration : /etc/kubernetes/manifests/kube-apiserver.yaml http://paste.openstack.org/show/787704/ kuryr config file : /etc/kuryr/kuryr.conf http://paste.openstack.org/show/787705/ Error while starting kuryr : #> kuryr-k8s-controller --config-file /etc/kuryr/kuryr.conf http://paste.openstack.org/show/787706/ Let me what I am missing in "/etc/kuryr/kuryr.conf" Thanks, Veera. -------------- next part -------------- An HTML attachment was scrubbed... URL: From neil at tigera.io Wed Dec 18 09:17:08 2019 From: neil at tigera.io (Neil Jerram) Date: Wed, 18 Dec 2019 09:17:08 +0000 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories In-Reply-To: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> Message-ID: Also x/fuel-plugin-calico can be retired. (If it isn't already? How can I tell if a repository is already retired?) The Calico team has no remaining interest in it, and I'm not aware of anyone else who does either. Best wishes, Neil On Wed, Dec 18, 2019 at 9:04 AM Andreas Jaeger wrote: > Part of fuel was also openstack/tuning-box, > > We will retire this one as well, > > Andreas > > On 18/12/2019 09.18, Andreas Jaeger wrote: > > The fuel project has been removed from from OpenStack in 2017 and has no > > action for some time, the fuel-ccp repositories are dead as well. > > > > I've talked with Sergey from Mirantis and he confirmed now that those > > can be retired. > > > > I'll take care of retiring these repositories now and will retire > > all fuel repos in OpenStack namespace as well as the fuel-ccp > > repositories in the x namespace. > > > > I'll use topic retire-fuel for the changes, > > > > Andreas > > > > P.S. full list of repos: > > > > openstack/fuel-agent > > openstack/fuel-astute > > openstack/fuel-dev-tools > > openstack/fuel-devops > > openstack/fuel-docs > > openstack/fuel-library > > openstack/fuel-main > > openstack/fuel-menu > > openstack/fuel-mirror > > openstack/fuel-nailgun-agent > > openstack/fuel-nailgun-extension-cluster-upgrade > > openstack/fuel-nailgun-extension-converted-serializers > > openstack/fuel-nailgun-extension-iac > > openstack/fuel-octane > > openstack/fuel-ostf > > openstack/fuel-plugin-murano > > openstack/fuel-plugin-murano-tests > > openstack/fuel-plugins > > openstack/fuel-qa > > openstack/fuel-specs > > openstack/fuel-stats > > openstack/fuel-ui > > openstack/fuel-upgrade > > openstack/fuel-virtualbox > > openstack/fuel-web > > openstack/python-fuelclient > > x/fuel-ccp > > x/fuel-ccp-ceph > > x/fuel-ccp-ci-config > > x/fuel-ccp-cinder > > x/fuel-ccp-debian-base > > x/fuel-ccp-designate > > x/fuel-ccp-elasticsearch > > x/fuel-ccp-entrypoint > > x/fuel-ccp-etcd > > x/fuel-ccp-galera > > x/fuel-ccp-glance > > x/fuel-ccp-grafana > > x/fuel-ccp-heat > > x/fuel-ccp-horizon > > x/fuel-ccp-installer > > x/fuel-ccp-ironic > > x/fuel-ccp-keystone > > x/fuel-ccp-mariadb > > x/fuel-ccp-memcached > > x/fuel-ccp-murano > > x/fuel-ccp-neutron > > x/fuel-ccp-nginx > > x/fuel-ccp-nova > > x/fuel-ccp-openstack-base > > x/fuel-ccp-rabbitmq > > x/fuel-ccp-rally > > x/fuel-ccp-sahara > > x/fuel-ccp-searchlight > > x/fuel-ccp-specs > > x/fuel-ccp-stacklight > > x/fuel-ccp-tests > > x/fuel-ccp-zmq > > > > > -- > Andreas Jaeger aj at suse.com Twitter: jaegerandi > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg > (HRB 36809, AG Nürnberg) GF: Felix Imendörffer > GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aj at suse.com Wed Dec 18 09:23:49 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 10:23:49 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories In-Reply-To: References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> Message-ID: On 18/12/2019 10.17, Neil Jerram wrote: > Also x/fuel-plugin-calico can be retired.  (If it isn't already?  How > can I tell if a repository is already retired?)  The Calico team has no > remaining interest in it, and I'm not aware of anyone else who does either. > Neil, do you want to do this following the process at: https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project Or shall I do it for you? Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From neil at tigera.io Wed Dec 18 09:25:40 2019 From: neil at tigera.io (Neil Jerram) Date: Wed, 18 Dec 2019 09:25:40 +0000 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories In-Reply-To: References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> Message-ID: On Wed, Dec 18, 2019 at 9:23 AM Andreas Jaeger wrote: > On 18/12/2019 10.17, Neil Jerram wrote: > > Also x/fuel-plugin-calico can be retired. (If it isn't already? How > > can I tell if a repository is already retired?) The Calico team has no > > remaining interest in it, and I'm not aware of anyone else who does > either. > > > > Neil, do you want to do this following the process at: > > https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project > > Or shall I do it for you? > > Andreas > If you don't mind, I'd be very happy for you to do it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdulko at redhat.com Wed Dec 18 09:33:58 2019 From: mdulko at redhat.com (=?UTF-8?Q?Micha=C5=82?= Dulko) Date: Wed, 18 Dec 2019 10:33:58 +0100 Subject: [openstack-dev][kuryr] Unable to connect to kube-apiserver In-Reply-To: References: Message-ID: <84bc9e1d4161c0608f0d3e58526c61a88f95fc8f.camel@redhat.com> Hi, It's RBAC issue - the account Kuryr uses to connect to K8s API doesn't have enough privileges. You should create an account for Kuryr that will have them. See [1] for the list of required ones. To login as particular user use options to specify certificates [2]. An alternative is to deploy Kuryr services as pods on that K8s cluster. In that case, a ServiceAccount definition with required privileges is created and Kuryr pods get attached to it, so they get a token allowing them to authenticate through that account. See [3] for more details on that. Thanks, Michał [1] https://opendev.org/openstack/kuryr-kubernetes/src/branch/master/devstack/lib/kuryr_kubernetes#L418-L456 [2] https://opendev.org/openstack/kuryr-kubernetes/src/branch/master/kuryr_kubernetes/config.py#L82-L87 [3] https://docs.openstack.org/kuryr-kubernetes/latest/installation/containerized.html On Wed, 2019-12-18 at 09:10 +0000, Veera.reddy B wrote: > Hi , > I install Kubernetes using “kubeadm init”. > > Followed below link to install “kuryr-kubernetes”. > https://docs.openstack.org/kuryr-kubernetes/latest/installation/manual.html#configure-kuryr-k8s-controller > > Kube api configuration : /etc/kubernetes/manifests/kube-apiserver.yaml > http://paste.openstack.org/show/787704/ > > kuryr config file : /etc/kuryr/kuryr.conf > http://paste.openstack.org/show/787705/ > > Error while starting kuryr : #> kuryr-k8s-controller --config-file /etc/kuryr/kuryr.conf > http://paste.openstack.org/show/787706/ > > Let me what I am missing in “/etc/kuryr/kuryr.conf” > > Thanks, > Veera. > > From tobias.urdin at binero.se Wed Dec 18 09:57:12 2019 From: tobias.urdin at binero.se (Tobias Urdin) Date: Wed, 18 Dec 2019 10:57:12 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> Message-ID: <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> https://github.com/gnocchixyz/gnocchi/issues/1049#issuecomment-555768072 On 12/18/19 10:15 AM, Tobias Urdin wrote: > As an operator I second pretty much everything Samsaid, using > ceilometer-api never really worked without hand holding all the time. > We migrated over to Gnocchi as that was "the way forward" from the > Telemetry team and it has worked great. > > Gnocchi has a learning curve but after that it has been running > flawlessly even at a larger scale, just introduced more workers and > everything is set. > > I think the long term plan from the Telemetry team was to move out any > storage abstraction and cultivate ceilometer to a single focus area around > collecting metrics. Moving any API, transformations, storage etc to > another service. > > I think it's said to see Gnocchi, the actual solutions to the problem, > being unmaintained and out of the OpenStack developer ecosystem. I > assume there > is a cost to bringing it back in after it was moved out but maybe it's > something that is needed? > > While I don't have a deep understand in Gnocchi I would have no choice > but to try to spend more time learning it and fixing any issues that we > might > see since at this point we can't live without it, as our billing > providers supports the Gnocchi API, we using Heat with Gnocchi and Aodh > to autoscale etc. > > As a final note; thanks for bringing back the cpu_util metric, means I > can drop the ugly customized code that was required to bring that metric > back while > it was removed :) > > Best regards > Tobias > > On 12/18/19 5:39 AM, Sam Morrison wrote: >>> On 17 Dec 2019, at 10:14 pm, Thierry Carrez wrote: >>> >>> Zane Bitter wrote: >>>> On 15/12/19 10:20 pm, Rong Zhu wrote: >>>>> 1.Add Ceilometer API back >>>>> Since Gnocchi is out of OpenStack and is unmaintained, we need to add Ceilometer API back again. >>>> This is concerning because even the people who wrote it don't consider it adequate to the job. That inadequacy has been the source of significant reputational damage to OpenStack in the past, and many folks (me included) are anxious to avoid a repeat. >>> Yes this concerns me too, and even if we workaround the issue by adding Ceilo API back, I'd like to have a long-term plan to solve this issue. It seems there are several options on the table (including integrating Monasca and Ceilometer into a single stack, and other big-bang replacements) but it's never a one-for-one comparison as the solutions seem to address slightly disjoint problem spaces. >>> >>> I'd like to hear from more Ceilometer users. What are they using Ceilometer for, and what long-term plans would be acceptable. There is a trade-off between adopting short-term workarounds that reintroduce performance issues vs. undergoing a complex migration to the "right" way of fixing this. Like for example there is little point in pushing Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud seems to say, that they would rather have a slow performing Ceilometer API back. >> Nectar Cloud has been a ceilometer user from the early days. Well we tried to be and couldn’t use it as ceilometer api and mongo db just didn’t work at our scale. Gnocchi solved all these issues for us and we use ceilometer/aodh/gnocchi happily in production for several years now. >> If telemetry project is going down the path of the old days it will mean we will either drop ceilometer all together and look at alternative solutions like monasca or Prometheus etc. I just can’t see how the old architecture of ceilometer is ever going to be usable. >> >> If there is some confidence that gnocchi publisher will be supported in the future we would keep using gnocchi and just maintain it ourselves. It’s an open source project and I was hoping the openstack community could keep it going. We’d be happy to help maintain it at least. >> >> We use ceilometer/gnocchi to collect and store all metrics from openstack services. We have also written some custom pollsters and gnocchi is quite flexible here to allow this. With these metrics we build reports for our users, our operators, our funders (the government) >> >> >> Please reconsider your direction much like adding cpu_util back in (thank you for this!) >> >> Cheers, >> Sam >> >> >> >>>> Telemetry is part of the TC "Approved Release" that is eligible for the trademark program; I think at a minimum the TC will want to remove the resurrected Ceilometer API from the "designated sections" that users are required to run to participate in any trademark program that includes the functionality in question. But I think that we should explore other ways of reducing the chance of anyone confusing this for a viable way of building a cloud, including possibly changing the name (Antediluvian API?) and having this part of the stack live outside of the official OpenStack project. >>> Legacy API? >>> >>> -- >>> Thierry Carrez (ttx) >>> >> > > From rdopiera at redhat.com Wed Dec 18 10:17:14 2019 From: rdopiera at redhat.com (Radek Dopieralski) Date: Wed, 18 Dec 2019 11:17:14 +0100 Subject: [kolla][horizon] Get 503 Service unavailable after set RETRIEVE_PASSWORD. In-Reply-To: References: Message-ID: Most likely you have a syntax error in your settings somehow. Please check the container's logs (with "sudo podman logs horizon" on the controller node) to see the exact error. On Wed, Dec 18, 2019 at 6:51 AM Eddie Yen wrote: > Hi, > > Below is the information about this environment. > > Openstack release: Rocky > Kolla container source type: binary > OS distro: Ubuntu 18.04 > > We tried to set OPENSTACK_ENABLE_PASSWORD_RETRIEVE to True in > local_settings, but the dashboard got "503 Service Unavailable" after > restart the Horizon container. > And there's no error log or something in horizon.log > > Is there anyway to see what's going on? > > - Eddie > -- Radomir Dopieralski -------------- next part -------------- An HTML attachment was scrubbed... URL: From missile0407 at gmail.com Wed Dec 18 10:58:21 2019 From: missile0407 at gmail.com (Eddie Yen) Date: Wed, 18 Dec 2019 18:58:21 +0800 Subject: [kolla][horizon] Get 503 Service unavailable after set RETRIEVE_PASSWORD. In-Reply-To: References: Message-ID: Hi Radek, I found that it just took compress so long after restart the horizon container. Because it stuck at "manage.py compress --force" so long when check the log. Waiting about 3~5 minutes and it came back, and the option shows up. Thanks for your hint! - Eddie Radek Dopieralski 於 2019年12月18日 週三 下午6:17寫道： > Most likely you have a syntax error in your settings somehow. Please check > the container's logs (with "sudo podman logs horizon" on the controller > node) to see the exact error. > > On Wed, Dec 18, 2019 at 6:51 AM Eddie Yen wrote: > >> Hi, >> >> Below is the information about this environment. >> >> Openstack release: Rocky >> Kolla container source type: binary >> OS distro: Ubuntu 18.04 >> >> We tried to set OPENSTACK_ENABLE_PASSWORD_RETRIEVE to True in >> local_settings, but the dashboard got "503 Service Unavailable" after >> restart the Horizon container. >> And there's no error log or something in horizon.log >> >> Is there anyway to see what's going on? >> >> - Eddie >> > > > -- > Radomir Dopieralski > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aj at suse.com Wed Dec 18 11:16:27 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 12:16:27 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories - also x/fuel-plugin-calico In-Reply-To: References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> Message-ID: <6d40d2b2-92db-2c31-e852-067aeebba0ef@suse.com> On 18/12/2019 10.25, Neil Jerram wrote: > On Wed, Dec 18, 2019 at 9:23 AM Andreas Jaeger > wrote: > > On 18/12/2019 10.17, Neil Jerram wrote: > > Also x/fuel-plugin-calico can be retired.  (If it isn't already?  How > > can I tell if a repository is already retired?)  The Calico team > has no > > remaining interest in it, and I'm not aware of anyone else who > does either. > > > > Neil, do you want to do this following the process at: > > https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project > > Or shall I do it for you? > > Andreas > > > If you don't mind, I'd be very happy for you to do it. I've just pushed the patches up for review. Neil, please approve https://review.opendev.org/699662 and abandon all open reviews. Once that is done, we can retire it with https://review.opendev.org/699664 andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From aj at suse.com Wed Dec 18 11:27:37 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 12:27:37 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories - what about fuel-plugins? In-Reply-To: References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> Message-ID: <57d59b89-ec82-2181-aeb0-40b2e9abf96a@suse.com> Since fuel is retired, I wonder about all the fuel-plugin repositories that still exist in the "x" namespace. If any team wants a single repo to have retired, please tell me and I'll help. Should we retire all of them? There was no single merge to them since October 2017 (with exception of Ian's force-merge of URL replacements), see https://review.opendev.org/#/q/projects:x/fuel-plugin+is:merged Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From lyarwood at redhat.com Wed Dec 18 11:30:15 2019 From: lyarwood at redhat.com (Lee Yarwood) Date: Wed, 18 Dec 2019 11:30:15 +0000 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: References: Message-ID: <20191218113015.ppyqla4kjljjhfmx@lyarwood.usersys.redhat.com> On 17-12-19 14:51:08, Matt Riedemann wrote: > The ceph job on stable/pike has been broken for awhile [1]. > > The fix to make devstack-plugin-ceph (which is branchless) work on > stable/pike depends on this devstack change on stable/pike to introduce the > TARGET_BRANCH variable [2]. > > Question is, can we move [2] forward? If not, I make a motion that we drop > ceph jobs from stable/pike (and likely stable/ocata if we are still running > them there) because they won't work. > > The ceph job is non-voting anyway so that's why it goes unnoticed for so > long when it's broken. If we aren't going to fix this stuff we shouldn't be > wasting CI resources running broken jobs. > > Did I mention that you should all get off my lawn? :) > > [1] https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1835627 > [2] https://review.opendev.org/#/c/684756/ I'm +1 on dropping the job on stable/pike if we can't push [2] forward. This was always going to happen with EM branches anyway right? -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From sfinucan at redhat.com Wed Dec 18 12:02:39 2019 From: sfinucan at redhat.com (Stephen Finucane) Date: Wed, 18 Dec 2019 12:02:39 +0000 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: References: Message-ID: On Tue, 2019-12-17 at 14:51 -0600, Matt Riedemann wrote: > The ceph job on stable/pike has been broken for awhile [1]. > > The fix to make devstack-plugin-ceph (which is branchless) work on > stable/pike depends on this devstack change on stable/pike to introduce > the TARGET_BRANCH variable [2]. > > Question is, can we move [2] forward? If not, I make a motion that we > drop ceph jobs from stable/pike (and likely stable/ocata if we are still > running them there) because they won't work. Drop em. If ceph was actually broken on these branches, we'd have been told by now. This will only be an issue if we backport ceph'y bugfixes that inadvertently break stuff, but who's doing that on these branches now? Stephen > The ceph job is non-voting anyway so that's why it goes unnoticed for so > long when it's broken. If we aren't going to fix this stuff we shouldn't > be wasting CI resources running broken jobs. > > Did I mention that you should all get off my lawn? :) > > [1] https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1835627 > [2] https://review.opendev.org/#/c/684756/ > From witold.bedyk at suse.com Wed Dec 18 12:19:19 2019 From: witold.bedyk at suse.com (Witek Bedyk) Date: Wed, 18 Dec 2019 13:19:19 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: <742f67fb-8b47-9cca-ef34-dc8fd54e8d0c@suse.com> Hi, we have held a presentation in Shanghai on that topic if that's of help for anyone: https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/23932/transitioning-your-cloud-monitoring-from-ceilometer-to-monasca We've compared the technology of Telemetry and Monasca and how specific use cases like billing or auto-scaling are implemented. Monasca is a valid replacement option for Gnocchi. On 12/18/19 6:37 AM, Lingxian Kong wrote: > 2. Monasca is also good, although it doesn't support auditing i.e. > storing samples which is needed by billing purpose but I didn't see it's > mentioned by anyone in this thread. I have to disagree with that. Monasca is used for billing in production. > What's more, it introduces kafaka > which is not usually used in all other openstack core services. Yes, we decided to use Apache Kafka as it's better suited than RabbitMQ to pass large loads of data with high throughput. Building the system around Kafka allows us to provide fault-tolerance and scalability. Best greetings Witek From thierry at openstack.org Wed Dec 18 13:03:04 2019 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 18 Dec 2019 14:03:04 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: Lingxian Kong wrote: > [...] > 1. Maybe Gnocchi is good, but it's not part of OpenStack, currently > there is no one in the Telemetry team shows any interest in maintaining > it, I'd be very happy to see if there is someone raises hand and say: I can > commit to spend some time on this project, fixing bugs, maintaining > document and answering questions either on IRC or via email. If yes, We > will see how it goes in this dev cycle. Is there any specific standing issue with Gnocchi that you are blocked on ? Or is it more of a general concern that you should not depend on an unmaintained project ? If it's the latter (and there is no real burning issue with it), it feels like forking or taking over Gnocchi, and then keep it barely functional enough for OpenStack use case might not be that much of work (compared to other options) and we might find volunteers in this thread to help. To me that would be a better short-term plan, while some lower-maintenance long-term solution is designed (be it combining Monasca and Ceilometer, or starting from scratch and leveraging some new cool tech). -- Thierry Carrez (ttx) From neil at tigera.io Wed Dec 18 13:22:15 2019 From: neil at tigera.io (Neil Jerram) Date: Wed, 18 Dec 2019 13:22:15 +0000 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories - also x/fuel-plugin-calico In-Reply-To: <6d40d2b2-92db-2c31-e852-067aeebba0ef@suse.com> References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> <6d40d2b2-92db-2c31-e852-067aeebba0ef@suse.com> Message-ID: On Wed, Dec 18, 2019 at 11:16 AM Andreas Jaeger wrote: > On 18/12/2019 10.25, Neil Jerram wrote: > > On Wed, Dec 18, 2019 at 9:23 AM Andreas Jaeger > > wrote: > > > > On 18/12/2019 10.17, Neil Jerram wrote: > > > Also x/fuel-plugin-calico can be retired. (If it isn't already? > How > > > can I tell if a repository is already retired?) The Calico team > > has no > > > remaining interest in it, and I'm not aware of anyone else who > > does either. > > > > > > > Neil, do you want to do this following the process at: > > > > > https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project > > > > Or shall I do it for you? > > > > Andreas > > > > > > If you don't mind, I'd be very happy for you to do it. > > I've just pushed the patches up for review. > Thanks. > > Neil, please approve https://review.opendev.org/699662 and abandon all > open reviews. > I've done those now. > > Once that is done, we can retire it with https://review.opendev.org/699664 > > andreas > -- > Andreas Jaeger aj at suse.com Twitter: jaegerandi > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg > (HRB 36809, AG Nürnberg) GF: Felix Imendörffer > GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Wed Dec 18 13:41:45 2019 From: thierry at openstack.org (Thierry Carrez) Date: Wed, 18 Dec 2019 14:41:45 +0100 Subject: [largescale-sig] Meeting summary and next actions Message-ID: <3c3a6232-9a3b-d240-ab82-c7ac4997f5c0@openstack.org> Hi everyone, The Large Scale SIG held a meeting earlier today. You can access the summary and logs of the meeting at: http://eavesdrop.openstack.org/meetings/large_scale_sig/2019/large_scale_sig.2019-12-18-09.00.html This meeting was focused on immediate next steps on the two initial goals for the SIG. For the "Scaling within one cluster, and instrumentation of the bottlenecks" goal, after the holidays we will collect user stories of what happens (or what breaks first) as you scale up single clusters. We will also produce a first RFC draft for an oslo.metric blueprint, and learn more about the "Golden signals" concept. Standing TODOs: - all prepare a short description of what happens (what breaks first) when scaling up a single cluster - ttx to prepare etherpad and send ML thread asking for user scaling stories, to be posted after the end-of-year holidays - masahito to produce first draft for the oslo.metric blueprint - all learn more about golden signals concept as described in https://landing.google.com/sre/book.html For the "Document large scale configuration and tips &tricks" goal we will start collecting relevant articles and blogposts. We agreed that in order to properly document large scale configuration defaults, we should probably both patch existing docs and create new documentation. Standing TODOs: - all add links to relevant articles around large scale openstack (if you find any) to the etherpad - oneswig to follow up with Scientific community to find such articles - amorin to start a thread on documenting configuration defaults for large scale, introduce the "mixture of both" tactic The next meeting will happen on January 15, at 9:00 UTC on #openstack-meeting. Cheers, -- Thierry Carrez (ttx) From i at liuyulong.me Wed Dec 18 13:50:15 2019 From: i at liuyulong.me (=?utf-8?B?TElVIFl1bG9uZw==?=) Date: Wed, 18 Dec 2019 21:50:15 +0800 Subject: [neutron] Cancle neutron L3 meeting Message-ID: Hi all, Due to some local reasons, I'm not be able to host the L3 meeting today. So let's cancel it. And also because contributors from most countries will have the Christmas and new year's holidays in the next two weeks, let's also cancel the following two L3 meetings. So the next L3 meeting will be raised on 8th Jan, 2020. OK, see you guys then.  And happy new year! Regards, LIU Yulong -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Wed Dec 18 13:58:48 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Wed, 18 Dec 2019 07:58:48 -0600 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: <20191218113015.ppyqla4kjljjhfmx@lyarwood.usersys.redhat.com> References: <20191218113015.ppyqla4kjljjhfmx@lyarwood.usersys.redhat.com> Message-ID: On 12/18/2019 5:30 AM, Lee Yarwood wrote: > This was always going to happen with EM branches anyway right? EM doesn't mean we just drop stuff if there is a reasonable way to keep it working, which we have. EM just allows us to officially no longer care if things aren't working and no one steps up to fix them. I did the latter several months ago but if I'm the only one that cares then it's not really sustainable. -- Thanks, Matt From mriedemos at gmail.com Wed Dec 18 14:01:43 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Wed, 18 Dec 2019 08:01:43 -0600 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: References: Message-ID: <76c034a9-75aa-d086-a120-79654d5d80ec@gmail.com> On 12/18/2019 6:02 AM, Stephen Finucane wrote: > If ceph was actually broken on these branches, we'd have been > told by now. This will only be an issue if we backport ceph'y bugfixes > that inadvertently break stuff, but who's doing that on these branches > now? True that I suspect most ceph-related fixes to nova would come through Red Hat people and Red Hat people don't care about anything past Queens (which is also EM). Having said that, stable/pike has been one of our more active branches lately (in nova anyway) because of gibi and elod pushing fixes there because their product at Ericsson is based on Pike. Note that stable/ocata is still running a working ceph job (I haven't dug into why it's working but the pike job isn't, maybe something to do with how the devstack-plugin-ceph repo is used in the legacy ocata job), so it'd be kind of weird to have no ceph testing on pike but have it on ocata. Maybe we should drop from both branches - it's non-voting anyway (on *all* branches). -- Thanks, Matt From emilien at redhat.com Wed Dec 18 15:35:10 2019 From: emilien at redhat.com (Emilien Macchi) Date: Wed, 18 Dec 2019 10:35:10 -0500 Subject: [tripleo] Gate is unstable, do not recheck/approve anything now Message-ID: We're encountering a bug in TripleO, see https://bugs.launchpad.net/tripleo/+bug/1856864 In the meantime please do not recheck or approve a patch, the gate is already fully loaded; we'll purge it, fix the bug and recheck what needs to be done. You can follow the progress of that work on our IRC channel #tripleo. Thanks for your patience. -- Emilien Macchi -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Wed Dec 18 16:22:20 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 18 Dec 2019 11:22:20 -0500 Subject: [cinder] next meeting 8 January 2020 Message-ID: <024b170c-c05b-bfe4-4969-9e14c267522e@gmail.com> As discussed during today's meeting, we will *not* hold the Cinder weekly meeting over the next 2 weeks. The next meeting will be on 8 January 2020. https://etherpad.openstack.org/p/cinder-ussuri-meetings cheers, brian From mordred at inaugust.com Wed Dec 18 16:59:28 2019 From: mordred at inaugust.com (Monty Taylor) Date: Wed, 18 Dec 2019 11:59:28 -0500 Subject: [all][tripleo][openstack-ansible] Openstack Ansible modules - how to move the current modules? In-Reply-To: References: Message-ID: > On Dec 16, 2019, at 11:46 AM, Sagi Shnaidman wrote: > > Hi, all > > recently we had an Openstack Ansible modules meeting, when were discussing how we will manage current modules and new planned modules. Because it was a hot topic which took most of the time and no agreement was reached, I think it's worth to discuss it here in ML. > > Options that have been raised in the discussion[1]: > 1) To freeze current modules and start writing new modules from scratch > 2) To freeze current modules and based on them develop new modules > 3) To continue to work on current modules and change them step by step I believe we should, generally speaking, do this. The ansible folks have some things in place (or plan to have some things in place) that should make this reorg otherwise transparent, so I don’t think we should just change things on users. I followed up with gundalow about how things are going to work - here is a summary: In ansible/ansible there is a file called BOTMETA.yml. In that file, we’ll put in entries like: $modules/cloud/openstack/openstack/os_server: migrated_to: openstack.cloud.server This lets us map old module names `os_server` to new collection paths `openstack.cloud.server`. When the ansible source tarball or wheel are made, BOTMETA will be consulted, needed collections will be downloaded and symlinks will be made from the old module names to the new module names. So - for a user who runs “pip install ansible” (and presumably also dnf install ansible or apt-get install ansible) - they will get things installed as they always have been and they will not need to break any of their playbooks. This is intended to be done in perpetuity. The lifecycle of our new modules n the collection is a continuation of the existing modules - so I believe we should take care to maintain backwards compat. Also - since someone asked at the meeting - yes, this includes things like the dynamic inventory plugin - so we’ll be moving all of the content over. It is currently undefined what the behavior will be if someone says “pip install ansible ; ansible-galaxy collection install openstack.cloud” For sure doing that and then referencing openstack.cloud.server in a playbook will get the new content. It is undefined as of now whether that will update os_server. Toshio is working on defining this, so if we have opinions we should talk to him about it. Because we have the renaming capability via BOTMETA - we get a nice opportunity to clean things up in our collection. For that reason, I believe we are all agreed to: * Drop os_ underscore in the collection version * Drop service_type prefix on ‘admin” modules So that will have: * os_server -> oppenstack.cloud.server * os_ironic_node -> openstack.cloud.baremetal_node (or something better) NOW - there are some modules we may need to be more aggressive with as far as changes go. I think for those, we have a few options: - Do the breaking changes behind a feature flag in the parameters, set a deprecation on the old behavior and after 2 releases change the default on the feature flag. - Make a completely new module and deprecate the old one. But I think we should take each case of that as it comes as a specific case. The one thing that has been brought up specifically is Dmitry mentioned we may need to improve the auth story for standalone ironic. That may be true - but I’m not yet convinced this needs to be breaking. Yet. But if it needs to be - we can solve it. Because we have control over the whole collection - we can do a better job with module_utils - and we can make additional constructor methods if needed. I’ve got a patch up: https://review.opendev.org/#/c/698044/ With an example of making a base class instead of a factory function. It might not yet be the right design, but hopefully it shows how we can clean some things up as we move forward. > 4) In case of freezing current modules, deprecate them later > > Things to consider: > 1) People are using current modules in playbooks and we don't want to break them, so current modules interfaces should stay available and not change for not breaking backward compatibility > 2) We might redesign some of modules, including big changes in common modules parts > 3) We might redistribute current module functionality over other and new modules > > I think it can be a start for the discussion on how to move further, please comment. > > Thanks > > [1] http://eavesdrop.openstack.org/meetings/api_sig/2019/api_sig.2019-12-12-16.00.log.html From aj at suse.com Wed Dec 18 17:59:54 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 18:59:54 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories - also x/fuel-plugin-calico In-Reply-To: References: <2d8f7270-0cbe-c25c-484d-33dbef3f06d7@suse.com> <6d40d2b2-92db-2c31-e852-067aeebba0ef@suse.com> Message-ID: On 18/12/2019 14.22, Neil Jerram wrote: > [...] > I've done those now.Thanks. The project-config change is merged, so all is fine now, Andreas -- -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From aj at suse.com Wed Dec 18 18:22:55 2019 From: aj at suse.com (Andreas Jaeger) Date: Wed, 18 Dec 2019 19:22:55 +0100 Subject: [fuel] fuel-plugins, x/network-checker, x/fuel-noop-fixtures retirement Message-ID: <0e6c1346-e069-fc08-2032-ef777c2d7985@suse.com> Since we have now the fuel repositories retired, I propose to retire the fuel repos in the x/ namespace as well: all the plugins, network-checker and fuel-noop-fixuters There was no single merge to them since October 2017 (with exception of Ian's force-merge of URL replacements), see https://review.opendev.org/#/q/(projects:x/fuel-plugin+OR+project:x/network-checker+OR+project:x/fuel-noop-fixtures)+is:merged I will push changes to repositories and would appreciate if teams merge them and abandon all open reviews. If anybody wants to keep the repos, please speak up here and/or -1 my changes. I'll use topic retire-x-fuel, https://review.opendev.org/#/q/topic:retire-x-fuel I'll just propose changes for now, we'll force-merge them in January 2020, Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From opensrloo at gmail.com Wed Dec 18 18:30:52 2019 From: opensrloo at gmail.com (Ruby Loo) Date: Wed, 18 Dec 2019 13:30:52 -0500 Subject: [ironic] ibmc driver In-Reply-To: References: Message-ID: On Fri, Dec 13, 2019 at 9:18 AM Julia Kreger wrote: > On Fri, Dec 13, 2019 at 4:24 AM Dmitry Tantsur > wrote: > > > > Hi, > > > > On Thu, Dec 12, 2019 at 10:30 PM Julia Kreger < > juliaashleykreger at gmail.com> wrote: > >> > >> Greetings fellow ironicans and stackers! > >> > >> I'd like some thoughts on the `ibmc` driver[1] that was added early > >> this year. Specifically it merged into ironic with an operating > >> Third-Party CI, however soon after a series of unfortunate events > >> occurred where the Third-Party CI stopped reporting sometime in early > >> May, and shortly their after restrictions were put into place[2]. With > >> that being said, I feel like we should go ahead and remove this driver > >> since it does not seem to be maintained nor tested. > > > > > > Do you mean deprecate and remove in V or just straight remove? I'd > follow the regular procedure. Otherwise, it seems that we have no other > options :( > > Either... Kind of why I emailed the mailing list. :) I'd lean for > deprecated and take that route, but maybe someone has interest in > keeping it working > ++ Let's follow the regular procedure unless [2] has ramifications that I don't understand (or maybe don't want to understand). I don't recall the regular procedure though. Was it: deprecate in cycle A (and notify community to allow someones to step up to support), and remove in cycle A+1? If so, let's deprecate now, in Train. We can un-deprecate if someone comes forward. > > > > > An interesting topic to discuss is how to avoid such situations when a > driver only lives 1-2 cycles.. > > Knowing a preference may exist for us to avoid the word support, I > feel like it was a preview that didn't pan out. I don't feel that fear > of this happening again should change any of our patterns moving > forward though, since that would be the exact opposite of what is > needed to encourage adoption. > > This hopefully doesn't happen frequently, does it? It is hard to see the future so I don't know how we could have avoided this situation. I think as long as the driver makes sense at-the-time and meets our requirements, it ought to land. I don't want to add more processes/steps/hurdles... :) --ruby -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosmaita.fossdev at gmail.com Wed Dec 18 20:14:37 2019 From: rosmaita.fossdev at gmail.com (Brian Rosmaita) Date: Wed, 18 Dec 2019 15:14:37 -0500 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: References: Message-ID: On 12/17/19 3:51 PM, Matt Riedemann wrote: > The ceph job on stable/pike has been broken for awhile [1]. > > The fix to make devstack-plugin-ceph (which is branchless) work on > stable/pike depends on this devstack change on stable/pike to introduce > the TARGET_BRANCH variable [2]. > > Question is, can we move [2] forward? If not, I make a motion that we > drop ceph jobs from stable/pike (and likely stable/ocata if we are still > running them there) because they won't work. We discussed this at today's Cinder meeting. We don't see a problem with moving [2] forward. We also don't object to simply removing non-voting jobs from stable/ocata and stable/pike. (I'm being no help here, but we did discuss it.) > The ceph job is non-voting anyway so that's why it goes unnoticed for so > long when it's broken. If we aren't going to fix this stuff we shouldn't > be wasting CI resources running broken jobs. Agreed. > Did I mention that you should all get off my lawn? :) OK, boomer. :) > [1] https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1835627 > [2] https://review.opendev.org/#/c/684756/ > From Arkady.Kanevsky at dell.com Wed Dec 18 20:20:12 2019 From: Arkady.Kanevsky at dell.com (Arkady.Kanevsky at dell.com) Date: Wed, 18 Dec 2019 20:20:12 +0000 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: <20191218113015.ppyqla4kjljjhfmx@lyarwood.usersys.redhat.com> References: <20191218113015.ppyqla4kjljjhfmx@lyarwood.usersys.redhat.com> Message-ID: <506823521bc34d6c82b1dda7129a7f59@AUSX13MPS308.AMER.DELL.COM> +1 for delete -----Original Message----- From: Lee Yarwood Sent: Wednesday, December 18, 2019 5:30 AM To: Matt Riedemann Cc: openstack-discuss at lists.openstack.org Subject: Re: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? On 17-12-19 14:51:08, Matt Riedemann wrote: > The ceph job on stable/pike has been broken for awhile [1]. > > The fix to make devstack-plugin-ceph (which is branchless) work on > stable/pike depends on this devstack change on stable/pike to > introduce the TARGET_BRANCH variable [2]. > > Question is, can we move [2] forward? If not, I make a motion that we > drop ceph jobs from stable/pike (and likely stable/ocata if we are > still running them there) because they won't work. > > The ceph job is non-voting anyway so that's why it goes unnoticed for > so long when it's broken. If we aren't going to fix this stuff we > shouldn't be wasting CI resources running broken jobs. > > Did I mention that you should all get off my lawn? :) > > [1] https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1835627 > [2] https://review.opendev.org/#/c/684756/ I'm +1 on dropping the job on stable/pike if we can't push [2] forward. This was always going to happen with EM branches anyway right? -- Lee Yarwood A5D1 9385 88CB 7E5F BE64 6618 BCA6 6E33 F672 2D76 From anlin.kong at gmail.com Wed Dec 18 20:25:32 2019 From: anlin.kong at gmail.com (Lingxian Kong) Date: Thu, 19 Dec 2019 09:25:32 +1300 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: On Thu, Dec 19, 2019 at 2:08 AM Thierry Carrez wrote: > Lingxian Kong wrote: > > [...] > > 1. Maybe Gnocchi is good, but it's not part of OpenStack, currently > > there is no one in the Telemetry team shows any interest in maintaining > > it, I'd be very happy to see if there is someone raises hand and say: I > can > > commit to spend some time on this project, fixing bugs, maintaining > > document and answering questions either on IRC or via email. If yes, We > > will see how it goes in this dev cycle. > > Is there any specific standing issue with Gnocchi that you are blocked > on ? Or is it more of a general concern that you should not depend on an > unmaintained project ? > Many people were asking for the cpu_util support that was in Ceilometer but not in Gnocchi which is important in auto-scaling scenario, but no one knows the history. Gnocchi doc website is broken and no one is available to fix. No one maintains Gnocchi CI. Sometimes people are coming to #openstack-telemetry for questions or potential bug report, but on one is answering, which makes people rather disappointed. > If it's the latter (and there is no real burning issue with it), it > feels like forking or taking over Gnocchi, and then keep it barely > functional enough for OpenStack use case might not be that much of work > (compared to other options) and we might find volunteers in this thread > to help. > +1 - Best regards, Lingxian Kong Catalyst Cloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From anlin.kong at gmail.com Wed Dec 18 20:36:46 2019 From: anlin.kong at gmail.com (Lingxian Kong) Date: Thu, 19 Dec 2019 09:36:46 +1300 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <742f67fb-8b47-9cca-ef34-dc8fd54e8d0c@suse.com> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <742f67fb-8b47-9cca-ef34-dc8fd54e8d0c@suse.com> Message-ID: On Thu, Dec 19, 2019 at 1:24 AM Witek Bedyk wrote: > > Yes, we decided to use Apache Kafka as it's better suited than RabbitMQ > to pass large loads of data with high throughput. Building the system > around Kafka allows us to provide fault-tolerance and scalability. > Hi Witek, I think I've mentioned in this thread or other related emails MANY times, or even face-to-face with you in the Shanghai Summit. I never doubt that Monasca is doing great job with the current architecture, I totally understand that people choose it in production. As an open source community, nothing could stop us from integrating it with Ceilometer(and I think it's already supported[1]). [1]: https://github.com/openstack/ceilometer/blob/master/setup.cfg#L209 - Best regards, Lingxian Kong Catalyst Cloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre at stackhpc.com Wed Dec 18 22:20:14 2019 From: pierre at stackhpc.com (Pierre Riteau) Date: Wed, 18 Dec 2019 23:20:14 +0100 Subject: [bifrost][ironic][kayobe][kolla][requirements] Kayobe CI breakage following ironic-lib release Message-ID: Hello, I have been looking into CI failures that started impacting Kayobe today on the master branch, just as we're planning to branch stable/train this week (Kayobe is cycle trailing). Our seed jobs are failing because ironic doesn't start correctly [1]. The traceback shows that it is using code from the latest ironic-lib (3.0.0) which was released just last week [2] and dropped Python 2 compatibility. We're using Kolla Train images [3], but the latest kolla/centos-source-bifrost-deploy:train includes Bifrost 7.0.0 which fetches all its dependencies using Git master branches [4]. This only got changed later on stable/train [5], but there is no newer 7.x release. One of these dependencies is the requirements repo, which has just bumped the upper constraint for ironic-lib to 3.0.0 [6]. I believe this is what caused our jobs to start failing today. So I think we need: - a 7.0.1 release of Bifrost - an update of Kolla to use this release instead of 7.0.0 Would the ironic and kolla project teams be willing to help with this to fix our gate? Thanks a lot. [1] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/kolla/ironic-api/ironic-api-journal.txt.gz [2] https://review.opendev.org/#/c/698500/ [3] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/system_logs/docker-info.txt.gz [4] https://opendev.org/openstack/bifrost/src/tag/7.0.0/playbooks/roles/bifrost-prep-for-install/defaults/main.yml#L30 [5] https://review.opendev.org/#/q/I3005fc59d6c5e1f5000874342f956977d9929907 [6] https://review.opendev.org/#/c/698981/ From sorrison at gmail.com Wed Dec 18 22:38:10 2019 From: sorrison at gmail.com (Sam Morrison) Date: Thu, 19 Dec 2019 09:38:10 +1100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <742f67fb-8b47-9cca-ef34-dc8fd54e8d0c@suse.com> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <742f67fb-8b47-9cca-ef34-dc8fd54e8d0c@suse.com> Message-ID: <5827ACF7-99FD-46C1-88F5-E63AF31ED7FB@gmail.com> > On 18 Dec 2019, at 11:19 pm, Witek Bedyk wrote: > > Hi, > > we have held a presentation in Shanghai on that topic if that's of help for anyone: > > https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/23932/transitioning-your-cloud-monitoring-from-ceilometer-to-monasca > > We've compared the technology of Telemetry and Monasca and how specific use cases like billing or auto-scaling are implemented. Monasca is a valid replacement option for Gnocchi. > > > On 12/18/19 6:37 AM, Lingxian Kong wrote: >> 2. Monasca is also good, although it doesn't support auditing i.e. >> storing samples which is needed by billing purpose but I didn't see it's >> mentioned by anyone in this thread. > > I have to disagree with that. Monasca is used for billing in production. > > >> What's more, it introduces kafaka >> which is not usually used in all other openstack core services. > > Yes, we decided to use Apache Kafka as it's better suited than RabbitMQ to pass large loads of data with high throughput. Building the system around Kafka allows us to provide fault-tolerance and scalability. It would make it a whole lot easier to switch to Monasca if it supported RabbitMQ too. RabbitMQ also provides fault tolerance, scalability and can handle high throughput etc. Having to operate and maintain another queuing system is a big hurdle for us. Sam From juliaashleykreger at gmail.com Wed Dec 18 22:54:41 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 18 Dec 2019 14:54:41 -0800 Subject: [bifrost][ironic][kayobe][kolla][requirements] Kayobe CI breakage following ironic-lib release In-Reply-To: References: Message-ID: Pierre, Bifrost's train branch can be patched to explicitly set stable/train instead of master and then we can release 7.0.1. That seems reasonable to me. Is that what your hoping for? If not, it is going to be a little more unplanned work at the moment because bifrost needs more work for python3. -Julia On Wed, Dec 18, 2019 at 2:22 PM Pierre Riteau wrote: > > Hello, > > I have been looking into CI failures that started impacting Kayobe > today on the master branch, just as we're planning to branch > stable/train this week (Kayobe is cycle trailing). > > Our seed jobs are failing because ironic doesn't start correctly [1]. > The traceback shows that it is using code from the latest ironic-lib > (3.0.0) which was released just last week [2] and dropped Python 2 > compatibility. > > We're using Kolla Train images [3], but the latest > kolla/centos-source-bifrost-deploy:train includes Bifrost 7.0.0 which > fetches all its dependencies using Git master branches [4]. This only > got changed later on stable/train [5], but there is no newer 7.x > release. > > One of these dependencies is the requirements repo, which has just > bumped the upper constraint for ironic-lib to 3.0.0 [6]. I believe > this is what caused our jobs to start failing today. > > So I think we need: > > - a 7.0.1 release of Bifrost > - an update of Kolla to use this release instead of 7.0.0 > > Would the ironic and kolla project teams be willing to help with this > to fix our gate? Thanks a lot. > > [1] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/kolla/ironic-api/ironic-api-journal.txt.gz > [2] https://review.opendev.org/#/c/698500/ > [3] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/system_logs/docker-info.txt.gz > [4] https://opendev.org/openstack/bifrost/src/tag/7.0.0/playbooks/roles/bifrost-prep-for-install/defaults/main.yml#L30 > [5] https://review.opendev.org/#/q/I3005fc59d6c5e1f5000874342f956977d9929907 > [6] https://review.opendev.org/#/c/698981/ > From pierre at stackhpc.com Wed Dec 18 23:03:55 2019 From: pierre at stackhpc.com (Pierre Riteau) Date: Thu, 19 Dec 2019 00:03:55 +0100 Subject: [bifrost][ironic][kayobe][kolla][requirements] Kayobe CI breakage following ironic-lib release In-Reply-To: References: Message-ID: Hi Julia, Thanks for the quick response. The stable/train branch was already patched by Mark a few weeks ago [1] and I have just noticed he proposed a release of 7.1.0 last week [2], so we just need someone from the release team to approve it. I can then submit a Kolla patch to use the new tarball. [1] https://review.opendev.org/#/c/687820/ [2] https://review.opendev.org/#/c/698940/ On Wed, 18 Dec 2019 at 23:54, Julia Kreger wrote: > > Pierre, > > Bifrost's train branch can be patched to explicitly set stable/train > instead of master and then we can release 7.0.1. That seems reasonable > to me. > > Is that what your hoping for? If not, it is going to be a little more > unplanned work at the moment because bifrost needs more work for > python3. > > -Julia > > On Wed, Dec 18, 2019 at 2:22 PM Pierre Riteau wrote: > > > > Hello, > > > > I have been looking into CI failures that started impacting Kayobe > > today on the master branch, just as we're planning to branch > > stable/train this week (Kayobe is cycle trailing). > > > > Our seed jobs are failing because ironic doesn't start correctly [1]. > > The traceback shows that it is using code from the latest ironic-lib > > (3.0.0) which was released just last week [2] and dropped Python 2 > > compatibility. > > > > We're using Kolla Train images [3], but the latest > > kolla/centos-source-bifrost-deploy:train includes Bifrost 7.0.0 which > > fetches all its dependencies using Git master branches [4]. This only > > got changed later on stable/train [5], but there is no newer 7.x > > release. > > > > One of these dependencies is the requirements repo, which has just > > bumped the upper constraint for ironic-lib to 3.0.0 [6]. I believe > > this is what caused our jobs to start failing today. > > > > So I think we need: > > > > - a 7.0.1 release of Bifrost > > - an update of Kolla to use this release instead of 7.0.0 > > > > Would the ironic and kolla project teams be willing to help with this > > to fix our gate? Thanks a lot. > > > > [1] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/kolla/ironic-api/ironic-api-journal.txt.gz > > [2] https://review.opendev.org/#/c/698500/ > > [3] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/system_logs/docker-info.txt.gz > > [4] https://opendev.org/openstack/bifrost/src/tag/7.0.0/playbooks/roles/bifrost-prep-for-install/defaults/main.yml#L30 > > [5] https://review.opendev.org/#/q/I3005fc59d6c5e1f5000874342f956977d9929907 > > [6] https://review.opendev.org/#/c/698981/ > > From juliaashleykreger at gmail.com Wed Dec 18 23:06:30 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 18 Dec 2019 15:06:30 -0800 Subject: [bifrost][ironic][kayobe][kolla][requirements] Kayobe CI breakage following ironic-lib release In-Reply-To: References: Message-ID: And I clearly didn't actually look at the branch status presently. Yeah, I'll submit a 7.0.1 release right now On Wed, Dec 18, 2019 at 2:54 PM Julia Kreger wrote: > > Pierre, > > Bifrost's train branch can be patched to explicitly set stable/train > instead of master and then we can release 7.0.1. That seems reasonable > to me. > > Is that what your hoping for? If not, it is going to be a little more > unplanned work at the moment because bifrost needs more work for > python3. > > -Julia > > On Wed, Dec 18, 2019 at 2:22 PM Pierre Riteau wrote: > > > > Hello, > > > > I have been looking into CI failures that started impacting Kayobe > > today on the master branch, just as we're planning to branch > > stable/train this week (Kayobe is cycle trailing). > > > > Our seed jobs are failing because ironic doesn't start correctly [1]. > > The traceback shows that it is using code from the latest ironic-lib > > (3.0.0) which was released just last week [2] and dropped Python 2 > > compatibility. > > > > We're using Kolla Train images [3], but the latest > > kolla/centos-source-bifrost-deploy:train includes Bifrost 7.0.0 which > > fetches all its dependencies using Git master branches [4]. This only > > got changed later on stable/train [5], but there is no newer 7.x > > release. > > > > One of these dependencies is the requirements repo, which has just > > bumped the upper constraint for ironic-lib to 3.0.0 [6]. I believe > > this is what caused our jobs to start failing today. > > > > So I think we need: > > > > - a 7.0.1 release of Bifrost > > - an update of Kolla to use this release instead of 7.0.0 > > > > Would the ironic and kolla project teams be willing to help with this > > to fix our gate? Thanks a lot. > > > > [1] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/kolla/ironic-api/ironic-api-journal.txt.gz > > [2] https://review.opendev.org/#/c/698500/ > > [3] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/system_logs/docker-info.txt.gz > > [4] https://opendev.org/openstack/bifrost/src/tag/7.0.0/playbooks/roles/bifrost-prep-for-install/defaults/main.yml#L30 > > [5] https://review.opendev.org/#/q/I3005fc59d6c5e1f5000874342f956977d9929907 > > [6] https://review.opendev.org/#/c/698981/ > > From juliaashleykreger at gmail.com Wed Dec 18 23:17:11 2019 From: juliaashleykreger at gmail.com (Julia Kreger) Date: Wed, 18 Dec 2019 15:17:11 -0800 Subject: [bifrost][ironic][kayobe][kolla][requirements] Kayobe CI breakage following ironic-lib release In-Reply-To: References: Message-ID: Ack, that works. Thanks! On Wed, Dec 18, 2019 at 3:04 PM Pierre Riteau wrote: > > Hi Julia, > > Thanks for the quick response. > > The stable/train branch was already patched by Mark a few weeks ago > [1] and I have just noticed he proposed a release of 7.1.0 last week > [2], so we just need someone from the release team to approve it. I > can then submit a Kolla patch to use the new tarball. > > [1] https://review.opendev.org/#/c/687820/ > [2] https://review.opendev.org/#/c/698940/ > > On Wed, 18 Dec 2019 at 23:54, Julia Kreger wrote: > > > > Pierre, > > > > Bifrost's train branch can be patched to explicitly set stable/train > > instead of master and then we can release 7.0.1. That seems reasonable > > to me. > > > > Is that what your hoping for? If not, it is going to be a little more > > unplanned work at the moment because bifrost needs more work for > > python3. > > > > -Julia > > > > On Wed, Dec 18, 2019 at 2:22 PM Pierre Riteau wrote: > > > > > > Hello, > > > > > > I have been looking into CI failures that started impacting Kayobe > > > today on the master branch, just as we're planning to branch > > > stable/train this week (Kayobe is cycle trailing). > > > > > > Our seed jobs are failing because ironic doesn't start correctly [1]. > > > The traceback shows that it is using code from the latest ironic-lib > > > (3.0.0) which was released just last week [2] and dropped Python 2 > > > compatibility. > > > > > > We're using Kolla Train images [3], but the latest > > > kolla/centos-source-bifrost-deploy:train includes Bifrost 7.0.0 which > > > fetches all its dependencies using Git master branches [4]. This only > > > got changed later on stable/train [5], but there is no newer 7.x > > > release. > > > > > > One of these dependencies is the requirements repo, which has just > > > bumped the upper constraint for ironic-lib to 3.0.0 [6]. I believe > > > this is what caused our jobs to start failing today. > > > > > > So I think we need: > > > > > > - a 7.0.1 release of Bifrost > > > - an update of Kolla to use this release instead of 7.0.0 > > > > > > Would the ironic and kolla project teams be willing to help with this > > > to fix our gate? Thanks a lot. > > > > > > [1] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/kolla/ironic-api/ironic-api-journal.txt.gz > > > [2] https://review.opendev.org/#/c/698500/ > > > [3] https://dafcf7323a7c828b67e0-814491f7a8e968b50eeaf14b0c7f115f.ssl.cf1.rackcdn.com/698882/7/check/kayobe-seed-centos/8488533/primary/system_logs/docker-info.txt.gz > > > [4] https://opendev.org/openstack/bifrost/src/tag/7.0.0/playbooks/roles/bifrost-prep-for-install/defaults/main.yml#L30 > > > [5] https://review.opendev.org/#/q/I3005fc59d6c5e1f5000874342f956977d9929907 > > > [6] https://review.opendev.org/#/c/698981/ > > > From aaronzhu1121 at gmail.com Thu Dec 19 01:41:49 2019 From: aaronzhu1121 at gmail.com (Rong Zhu) Date: Thu, 19 Dec 2019 09:41:49 +0800 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> Message-ID: I just want to know any other guys from the thread want to as volunteers to take over gnocchi? About Monasca, we discuss a lot in Shanghai, if users already use monasca, as Lingxian mentioned ceilometer has already support publish to monasca. but I think most of the user didn't use monasca, as a community you can just say use monasca, but for company want to use in production, add a component and the depends extra tools would be very very difficult, this will need many many resources to do the things. Tobias Urdin 于2019年12月18日 周三18:00写道： > https://github.com/gnocchixyz/gnocchi/issues/1049#issuecomment-555768072 > > On 12/18/19 10:15 AM, Tobias Urdin wrote: > > As an operator I second pretty much everything Samsaid, using > > ceilometer-api never really worked without hand holding all the time. > > We migrated over to Gnocchi as that was "the way forward" from the > > Telemetry team and it has worked great. > > > > Gnocchi has a learning curve but after that it has been running > > flawlessly even at a larger scale, just introduced more workers and > > everything is set. > > > > I think the long term plan from the Telemetry team was to move out any > > storage abstraction and cultivate ceilometer to a single focus area > around > > collecting metrics. Moving any API, transformations, storage etc to > > another service. > > > > I think it's said to see Gnocchi, the actual solutions to the problem, > > being unmaintained and out of the OpenStack developer ecosystem. I > > assume there > > is a cost to bringing it back in after it was moved out but maybe it's > > something that is needed? > > > > While I don't have a deep understand in Gnocchi I would have no choice > > but to try to spend more time learning it and fixing any issues that we > > might > > see since at this point we can't live without it, as our billing > > providers supports the Gnocchi API, we using Heat with Gnocchi and Aodh > > to autoscale etc. > > > > As a final note; thanks for bringing back the cpu_util metric, means I > > can drop the ugly customized code that was required to bring that metric > > back while > > it was removed :) > > > > Best regards > > Tobias > > > > On 12/18/19 5:39 AM, Sam Morrison wrote: > >>> On 17 Dec 2019, at 10:14 pm, Thierry Carrez > wrote: > >>> > >>> Zane Bitter wrote: > >>>> On 15/12/19 10:20 pm, Rong Zhu wrote: > >>>>> 1.Add Ceilometer API back > >>>>> Since Gnocchi is out of OpenStack and is unmaintained, we > need to add Ceilometer API back again. > >>>> This is concerning because even the people who wrote it don't > consider it adequate to the job. That inadequacy has been the source of > significant reputational damage to OpenStack in the past, and many folks > (me included) are anxious to avoid a repeat. > >>> Yes this concerns me too, and even if we workaround the issue by > adding Ceilo API back, I'd like to have a long-term plan to solve this > issue. It seems there are several options on the table (including > integrating Monasca and Ceilometer into a single stack, and other big-bang > replacements) but it's never a one-for-one comparison as the solutions seem > to address slightly disjoint problem spaces. > >>> > >>> I'd like to hear from more Ceilometer users. What are they using > Ceilometer for, and what long-term plans would be acceptable. There is a > trade-off between adopting short-term workarounds that reintroduce > performance issues vs. undergoing a complex migration to the "right" way of > fixing this. Like for example there is little point in pushing > Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud > seems to say, that they would rather have a slow performing Ceilometer API > back. > >> Nectar Cloud has been a ceilometer user from the early days. Well we > tried to be and couldn’t use it as ceilometer api and mongo db just didn’t > work at our scale. Gnocchi solved all these issues for us and we use > ceilometer/aodh/gnocchi happily in production for several years now. > >> If telemetry project is going down the path of the old days it will > mean we will either drop ceilometer all together and look at alternative > solutions like monasca or Prometheus etc. I just can’t see how the old > architecture of ceilometer is ever going to be usable. > >> > >> If there is some confidence that gnocchi publisher will be supported in > the future we would keep using gnocchi and just maintain it ourselves. It’s > an open source project and I was hoping the openstack community could keep > it going. We’d be happy to help maintain it at least. > >> > >> We use ceilometer/gnocchi to collect and store all metrics from > openstack services. We have also written some custom pollsters and gnocchi > is quite flexible here to allow this. With these metrics we build reports > for our users, our operators, our funders (the government) > >> > >> > >> Please reconsider your direction much like adding cpu_util back in > (thank you for this!) > >> > >> Cheers, > >> Sam > >> > >> > >> > >>>> Telemetry is part of the TC "Approved Release" that is eligible for > the trademark program; I think at a minimum the TC will want to remove the > resurrected Ceilometer API from the "designated sections" that users are > required to run to participate in any trademark program that includes the > functionality in question. But I think that we should explore other ways of > reducing the chance of anyone confusing this for a viable way of building a > cloud, including possibly changing the name (Antediluvian API?) and having > this part of the stack live outside of the official OpenStack project. > >>> Legacy API? > >>> > >>> -- > >>> Thierry Carrez (ttx) > >>> > >> > > > > > > > -- Thanks, Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Thu Dec 19 02:07:06 2019 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Thu, 19 Dec 2019 10:07:06 +0800 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: On Wed, Dec 18, 2019 at 1:43 PM Lingxian Kong wrote: > > Hi all, > > 1. Maybe Gnocchi is good, but it's not part of OpenStack, currently > there is no one in the Telemetry team shows any interest in maintaining > it, I'd be very happy to see if there is someone raises hand and say: I can > commit to spend some time on this project, fixing bugs, maintaining > document and answering questions either on IRC or via email. If yes, We > will see how it goes in this dev cycle. > If Gnocchi can't provide documents and CI, also can't close the issue said `Project is unmaintained`[1], we should ask if anyone here volunteer to resolved that part. Anyone?:) If you really care about Gnocchi+Telemetry, join the team please. > > 3. The Telemetry team did have discussed to bring the Ceilometer API and > MongoDB support back, given neither Gnocchi nor Monasca is able to store > the original samples. However, nothing is happening, so don't be panic. >From my perspective, to bring back Ceilometer API and MongoDB as another driver for Telemetry isn't a bad idea at all as long as other driver can keep their tasks forward. No one (at least I didn't see any one) mentioned about if some people in Telemetry team start the effort on Ceilometer API and MongoDB, no others can keep their effort on Gnocchi. The one thing I'm interested on is how to tune the MongoDB performance with Ceilometer. How can you make Ceilometer API+MongoDB more robust? So if any chances that's happening, will be create to provide related documents too. Also the CI need to be there when we bring back Ceilometer API and MongoDB support. Despite the detail is not yet develop here, I'm agree if that's a short-term plan. [1] https://github.com/gnocchixyz/gnocchi/issues/1049 > the current project core members are also the Telemetry service cloud > providers, we know how important it is to not break anything, to not > bring any more overhead than before. we are so glad to see this I believe that's really important for you guys too:) > discussion, at least there are still so many people using > Ceilometer/Gnocchi and have concerns about the current upstream > situation. It would be much better that people involved in this > discussion could participate in the design and implementation of the > future tasks. Yes, the only way is to join the team, stand your ground, and involve in the design. In long term, IMO we need to have a ultimate solution for all. A standard way so whatever other tools or service is developing, on the other can use it too. An tool/sdk/interface (and no need to build another service) to support both project. Or if there's better solution for it. And what need to done if we would like that. Will Telemetry and Monasca team interested on this? I would like to propose we move that discussion somewhere else, like in Monasca or Telemetry's team meeting? Will ask if any TC can help to coordinate. -- May The Force of OpenStack Be With You, Rico Lin irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Thu Dec 19 02:15:13 2019 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Thu, 19 Dec 2019 10:15:13 +0800 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: Just another ML earlier people need to also aware of -> http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010923.html On Thu, Dec 19, 2019 at 10:07 AM Rico Lin wrote: > > > On Wed, Dec 18, 2019 at 1:43 PM Lingxian Kong > wrote: > > > > Hi all, > > > > 1. Maybe Gnocchi is good, but it's not part of OpenStack, currently > > there is no one in the Telemetry team shows any interest in maintaining > > it, I'd be very happy to see if there is someone raises hand and say: I > can > > commit to spend some time on this project, fixing bugs, maintaining > > document and answering questions either on IRC or via email. If yes, We > > will see how it goes in this dev cycle. > > > If Gnocchi can't provide documents and CI, also can't close the issue > said `Project is unmaintained`[1], > we should ask if anyone here volunteer to resolved that part. Anyone?:) > If you really care about Gnocchi+Telemetry, join the team please. > > > > > 3. The Telemetry team did have discussed to bring the Ceilometer API and > > MongoDB support back, given neither Gnocchi nor Monasca is able to store > > the original samples. However, nothing is happening, so don't be panic. > > From my perspective, to bring back Ceilometer API and MongoDB as another > driver for Telemetry isn't a bad idea at all as long as other driver can > keep their tasks forward. > No one (at least I didn't see any one) mentioned about if some people in > Telemetry team > start the effort on Ceilometer API and MongoDB, no others can keep their > effort on Gnocchi. > > The one thing I'm interested on is how to tune the MongoDB performance > with Ceilometer. > How can you make Ceilometer API+MongoDB more robust? > So if any chances that's happening, will be create to provide related > documents too. > Also the CI need to be there when we bring back Ceilometer API and MongoDB > support. > > Despite the detail is not yet develop here, I'm agree if that's a > short-term plan. > > [1] https://github.com/gnocchixyz/gnocchi/issues/1049 > > > the current project core members are also the Telemetry service cloud > > providers, we know how important it is to not break anything, to not > > bring any more overhead than before. we are so glad to see this > > I believe that's really important for you guys too:) > > > discussion, at least there are still so many people using > > Ceilometer/Gnocchi and have concerns about the current upstream > > situation. It would be much better that people involved in this > > discussion could participate in the design and implementation of the > > future tasks. > > Yes, the only way is to join the team, stand your ground, and involve in > the design. > > In long term, IMO we need to have a ultimate solution for all. A standard > way so whatever > other tools or service is developing, on the other can use it too. > An tool/sdk/interface (and no need to build another service) to support > both project. > Or if there's better solution for it. > And what need to done if we would like that. > Will Telemetry and Monasca team interested on this? > I would like to propose we move that discussion somewhere else, like > in Monasca or Telemetry's team meeting? > Will ask if any TC can help to coordinate. > > -- > May The Force of OpenStack Be With You, > Rico Lin > irc: ricolin > -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaronzhu1121 at gmail.com Thu Dec 19 03:50:52 2019 From: aaronzhu1121 at gmail.com (Rong Zhu) Date: Thu, 19 Dec 2019 11:50:52 +0800 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> Message-ID: Another worry is Monasca's main contributors are from SUSE. Rong Zhu 于2019年12月19日 周四09:41写道： > I just want to know any other guys from the thread want to as volunteers > to take over gnocchi? > > About Monasca, we discuss a lot in Shanghai, if users already use monasca, > as Lingxian mentioned ceilometer has already support publish to monasca. > but I think most of the user didn't use monasca, as a community you can > just say use monasca, but for company want to use in production, add a > component and the depends extra tools would be very very difficult, this > will need many many resources to do the things. > > Tobias Urdin 于2019年12月18日 周三18:00写道： > >> https://github.com/gnocchixyz/gnocchi/issues/1049#issuecomment-555768072 >> >> On 12/18/19 10:15 AM, Tobias Urdin wrote: >> > As an operator I second pretty much everything Samsaid, using >> > ceilometer-api never really worked without hand holding all the time. >> > We migrated over to Gnocchi as that was "the way forward" from the >> > Telemetry team and it has worked great. >> > >> > Gnocchi has a learning curve but after that it has been running >> > flawlessly even at a larger scale, just introduced more workers and >> > everything is set. >> > >> > I think the long term plan from the Telemetry team was to move out any >> > storage abstraction and cultivate ceilometer to a single focus area >> around >> > collecting metrics. Moving any API, transformations, storage etc to >> > another service. >> > >> > I think it's said to see Gnocchi, the actual solutions to the problem, >> > being unmaintained and out of the OpenStack developer ecosystem. I >> > assume there >> > is a cost to bringing it back in after it was moved out but maybe it's >> > something that is needed? >> > >> > While I don't have a deep understand in Gnocchi I would have no choice >> > but to try to spend more time learning it and fixing any issues that we >> > might >> > see since at this point we can't live without it, as our billing >> > providers supports the Gnocchi API, we using Heat with Gnocchi and Aodh >> > to autoscale etc. >> > >> > As a final note; thanks for bringing back the cpu_util metric, means I >> > can drop the ugly customized code that was required to bring that metric >> > back while >> > it was removed :) >> > >> > Best regards >> > Tobias >> > >> > On 12/18/19 5:39 AM, Sam Morrison wrote: >> >>> On 17 Dec 2019, at 10:14 pm, Thierry Carrez >> wrote: >> >>> >> >>> Zane Bitter wrote: >> >>>> On 15/12/19 10:20 pm, Rong Zhu wrote: >> >>>>> 1.Add Ceilometer API back >> >>>>> Since Gnocchi is out of OpenStack and is unmaintained, we >> need to add Ceilometer API back again. >> >>>> This is concerning because even the people who wrote it don't >> consider it adequate to the job. That inadequacy has been the source of >> significant reputational damage to OpenStack in the past, and many folks >> (me included) are anxious to avoid a repeat. >> >>> Yes this concerns me too, and even if we workaround the issue by >> adding Ceilo API back, I'd like to have a long-term plan to solve this >> issue. It seems there are several options on the table (including >> integrating Monasca and Ceilometer into a single stack, and other big-bang >> replacements) but it's never a one-for-one comparison as the solutions seem >> to address slightly disjoint problem spaces. >> >>> >> >>> I'd like to hear from more Ceilometer users. What are they using >> Ceilometer for, and what long-term plans would be acceptable. There is a >> trade-off between adopting short-term workarounds that reintroduce >> performance issues vs. undergoing a complex migration to the "right" way of >> fixing this. Like for example there is little point in pushing >> Monasca/Ceilometer stack integration if most users say, like Catalyst Cloud >> seems to say, that they would rather have a slow performing Ceilometer API >> back. >> >> Nectar Cloud has been a ceilometer user from the early days. Well we >> tried to be and couldn’t use it as ceilometer api and mongo db just didn’t >> work at our scale. Gnocchi solved all these issues for us and we use >> ceilometer/aodh/gnocchi happily in production for several years now. >> >> If telemetry project is going down the path of the old days it will >> mean we will either drop ceilometer all together and look at alternative >> solutions like monasca or Prometheus etc. I just can’t see how the old >> architecture of ceilometer is ever going to be usable. >> >> >> >> If there is some confidence that gnocchi publisher will be supported >> in the future we would keep using gnocchi and just maintain it ourselves. >> It’s an open source project and I was hoping the openstack community could >> keep it going. We’d be happy to help maintain it at least. >> >> >> >> We use ceilometer/gnocchi to collect and store all metrics from >> openstack services. We have also written some custom pollsters and gnocchi >> is quite flexible here to allow this. With these metrics we build reports >> for our users, our operators, our funders (the government) >> >> >> >> >> >> Please reconsider your direction much like adding cpu_util back in >> (thank you for this!) >> >> >> >> Cheers, >> >> Sam >> >> >> >> >> >> >> >>>> Telemetry is part of the TC "Approved Release" that is eligible for >> the trademark program; I think at a minimum the TC will want to remove the >> resurrected Ceilometer API from the "designated sections" that users are >> required to run to participate in any trademark program that includes the >> functionality in question. But I think that we should explore other ways of >> reducing the chance of anyone confusing this for a viable way of building a >> cloud, including possibly changing the name (Antediluvian API?) and having >> this part of the stack live outside of the official OpenStack project. >> >>> Legacy API? >> >>> >> >>> -- >> >>> Thierry Carrez (ttx) >> >>> >> >> >> > >> > >> >> >> -- > Thanks, > Rong Zhu > -- Thanks, Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From rico.lin.guanyu at gmail.com Thu Dec 19 03:55:23 2019 From: rico.lin.guanyu at gmail.com (Rico Lin) Date: Thu, 19 Dec 2019 11:55:23 +0800 Subject: [resource-management-sig] Status of the "Resource Management" SIG In-Reply-To: References: <5a84404f-0e10-9010-61ed-29aff08b5ec6@openstack.org> Message-ID: On Fri, Nov 22, 2019 at 11:57 PM Thierry Carrez wrote: > > Zhipeng Huang wrote: > > Intend to launch the activity during Shanghai PTG via Cyborg sessions, > > for those you are interested in topics like Kubernetes, OCP OAI, RISC-V > > you are more than welcomed to reach out to me. > > Was that effort successful? Should we keep the resource management SIG, > with status:forming and removing the other chairs? > Agree with Thierry, the best status for Resource Management is forming. which (I hope) will also attract other people's attention. > -- > Thierry > -- May The Force of OpenStack Be With You, Rico Lin irc: ricolin -------------- next part -------------- An HTML attachment was scrubbed... URL: From colleen at gazlene.net Thu Dec 19 05:13:07 2019 From: colleen at gazlene.net (Colleen Murphy) Date: Wed, 18 Dec 2019 21:13:07 -0800 Subject: [keystone] No meetings over the holidays Message-ID: <9c1f8ea7-8efe-4429-a64c-a5e948e307ea@www.fastmail.com> As discussed at yesterday's meeting, we'll cancel keystone team meetings for the next two weeks for the winter holidays. The next keystone meeting will be Tuesday, January 7 (17:00 UTC, #openstack-meeting-alt). Colleen From witold.bedyk at suse.com Thu Dec 19 09:40:55 2019 From: witold.bedyk at suse.com (Witek Bedyk) Date: Thu, 19 Dec 2019 10:40:55 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> Message-ID: <73f18228-1ded-7f00-70d7-331a635ab3a8@suse.com> On 12/19/19 3:07 AM, Rico Lin wrote: > In long term, IMO we need to have a ultimate solution for all. A > standard way so whatever > other tools or service is developing, on the other can use it too. > An tool/sdk/interface (and no need to build another service) to support > both project. > Or if there's better solution for it. > And what need to done if we would like that. > Will Telemetry and Monasca team interested on this? > I would like to propose we move that discussion somewhere else, like > in Monasca or Telemetry's team meeting? > Will ask if any TC can help to coordinate. I'm definitely interested in discussing long-term solution for monitoring as Thierry suggested. Cheers Witek From doug at doughellmann.com Thu Dec 19 09:46:25 2019 From: doug at doughellmann.com (Doug Hellmann) Date: Thu, 19 Dec 2019 10:46:25 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: Message-ID: > On Dec 17, 2019, at 9:03 PM, Albert Braden wrote: > > My present employer is just starting with Openstack and we aren't using Ceilometer yet, but when I was at eBay we had a whole team trying to make it work and it was still terrible. I was very happy when I heard that the shitty old ceilometer was going away and would never be heard from again. I hope the users of the services you provide never speak to you in this way. Irrespective of the accuracy of what you say, this sort of statement is poor form in a community discussion. Please have more respect for your peers in the future. Doug From tobias.urdin at binero.se Thu Dec 19 10:41:54 2019 From: tobias.urdin at binero.se (Tobias Urdin) Date: Thu, 19 Dec 2019 11:41:54 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> Message-ID: <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> I can  volounteer to spend time on Ceilomter and Gnocchi while I have some mimimal knowledge on Ceilomter and pretty much none on the Gnocchi codebase I would like to see the project continued. Another thing would be if Gnocchi should be moved back or if I should somehow get in contact with the former Gnocchi maintainers and see if we can get access to GitHub? Best regards On 12/19/19 2:46 AM, Rong Zhu wrote: > I just want to know any other guys from the thread want to as > volunteers to take over gnocchi? > > About Monasca, we discuss a lot in Shanghai, if users already use > monasca, as Lingxian mentioned ceilometer has already support publish > to monasca. but I think most of the user didn't use monasca, as a > community you can just say use monasca, but for company want to use in > production, add a component and the depends extra tools would be very > very difficult, this will need many many resources to do the things. > > Tobias Urdin >于2019年12月18日 周三18:00写道： > > https://github.com/gnocchixyz/gnocchi/issues/1049#issuecomment-555768072 > > On 12/18/19 10:15 AM, Tobias Urdin wrote: > > As an operator I second pretty much everything Samsaid, using > > ceilometer-api never really worked without hand holding all the > time. > > We migrated over to Gnocchi as that was "the way forward" from the > > Telemetry team and it has worked great. > > > > Gnocchi has a learning curve but after that it has been running > > flawlessly even at a larger scale, just introduced more workers and > > everything is set. > > > > I think the long term plan from the Telemetry team was to move > out any > > storage abstraction and cultivate ceilometer to a single focus > area around > > collecting metrics. Moving any API, transformations, storage etc to > > another service. > > > > I think it's said to see Gnocchi, the actual solutions to the > problem, > > being unmaintained and out of the OpenStack developer ecosystem. I > > assume there > > is a cost to bringing it back in after it was moved out but > maybe it's > > something that is needed? > > > > While I don't have a deep understand in Gnocchi I would have no > choice > > but to try to spend more time learning it and fixing any issues > that we > > might > > see since at this point we can't live without it, as our billing > > providers supports the Gnocchi API, we using Heat with Gnocchi > and Aodh > > to autoscale etc. > > > > As a final note; thanks for bringing back the cpu_util metric, > means I > > can drop the ugly customized code that was required to bring > that metric > > back while > > it was removed :) > > > > Best regards > > Tobias > > > > On 12/18/19 5:39 AM, Sam Morrison wrote: > >>> On 17 Dec 2019, at 10:14 pm, Thierry Carrez > > wrote: > >>> > >>> Zane Bitter wrote: > >>>> On 15/12/19 10:20 pm, Rong Zhu wrote: > >>>>> 1.Add Ceilometer API back > >>>>>        Since Gnocchi is out of OpenStack and is > unmaintained, we need to add Ceilometer API back again. > >>>> This is concerning because even the people who wrote it don't > consider it adequate to the job. That inadequacy has been the > source of significant reputational damage to OpenStack in the > past, and many folks (me included) are anxious to avoid a repeat. > >>> Yes this concerns me too, and even if we workaround the issue > by adding Ceilo API back, I'd like to have a long-term plan to > solve this issue. It seems there are several options on the table > (including integrating Monasca and Ceilometer into a single stack, > and other big-bang replacements) but it's never a one-for-one > comparison as the solutions seem to address slightly disjoint > problem spaces. > >>> > >>> I'd like to hear from more Ceilometer users. What are they > using Ceilometer for, and what long-term plans would be > acceptable. There is a trade-off between adopting short-term > workarounds that reintroduce performance issues vs. undergoing a > complex migration to the "right" way of fixing this. Like for > example there is little point in pushing Monasca/Ceilometer stack > integration if most users say, like Catalyst Cloud seems to say, > that they would rather have a slow performing Ceilometer API back. > >> Nectar Cloud has been a ceilometer user from the early days. > Well we tried to be and couldn’t use it as ceilometer api and > mongo db just didn’t work at our scale. Gnocchi solved all these > issues for us and we use ceilometer/aodh/gnocchi happily in > production for several years now. > >> If telemetry project is going down the path of the old days it > will mean we will either drop ceilometer all together and look at > alternative solutions like monasca or Prometheus etc. I just can’t > see how the old architecture of ceilometer is ever going to be usable. > >> > >> If there is some confidence that gnocchi publisher will be > supported in the future we would keep using gnocchi and just > maintain it ourselves. It’s an open source project and I was > hoping the openstack community could keep it going. We’d be happy > to help maintain it at least. > >> > >> We use ceilometer/gnocchi to collect and store all metrics from > openstack services. We have also written some custom pollsters and > gnocchi is quite flexible here to allow this. With these metrics > we build reports for our users, our operators, our funders (the > government) > >> > >> > >> Please reconsider your direction much like adding cpu_util back > in (thank you for this!) > >> > >> Cheers, > >> Sam > >> > >> > >> > >>>> Telemetry is part of the TC "Approved Release" that is > eligible for the trademark program; I think at a minimum the TC > will want to remove the resurrected Ceilometer API from the > "designated sections" that users are required to run to > participate in any trademark program that includes the > functionality in question. But I think that we should explore > other ways of reducing the chance of anyone confusing this for a > viable way of building a cloud, including possibly changing the > name (Antediluvian API?) and having this part of the stack live > outside of the official OpenStack project. > >>> Legacy API? > >>> > >>> -- > >>> Thierry Carrez (ttx) > >>> > >> > > > > > > > -- > Thanks, > Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Thu Dec 19 11:32:34 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 19 Dec 2019 12:32:34 +0100 Subject: [neutron] Team meeting cancelled Message-ID: <73C43D01-1D8F-4BF2-B663-39E6456D4126@redhat.com> Hi, Our Neutron team meetings which should be on 23.12.2019 and 31.12.2019 will be cancelled. See You all in new year :) Have a great holiday time. — Slawek Kaplonski Senior software engineer Red Hat From skaplons at redhat.com Thu Dec 19 11:33:19 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 19 Dec 2019 12:33:19 +0100 Subject: [neutron]CI meeting cancelled Message-ID: Hi, Our Neutron CI meetings which should be on 24.12.2019 and 31.12.2019 will be cancelled. See You all in new year :) Have a great holiday time. — Slawek Kaplonski Senior software engineer Red Hat From skaplons at redhat.com Thu Dec 19 11:35:02 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Thu, 19 Dec 2019 12:35:02 +0100 Subject: [neutron] PTL on PTO Message-ID: <7896AEAF-B76D-4DDF-A79E-65321426C69D@redhat.com> Hi, I will be on PTO from Friday 20.12.2019 up to 6.01.2020. I will probably not be available on IRC for most of the time but I will be checking emails from time to time. So if You have anything important, please email me. — Slawek Kaplonski Senior software engineer Red Hat From zigo at debian.org Thu Dec 19 13:25:06 2019 From: zigo at debian.org (Thomas Goirand) Date: Thu, 19 Dec 2019 14:25:06 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> Message-ID: On 12/19/19 11:41 AM, Tobias Urdin wrote: > I can  volounteer to spend time on Ceilomter and Gnocchi while I have > some mimimal knowledge on Ceilomter > and pretty much none on the Gnocchi codebase I would like to see the > project continued. > > Another thing would be if Gnocchi should be moved back or if I should > somehow get in contact with the former > Gnocchi maintainers and see if we can get access to GitHub? > > Best regards Hi Tobias, I believe Julien Danjou will be fine with both options. Just ask him, I'm convince he will reply quickly, and will be happy if someone takes over his work (and the other people involved in the former team). If my non-volunteering voice has some weight, I'd very much prefer if the project could go back to being OpenStack maintained. It would be reassuring in many ways. Cheers, Thomas Goirand (zigo) From thierry at openstack.org Thu Dec 19 13:59:13 2019 From: thierry at openstack.org (Thierry Carrez) Date: Thu, 19 Dec 2019 14:59:13 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> Message-ID: Tobias Urdin wrote: > I can  volounteer to spend time on Ceilomter and Gnocchi while I have > some mimimal knowledge on Ceilomter > and pretty much none on the Gnocchi codebase I would like to see the > project continued. > > Another thing would be if Gnocchi should be moved back or if I should > somehow get in contact with the former > Gnocchi maintainers and see if we can get access to GitHub? If the goal is to maintain Gnocchi as-is for the narrow OpenStack use case, my preference would be to fork / move it back to opendev, under a gnocchi namespace. That would ensure a clean cut from the unmaintained "gnocchi for all the things" project on GitHub, and reset expectations. If the goal is to take over Gnocchi and support all other use cases, then maybe taking it over on GitHub might be easier to ensure continuity. -- Thierry Carrez (ttx) From thomas at goirand.fr Thu Dec 19 10:31:26 2019 From: thomas at goirand.fr (Thomas Goirand) Date: Thu, 19 Dec 2019 11:31:26 +0100 Subject: [Telemetry][TC] Monasca in distributions like Debian and Ubuntu (was: Telemetry status) In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> Message-ID: <418be333-77f2-c749-9556-b4133257327f@goirand.fr> On 12/19/19 4:50 AM, Rong Zhu wrote: > Another worry is Monasca's main contributors are from SUSE. What I am not confident with, is that Monasca needs Java stuff. In here: https://github.com/openstack/monasca-common/blob/master/bindep.txt we can see openjdk-8-jdk. That's already obsolete. For example, Debian Buster was released last summer with openjdk 11, and version 8 isn't available anymore. With such situation, I don't think it's reasonable to attempt packaging Monasca for Debian and Ubuntu, and I'm not even scratching the Java module dependency issue: these would need to be packaged too, and I have no idea how much work that would be (can someone tell me?). If some upstream contributors of Monasca want to bring more light to this topic, please feel free. I obviously don't know enough about it. Cheers, Thomas Goirand (zigo) From doug at stackhpc.com Thu Dec 19 16:09:38 2019 From: doug at stackhpc.com (Doug Szumski) Date: Thu, 19 Dec 2019 16:09:38 +0000 Subject: [Telemetry][TC] Monasca in distributions like Debian and Ubuntu In-Reply-To: <418be333-77f2-c749-9556-b4133257327f@goirand.fr> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <418be333-77f2-c749-9556-b4133257327f@goirand.fr> Message-ID: On 19/12/2019 10:31, Thomas Goirand wrote: > On 12/19/19 4:50 AM, Rong Zhu wrote: >> Another worry is Monasca's main contributors are from SUSE. > What I am not confident with, is that Monasca needs Java stuff. In here: > https://github.com/openstack/monasca-common/blob/master/bindep.txt > > we can see openjdk-8-jdk. That's already obsolete. For example, Debian > Buster was released last summer with openjdk 11, and version 8 isn't > available anymore. > > With such situation, I don't think it's reasonable to attempt packaging > Monasca for Debian and Ubuntu, and I'm not even scratching the Java > module dependency issue: these would need to be packaged too, and I have > no idea how much work that would be (can someone tell me?). > > If some upstream contributors of Monasca want to bring more light to > this topic, please feel free. I obviously don't know enough about it. A non-Java Monasca deployment is possible if you don't require alerting. There is a plan to replace the alerting service, but I'm unsure if it will be complete this cycle. > > Cheers, > > Thomas Goirand (zigo) > From vhpc.dist at gmail.com Thu Dec 19 15:55:57 2019 From: vhpc.dist at gmail.com (VHPC 20) Date: Thu, 19 Dec 2019 16:55:57 +0100 Subject: CfP VHPC20: HPC Containers-Kubernetes Message-ID: ==================================================================== CALL FOR PAPERSa 15th Workshop on Virtualization in High-Performance Cloud Computing (VHPC 20) held in conjunction with the International Supercomputing Conference - High Performance, June 21-25, 2020, Frankfurt, Germany. (Springer LNCS Proceedings) ==================================================================== Date: June 25, 2020 Workshop URL: vhpc[dot]org Abstract registration Deadline: Jan 31st, 2020 Paper Submission Deadline: Apr 05th, 2020 Springer LNCS Call for Papers Containers and virtualization technologies constitute key enabling factors for flexible resource management in modern data centers, and particularly in cloud environments. Cloud providers need to manage complex infrastructures in a seamless fashion to support the highly dynamic and heterogeneous workloads and hosted applications customers deploy. Similarly, HPC environments have been increasingly adopting techniques that enable flexible management of vast computing and networking resources, close to marginal provisioning cost, which is unprecedented in the history of scientific and commercial computing. Most recently, Function as a Service (Faas) and Serverless computing, utilizing lightweight VMs-containers widens the spectrum of applications that can be deployed in a cloud environment, especially in an HPC context. Here, HPC-provided services can be become accessible to distributed workloads outside of large cluster environments. Various virtualization-containerization technologies contribute to the overall picture in different ways: machine virtualization, with its capability to enable consolidation of multiple under­utilized servers with heterogeneous software and operating systems (OSes), and its capability to live­-migrate a fully operating virtual machine (VM) with a very short downtime, enables novel and dynamic ways to manage physical servers; OS-­level virtualization (i.e., containerization), with its capability to isolate multiple user­-space environments and to allow for their co­existence within the same OS kernel, promises to provide many of the advantages of machine virtualization with high levels of responsiveness and performance; lastly, unikernels provide for many virtualization benefits with a minimized OS/library surface. I/O Virtualization in turn allows physical network interfaces to take traffic from multiple VMs or containers; network virtualization, with its capability to create logical network overlays that are independent of the underlying physical topology is furthermore enabling virtualization of HPC infrastructures. Publication Accepted papers will be published in a Springer LNCS proceedings volume. Topics of Interest The VHPC program committee solicits original, high-quality submissions related to virtualization across the entire software stack with a special focus on the intersection of HPC, containers-virtualization and the cloud. Major Topics: - HPC workload orchestration (Kubernetes) - Kubernetes HPC batch - HPC Container Environments Landscape - HW Heterogeneity - Container ecosystem (Docker alternatives) - Networking - Lightweight Virtualization - Unikernels / LibOS - State-of-the-art processor virtualization (RISC-V, EPI) - Containerizing HPC Stacks/Apps/Codes: Climate model containers each major topic encompassing design/architecture, management, performance management, modeling and configuration/tooling. Specifically, we invite papers that deal with the following topics: - HPC orchestration (Kubernetes) - Virtualizing Kubernetes for HPC - Deployment paradigms - Multitenancy - Serverless - Declerative data center integration - Network provisioning - Storage - OCI i.a. images - Isolation/security - HW Accelerators, including GPUs, FPGAs, AI, and others - State-of-practice/art, including transition to cloud - Frameworks, system software - Programming models, runtime systems, and APIs to facilitate cloud adoption - Edge use-cases - Application adaptation, success stories - Kubernetes Batch - Scheduling, job management - Execution paradigm - workflow - Data management - Deployment paradigm - Multi-cluster/scalability - Performance improvement - Workflow / execution paradigm - Podman: end-to-end Docker alternative container environment & use-cases - Creating, Running containers as non-root (rootless) - Running rootless containers with MPI - Container live migration - Running containers in restricted environments without setuid - Networking - Software defined networks and network virtualization - New virtualization NICs/Nitro alike ASICs for the data center? - Kubernetes SDN policy (Calico i.a.) - Kubernetes network provisioning (Flannel i.a.) - Lightweight Virtualization - Micro VMMs (Rust-VMM, Firecracker, solo5) - Xen - Nitro hypervisor (KVM) - RVirt - Cloud Hypervisor - Unikernels / LibOS - HPC Storage in Virtualization - HPC container storage - Cloud-native storage - Hypervisors in storage virtualization - Processor Virtualization - RISC-V hypervisor extensions - RISC-V Hypervisor ports - EPI - Composable HPC microservices - Containerizing Scientific Codes - Building - Deploying - Securing - Storage - Monitoring - Use case for containerizing HPC codes: Climate model containers for portability, reproducibility, traceability, immutability, provenance, data & software preservation The Workshop on Virtualization in High­-Performance Cloud Computing (VHPC) aims to bring together researchers and industrial practitioners facing the challenges posed by virtualization in order to foster discussion, collaboration, mutual exchange of knowledge and experience, enabling research to ultimately provide novel solutions for virtualized computing systems of tomorrow. The workshop will be one day in length, composed of 20 min paper presentations, each followed by 10 min discussion sections, plus lightning talks that are limited to 5 minutes. Presentations may be accompanied by interactive demonstrations. Important Dates Jan 31st, 2020 - Abstract Apr 5th, 2020 - Paper submission deadline (Springer LNCS) Apr 26th, 2020 - Acceptance notification June 25th, 2020 - Workshop Day July 10th, 2020 - Camera-ready version due Chair Michael Alexander (chair), BOKU, Vienna, Austria Anastassios Nanos (co-­chair), Sunlight.io, UK Program committee Stergios Anastasiadis, University of Ioannina, Greece Paolo Bonzini, Redhat, Italy Jakob Blomer, CERN, Europe Eduardo César, Universidad Autonoma de Barcelona, Spain Taylor Childers, Argonne National Laboratory, USA Stephen Crago, USC ISI, USA Tommaso Cucinotta, St. Anna School of Advanced Studies, Italy François Diakhaté CEA DAM Ile de France, France Kyle Hale, Northwestern University, USA Brian Kocoloski, Washington University, USA John Lange, University of Pittsburgh, USA Giuseppe Lettieri, University of Pisa, Italy Klaus Ma, Huawei, China Alberto Madonna, Swiss National Supercomputing Center, Switzerland Nikos Parlavantzas, IRISA, France Anup Patel, Western Digital, USA Kevin Pedretti, Sandia National Laboratories, USA Amer Qouneh, Western New England University, USA Carlos Reaño, Queen’s University Belfast, UK Adrian Reber, Redhat, Germany Riccardo Rocha, CERN, Europe Borja Sotomayor, University of Chicago, USA Jonathan Sparks, Cray, USA Kurt Tutschku, Blekinge Institute of Technology, Sweden John Walters, USC ISI, USA Yasuhiro Watashiba, Osaka University, Japan Chao-Tung Yang, Tunghai University, Taiwan Paper Submission-Publication Papers submitted to the workshop will be reviewed by at least two members of the program committee and external reviewers. Submissions should include abstract, keywords, the e-mail address of the corresponding author, and must not exceed 10 pages, including tables and figures at a main font size no smaller than 11 point. Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the conference to present the work. Accepted papers will be published in a Springer LNCS volume. The format must be according to the Springer LNCS Style. Initial submissions are in PDF; authors of accepted papers will be requested to provide source files. Abstract, Paper Submission Link: edas[dot]info/newPaper.php?c=26973 Lightning Talks Lightning Talks are non-paper track, synoptical in nature and are strictly limited to 5 minutes. They can be used to gain early feedback on ongoing research, for demonstrations, to present research results, early research ideas, perspectives and positions of interest to the community. Submit abstract via the main submission link. General Information The workshop is one day in length and will be held in conjunction with the International Supercomputing Conference - High Performance (ISC) 2019, June 21-25, Frankfurt, Germany. From tpb at dyncloud.net Thu Dec 19 17:19:05 2019 From: tpb at dyncloud.net (Tom Barron) Date: Thu, 19 Dec 2019 12:19:05 -0500 Subject: [manila] no weekly meeting 26 December 2019 Message-ID: <20191219171905.f5s4yh2e3f5vki67@barron.net> Just noting that there will be no weekly Manila meeting 26 December. We'll next meet at the scheduled time, 1500 UTC, on freenode #openstack-meeting-alt, on 2 January 2020. [1] See you in the New Year, or in #openstack-manila in the mean time! -- Tom Barron [1] https://wiki.openstack.org/wiki/Manila/Meetings From pierre at stackhpc.com Thu Dec 19 17:21:50 2019 From: pierre at stackhpc.com (Pierre Riteau) Date: Thu, 19 Dec 2019 18:21:50 +0100 Subject: [blazar] IRC meetings cancelled during the holidays Message-ID: Hello, The Blazar IRC meetings of December 24, December 31, and January 2 are cancelled. The next meeting will be on Tuesday, January 7. Have a great holiday, Pierre From cboylan at sapwetik.org Thu Dec 19 18:43:19 2019 From: cboylan at sapwetik.org (Clark Boylan) Date: Thu, 19 Dec 2019 10:43:19 -0800 Subject: =?UTF-8?Q?Re:_[Horizon][Designate][Tacker][Tobiko][OSH][Ironic]_Flaky_jo?= =?UTF-8?Q?bs_and_retries?= In-Reply-To: <85cde474-cc15-420a-b228-e7191ad8e494@www.fastmail.com> References: <85cde474-cc15-420a-b228-e7191ad8e494@www.fastmail.com> Message-ID: <1033b3b0-ebe5-48a5-92a8-4827ac3bf05c@www.fastmail.com> On Mon, Dec 16, 2019, at 10:20 AM, Clark Boylan wrote: > Hello, > > Zuulv3 has a feature where it will rerun jobs that it thinks failed due > to infrastructure problems up to some configured limit. For us the > limit is 3 job attempts. The circumstances where Zuul thinks it has > infrastructure issues are when pre-run playbooks fail (because these > are expected to be stable), and when Ansible returns with an exit code > signifying network connectivity issues. > > Recently I updated Zuul to report the job attempt in job inventories > and am using that to report this data into logstash and elasticsearch. > Now I can use queries like: 'zuul_attempts:"3" AND > filename:"job-output.txt" AND message:"Job console starting"' to see > which jobs are flaky and reattempting. I'm writing this email to report > on some of what I've found in the hopes that the problems we have can > be fixed and hopefully result in better jobs across the board. > snip > Tobiko > > tobiko-devstack-faults-centos-7 is failing because this job runs on > CentOS 7 using python2 but Nova needs python3.6 now. This fails in > pre-run, > https://zuul.opendev.org/t/openstack/build/8071780e7ba748169b447f1d42e069fc/log/controller/logs/devstacklog.txt.gz#11799, and forces the job to be rerun. I'm not sure what the best fix is here. Maybe run your jobs on Ubuntu until CentOS 8 is supported by devstack? > Looks like Tobiko was updated to build python3 on CentOS 7 to address this issue. Unfortunately, it looks like glance imports sqlite and the built python did not include sqlite bindings. This results in g-api failing to start and the job still fails with RETRY_LIMIT, https://zuul.opendev.org/t/openstack/build/880ded9fe11c41da9c7622d85746df23/log/controller/logs/screen-g-api.txt.gz#101. Note that this job ran against the proposed fix and was tested pre merge. You don't need to wait for the change to merge to observe these results. Instead Zuul allows you to push the fix up, confirm it works, then merge the change. The fix for the sqlite issue is in progress here, https://review.opendev.org/#/c/699945/2, but I wanted to point out that Zuul helps us to avoid these problems in the first place with its pre merge testing. Finally, CentOS 7 does provide python3.6 packages in the main repository now. I'm not sure if that is sufficient for Tobiko's needs but that may help simplify things here. Clark From elmiko at redhat.com Thu Dec 19 21:30:27 2019 From: elmiko at redhat.com (Michael McCune) Date: Thu, 19 Dec 2019 16:30:27 -0500 Subject: [API SIG] holiday office hours Message-ID: the API SIG will be closing it's office hours until the new year. have a peaceful and joyous holiday season and new years =) peace o/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Thu Dec 19 21:48:19 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Thu, 19 Dec 2019 15:48:19 -0600 Subject: [qa][nova][cinder] ceph job has been busted on stable/pike for awhile, fix or delete? In-Reply-To: <506823521bc34d6c82b1dda7129a7f59@AUSX13MPS308.AMER.DELL.COM> References: <20191218113015.ppyqla4kjljjhfmx@lyarwood.usersys.redhat.com> <506823521bc34d6c82b1dda7129a7f59@AUSX13MPS308.AMER.DELL.COM> Message-ID: <596c57ec-281d-00a1-bd01-88fddfc9687d@gmail.com> On 12/18/2019 2:20 PM, Arkady.Kanevsky at dell.com wrote: > +1 for delete Done: https://review.opendev.org/#/c/700072/ -- Thanks, Matt From mark at stackhpc.com Thu Dec 19 22:09:39 2019 From: mark at stackhpc.com (Mark Goddard) Date: Thu, 19 Dec 2019 22:09:39 +0000 Subject: [release] decentralising release approvals Message-ID: Hi, If this topic has been discussed many times already then someone please say so and we can leave it alone. As kolla PTL and ironic release liaison I've proposed a number of release patches recently. Generally the release team is good at churning through these, but sometimes patches can hang around for a while. Usually a ping on IRC will get things moving again within a day or so (thanks in particular to Sean who has been very responsive). Related to the recent discussion about decentralising stable branch maintenance, should we do this for releases too? To be clear, I'm proposing giving project teams more control over their own releases. I brought this up in IRC recently and there was a brief discussion. I suggested that releases are hard to undo, and Monty corrected me by saying they are impossible to undo. Still, if teams (or a subset of their members) had more ownership of their releases, they would become more familiar with the process, and the risk would be reduced. I'm not suggesting that release team should be disbanded - there is clearly work to do to maintain the tooling, determine and communicate the schedule, and more. But there's a lot of toil involved in checking and approving patches to the releases repo, and it's done by some of our most senior, busy colleagues. There are a number of checks that are performed automatically by the tooling and used to gate merges. I have a few questions for the release team about these reviews. * What manual checks do you do beyond those that are currently automated? * Could the above checks be automated? * What issues have you caught that were not caught by CI jobs? Hopefully I haven't offended anyone here. There's often more involved with these things than you first suspect. Cheers, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From feilong at catalyst.net.nz Thu Dec 19 22:27:07 2019 From: feilong at catalyst.net.nz (Feilong Wang) Date: Fri, 20 Dec 2019 11:27:07 +1300 Subject: [magnum] Next weekly team meeting will be on 22 Jan 2020 Message-ID: <82af2fbe-0021-d42d-bdfb-fad0933dd9e7@catalyst.net.nz> Hi team, We just had a good year and we deserve some rest. We will skip some meetings and the next one will be on 22 Jan 2020. Hope you all will have a merry Christmas and happy New Year. -- Cheers & Best regards, Feilong Wang (王飞龙) Head of R&D Catalyst Cloud - Cloud Native New Zealand -------------------------------------------------------------------------- Tel: +64-48032246 Email: flwang at catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington -------------------------------------------------------------------------- From anlin.kong at gmail.com Fri Dec 20 00:27:54 2019 From: anlin.kong at gmail.com (Lingxian Kong) Date: Fri, 20 Dec 2019 13:27:54 +1300 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> Message-ID: I'd prefer we fork Gnocchi to opendev/gerrit so the developers could have the same experiences with other openstack projects. Before people fixing any bugs or developing any features, the first task is to set up the CI and publish docs in openstack website. I can offer some help as well. - Best regards, Lingxian Kong Catalyst Cloud On Fri, Dec 20, 2019 at 3:06 AM Thierry Carrez wrote: > Tobias Urdin wrote: > > I can volounteer to spend time on Ceilomter and Gnocchi while I have > > some mimimal knowledge on Ceilomter > > and pretty much none on the Gnocchi codebase I would like to see the > > project continued. > > > > Another thing would be if Gnocchi should be moved back or if I should > > somehow get in contact with the former > > Gnocchi maintainers and see if we can get access to GitHub? > > If the goal is to maintain Gnocchi as-is for the narrow OpenStack use > case, my preference would be to fork / move it back to opendev, under a > gnocchi namespace. That would ensure a clean cut from the unmaintained > "gnocchi for all the things" project on GitHub, and reset expectations. > > If the goal is to take over Gnocchi and support all other use cases, > then maybe taking it over on GitHub might be easier to ensure continuity. > > -- > Thierry Carrez (ttx) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fsbiz at yahoo.com Fri Dec 20 01:27:41 2019 From: fsbiz at yahoo.com (fsbiz at yahoo.com) Date: Fri, 20 Dec 2019 01:27:41 +0000 (UTC) Subject: [ironic] - automated cleaning References: <1250556611.1729538.1576805261824.ref@mail.yahoo.com> Message-ID: <1250556611.1729538.1576805261824@mail.yahoo.com> We have enabled automatic cleaning and I have noticed that our UEFI enabled nodes most often end up in "clean failed" state. Basic troubleshooting reveals that PXE correctly boots IPXE, but the IPXE goes and boots the OS instead of downloading the deployed_kernel / ramdisk. Here's the timeline of events.Dec 18 12:19:39 sc-ironic04 Node 56e58642-12ac-4455-bc95-2a328198f845 moved to provision state "cleaning"Dec 18 12:20:32 sc-ironic04 Successfully set node 56e58642-12ac-4455-bc95-2a328198f845 power state to power onDec 18 12:21:14  DHCPACK(ns-601d0738-39) 10.33.23.7 6c:b3:11:4f:8b:18 (PXE Boot gets IP: 10.33.23.7)Dec 18 12:21:15 sc-ironic04 in.tftpd[367896]: Client 10.33.23.7 finished ipxe_x86_64.efi (TFTP of IPXE is complete)Dec 18 12:21:23 DHCPACK(ns-261f35c5-7e) 10.33.23.7 6c:b3:11:4f:8b:18 host-10-33-23-7 (IPXE acquires IP: 10.33.23.7)NO HTTP request for deploy_kernel Instead:Dec 18 12:22:40 sc-control04 dnsmasq-dhcp[3508449]: 3990148443 DHCPREQUEST(ns-261f35c5-7e) 10.33.22.188 6c:b3:11:4f:8b:18Dec 18 12:22:40 sc-control04 dnsmasq-dhcp[3508449]: 3990148443 DHCPNAK(ns-261f35c5-7e) 10.33.22.188 6c:b3:11:4f:8b:18 wrong address When the OS was running it had IP address of 10.33.22.188.  The OS came up and tries to renew its lease for 10.33.22.188 which was NAKed by the DHCP server. The OS then did a full DHCP (DISCOVER, etc.), got a valid IP address, did a cloud-init, etc.   Is this a known issue that has been fixed ? Any pointers would be appreciated. thanks,Farad. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sorrison at gmail.com Fri Dec 20 03:27:46 2019 From: sorrison at gmail.com (Sam Morrison) Date: Fri, 20 Dec 2019 14:27:46 +1100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> Message-ID: <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> I am happy to help out with Gnocchi too. I have experience with the code base and wrote an influxdb backend for gnocchi but it was around the time it got split off from openstack and it never made it in. [1] (we use an updated patch in production) I can’t think of much needed in the way of extra features for our use case but I would be able to help out on maintenance and also CI issues. Cheers. Sam [1] https://review.opendev.org/#/c/390260/ > On 20 Dec 2019, at 11:27 am, Lingxian Kong wrote: > > I'd prefer we fork Gnocchi to opendev/gerrit so the developers could > have the same experiences with other openstack projects. Before people > fixing any bugs or developing any features, the first task is to set up > the CI and publish docs in openstack website. > > I can offer some help as well. > > - > Best regards, > Lingxian Kong > Catalyst Cloud > > > On Fri, Dec 20, 2019 at 3:06 AM Thierry Carrez > wrote: > Tobias Urdin wrote: > > I can volounteer to spend time on Ceilomter and Gnocchi while I have > > some mimimal knowledge on Ceilomter > > and pretty much none on the Gnocchi codebase I would like to see the > > project continued. > > > > Another thing would be if Gnocchi should be moved back or if I should > > somehow get in contact with the former > > Gnocchi maintainers and see if we can get access to GitHub? > > If the goal is to maintain Gnocchi as-is for the narrow OpenStack use > case, my preference would be to fork / move it back to opendev, under a > gnocchi namespace. That would ensure a clean cut from the unmaintained > "gnocchi for all the things" project on GitHub, and reset expectations. > > If the goal is to take over Gnocchi and support all other use cases, > then maybe taking it over on GitHub might be easier to ensure continuity. > > -- > Thierry Carrez (ttx) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From akekane at redhat.com Fri Dec 20 06:36:53 2019 From: akekane at redhat.com (Abhishek Kekane) Date: Fri, 20 Dec 2019 12:06:53 +0530 Subject: [glance] no meeting on 26 December 2019 Message-ID: As discussed during yesterday's meeting, we will not hold the glance weekly meeting on 26th December 2019. The next meeting will be on 2 January 2020. Wish you all a happy christmas and new year in advance. Thanks & Best Regards, Abhishek Kekane -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Fri Dec 20 07:38:15 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Fri, 20 Dec 2019 08:38:15 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> Message-ID: +1 for forking. But that would probably mandate a slight rename to avoid confusion and announce the change in priorities (ceilometer backend vs general-purpose [however unlikely in practice]). Happy to help from Kolla(-Ansible) side to have it deployed. -yoctozepto pt., 20 gru 2019 o 04:35 Sam Morrison napisał(a): > > I am happy to help out with Gnocchi too. I have experience with the code base and wrote an influxdb backend for gnocchi but it was around the time it got split off from openstack and it never made it in. [1] (we use an updated patch in production) > I can’t think of much needed in the way of extra features for our use case but I would be able to help out on maintenance and also CI issues. > > Cheers. > Sam > > [1] https://review.opendev.org/#/c/390260/ > > > > On 20 Dec 2019, at 11:27 am, Lingxian Kong wrote: > > I'd prefer we fork Gnocchi to opendev/gerrit so the developers could > have the same experiences with other openstack projects. Before people > fixing any bugs or developing any features, the first task is to set up > the CI and publish docs in openstack website. > > I can offer some help as well. > > - > Best regards, > Lingxian Kong > Catalyst Cloud > > > On Fri, Dec 20, 2019 at 3:06 AM Thierry Carrez wrote: >> >> Tobias Urdin wrote: >> > I can volounteer to spend time on Ceilomter and Gnocchi while I have >> > some mimimal knowledge on Ceilomter >> > and pretty much none on the Gnocchi codebase I would like to see the >> > project continued. >> > >> > Another thing would be if Gnocchi should be moved back or if I should >> > somehow get in contact with the former >> > Gnocchi maintainers and see if we can get access to GitHub? >> >> If the goal is to maintain Gnocchi as-is for the narrow OpenStack use >> case, my preference would be to fork / move it back to opendev, under a >> gnocchi namespace. That would ensure a clean cut from the unmaintained >> "gnocchi for all the things" project on GitHub, and reset expectations. >> >> If the goal is to take over Gnocchi and support all other use cases, >> then maybe taking it over on GitHub might be easier to ensure continuity. >> >> -- >> Thierry Carrez (ttx) >> > From marcin.juszkiewicz at linaro.org Fri Dec 20 08:00:12 2019 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Fri, 20 Dec 2019 09:00:12 +0100 Subject: [kolla] broken images pushed Message-ID: We have merged publisher for aarch64 images and it turned out that it was done wrong way. Today both x86-64 and aarch64 images got published at same time so "kolla/debian-source-*" images are broken: 08:55 (0s) hrw at puchatek:~$ docker run --rm -it kolla/debian-source-heat-engine:master bash standard_init_linux.go:211: exec user process caused "exec format error" This is result of aarch64 image overwriting x86-64 one. We reverted arm64 publisher and will work on it in January. Sorry for potential problem. From amotoki at gmail.com Fri Dec 20 08:00:12 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Fri, 20 Dec 2019 17:00:12 +0900 Subject: [Horizon][Designate][Tacker][Tobiko][OSH][Ironic] Flaky jobs and retries In-Reply-To: <85cde474-cc15-420a-b228-e7191ad8e494@www.fastmail.com> References: <85cde474-cc15-420a-b228-e7191ad8e494@www.fastmail.com> Message-ID: On Tue, Dec 17, 2019 at 3:22 AM Clark Boylan wrote: > > Hello, > > Zuulv3 has a feature where it will rerun jobs that it thinks failed due to infrastructure problems up to some configured limit. For us the limit is 3 job attempts. The circumstances where Zuul thinks it has infrastructure issues are when pre-run playbooks fail (because these are expected to be stable), and when Ansible returns with an exit code signifying network connectivity issues. > > Recently I updated Zuul to report the job attempt in job inventories and am using that to report this data into logstash and elasticsearch. Now I can use queries like: 'zuul_attempts:"3" AND filename:"job-output.txt" AND message:"Job console starting"' to see which jobs are flaky and reattempting. I'm writing this email to report on some of what I've found in the hopes that the problems we have can be fixed and hopefully result in better jobs across the board. > > Designate Horizon Dashboard > > The nodejs jobs for stable/rocky expect to run against nodejs4 but run on Bionic which has no nodejs 4 packaging. This causes the pre-run to fail as it cannot install the requested package, https://zuul.opendev.org/t/openstack/build/498ebb052e2b4a3393b0939820ee8927/log/job-output.txt#391. Fix in progress here, https://review.opendev.org/#/c/699241/1, to pin to Xenial. Thank you! > > Horizon projects and dashboards may want to double check they don't have similar problems with nodejs version and node mismatches. "nodejs4-jobs job" in stable/rocky specifies "nodeset: ubuntu-xenial" in openstack-zuul-jobs, but most horizon plugins requires horizon as required-projects so they usually redefine nodejs4 related jobs. I checked about half of horizon plugins and confirm they specify "nodeset: ubuntu-xenial" if they redefine the nodejs4 jobs with openstack/horizon. Perhaps an ideal solution would be that the horizon team split out stuffs used by horizon plugins into a separate project like horizon-lib, but considering the current development resources I don't think it can happen in near future unless someone steps up. Thanks, Akihiro Motoki (irc: amotoki) > > Tacker > > tacker-functional-devstack-multinode-python3 (and possibly other tacker devstack jobs) attempt to download an openwrt image during devstack runs and the server isn't responding. This fails in pre-run, http://zuul.openstack.org/build/27ff514c77724250968d60469923f613/log/controller/logs/devstacklog.txt.gz#45426, and results in the job being retried. I am not aware of a fix for this, but you'll likely need to find a new image host. Let the infra team know if we can help. > > Tobiko > > tobiko-devstack-faults-centos-7 is failing because this job runs on CentOS 7 using python2 but Nova needs python3.6 now. This fails in pre-run, https://zuul.opendev.org/t/openstack/build/8071780e7ba748169b447f1d42e069fc/log/controller/logs/devstacklog.txt.gz#11799, and forces the job to be rerun. I'm not sure what the best fix is here. Maybe run your jobs on Ubuntu until CentOS 8 is supported by devstack? > > OpenStack Helm > > Ansible has decided that become_user is not valid on include_role tasks. https://opendev.org/openstack/openstack-helm-infra/src/branch/master/roles/deploy-docker/tasks/deploy-ansible-docker-support.yaml#L26-L49 seems to be the issue. This causes Airship's deckhand functional docker jobs to fail in pre-run, https://zuul.opendev.org/t/openstack/build/7493038ee1744465b2387b44e067d029/log/job-output.txt#396, and the jobs are retried. It is possible this is fallout from Zuul's default ansible version being bumped to 2.8 from 2.7. > > This also seems to affect OSH's kubernetes keystone auth job, https://zuul.opendev.org/t/openstack/build/68d937b3e5e3449cb6fe2e6947bbf0db/log/job-output.txt#397 > > Ironic > > The Ironic example is a bit different and runs into some fun Ansible behaviors. Ironic's IPA multinode jobs (possibly others too) are filling the root disk of the test nodes on some clouds, http://paste.openstack.org/show/787641/. Ansible uses /tmp to bootstrap its ssh connectivity and when /tmp is on / and / is full that results in Ansible returning a network failure error code. This then causes Zuul to rerun the job. Unfortunately because Ansible thinks networking is broken we have to do special things in a Zuul cleanup playbook to double check disk usage and that info is only available on the Zuul executors. But I've double checked for you and can confirm this is what is happening. > > The fix here is to use the ephemeral disk which is mounted on /opt and contains much more disk space. Another option would be to reduce the size of Ironic's fake baremetal images. In any case they are aware of this and we should expect a fix soon. > > There are likely more cases I haven't found yet, but I wanted to point out this Zuul behavior to people. The intent is that it handle exceptional cases and not paper over failures that happen often or even 100% of the time. We should do our best to fix these problems when they pop up. Thank you to everyone that has already stepped up to help fix some of these! > > Clark > From marcin.juszkiewicz at linaro.org Fri Dec 20 08:09:54 2019 From: marcin.juszkiewicz at linaro.org (Marcin Juszkiewicz) Date: Fri, 20 Dec 2019 09:09:54 +0100 Subject: [kolla] broken images pushed In-Reply-To: References: Message-ID: <67810df6-8463-6284-38d8-40d5f5437137@linaro.org> W dniu 20.12.2019 o 09:00, Marcin Juszkiewicz pisze: > We have merged publisher for aarch64 images and it turned out that > it was done wrong way. > > Today both x86-64 and aarch64 images got published at same time so > "kolla/debian-source-*" images are broken: > > 08:55 (0s) hrw at puchatek:~$ docker run --rm -it > kolla/debian-source-heat-engine:master bash > standard_init_linux.go:211: exec user process caused "exec format > error" > > This is result of aarch64 image overwriting x86-64 one. We reverted > arm64 publisher and will work on it in January. > > Sorry for potential problem. One of potential solution for future is building x86-64 images and pushing them with 'master-x86_64' tag, pushing aarch64 images with 'master-aarch64' tag and then once both publish run 3rd job to create multiarch images tagged 'master'. https://medium.com/@mauridb/docker-multi-architecture-images-365a44c26be6 From thierry at openstack.org Fri Dec 20 10:04:36 2019 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 20 Dec 2019 11:04:36 +0100 Subject: [release] making releases fast again (was: decentralising release approvals) In-Reply-To: References: Message-ID: Mark Goddard wrote: > [...] > As kolla PTL and ironic release liaison I've proposed a number of > release patches recently. Generally the release team is good at churning > through these, but sometimes patches can hang around for a while. > Usually a ping on IRC will get things moving again within a day or so > (thanks in particular to Sean who has been very responsive). I agree we've seen an increase in processing delay lately, and I'd like to correct that. There are generally three things that would cause a perceptible delay in release processing... 1- wait for two release managers +2 This is something we put in place some time ago, as we had a lot of new members and thought that would be a good way to onboard them. Lately it created delays as a lot of those were not as active. 2- stable releases Two subcases in there... Eitherthe deliverable is under stable policy and there are *significant* delays there as we have to pause to give a chance to stable-maint-core people to voice an opinion. Or the deliverable is not under stable policy, but we do a manual check on the changes, as a way to educate the requester on semver. 3- waiting for PTL/release liaison to approve That can take a long time, but the release management team is not really at fault there. Could you describe where you've seen "sometimes patches can hang around for a while"? I suspect they belong in the (2) category? > [...] > I have a few questions for the release team about these reviews. > > * What manual checks do you do beyond those that are currently automated? See https://releases.openstack.org/reference/reviewer_guide.html > * Could the above checks be automated? We aggressively automate everything that can be. Like I'm currently working to automate the check that the release was approved by the PTL or release liaison. > * What issues have you caught that were not caught by CI jobs? It's generally semver violations, or timing issues (like requesting a release during a freeze). Sometimes it's corner cases not handled (yet) by automation, like incompatibility between the release version asked and the deliverable release model. You can look at the history of releases for examples. > Hopefully I haven't offended anyone here. There's often more involved > with these things than you first suspect. Decentralizing would be a lot of work to create new systems and processes... and I don't think we can automate everything. It's unreasonable to expect everyone to know the release process by heart and respect timing and freezes. And releases are the only thing we produce that we can't undo. I would rather eliminate the issue by making sure release processing is back to fast. So here is my proposal: - go back to single release manager approval - directly approve stable releases after a cursory semver check, not waiting for stable-maint-core approval. That should make sure all releases are processed within a couple of days, which I think is a good trade-off between retaining some releases for 10+ days and not having a chance to catch odd cases before releases at all. Thoughts? -- Thierry Carrez (ttx) From thierry at openstack.org Fri Dec 20 10:43:22 2019 From: thierry at openstack.org (Thierry Carrez) Date: Fri, 20 Dec 2019 11:43:22 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> Message-ID: Radosław Piliszek wrote: > +1 for forking. But that would probably mandate a slight rename to > avoid confusion and announce the change in priorities (ceilometer > backend vs general-purpose [however unlikely in practice]). +1 to renaming if we change the scope of the original project. Something like "ceilodb" would make it clear it's just a data backend for Ceilometer... but I'll let the volunteers taking it over choose :) Happy to help with the infrastructure side of things. -- Thierry From mark at stackhpc.com Fri Dec 20 14:30:56 2019 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 20 Dec 2019 14:30:56 +0000 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> Message-ID: On Fri, 20 Dec 2019, 10:44 Thierry Carrez, wrote: > Radosław Piliszek wrote: > > +1 for forking. But that would probably mandate a slight rename to > > avoid confusion and announce the change in priorities (ceilometer > > backend vs general-purpose [however unlikely in practice]). > > +1 to renaming if we change the scope of the original project. Something > like "ceilodb" would make it clear it's just a data backend for > Ceilometer... but I'll let the volunteers taking it over choose :) > Sounds like a bad idea IMO. I'm sure the codebase is littered with the name gnocchi, and everyone knows it as such. Projects can change scope and direction over time, that's ok if it's advertised. > Happy to help with the infrastructure side of things. > > -- > Thierry > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Fri Dec 20 14:46:31 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Fri, 20 Dec 2019 15:46:31 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> Message-ID: pt., 20 gru 2019 o 15:34 Mark Goddard napisał(a): > On Fri, 20 Dec 2019, 10:44 Thierry Carrez, wrote: >> Radosław Piliszek wrote: >> > +1 for forking. But that would probably mandate a slight rename to >> > avoid confusion and announce the change in priorities (ceilometer >> > backend vs general-purpose [however unlikely in practice]). >> >> +1 to renaming if we change the scope of the original project. Something >> like "ceilodb" would make it clear it's just a data backend for >> Ceilometer... but I'll let the volunteers taking it over choose :) > > > Sounds like a bad idea IMO. I'm sure the codebase is littered with the name gnocchi, and everyone knows it as such. Projects can change scope and direction over time, that's ok if it's advertised. If advertised, approved and if you are really taking full ownership... Moving it from github to opendev as is would definitely cause some identity trouble. That said, it might be the case we are talking about ghosts - if gnocchi is used like 99.9% as ceilometer backend then it's probably fine whatever we do to make it thrive. Can we somehow get people from gnocchi to discuss the reality? -yoctozepto From luka.peschke at objectif-libre.com Fri Dec 20 15:13:02 2019 From: luka.peschke at objectif-libre.com (Luka Peschke) Date: Fri, 20 Dec 2019 16:13:02 +0100 Subject: [cloudkitty] Stepping down from PTL Message-ID: <6a879c96c9aa82cc31f4ffde7a6b2663@objectif-libre.com> Hi all, I'm moving to a new position that doesn't involve OpenStack, and won't leave me the required time to be Cloudkitty's PTL. This is why I have to step down from the PTL position. jferrieu will take my position for the end of the U cycle (he's been a major contributor recently), with the help of huats, who's been the Cloudkitty PTL before me, and has been around in the community for a long time. I've been the PTL for two and a half cycles, and I think that it is a good thing for the project to take a new lead, with a new vision. I'm grateful for my experience within the OpenStack community. Thanks, -- Luka Peschke (peschk_l) From mriedemos at gmail.com Fri Dec 20 15:16:50 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 20 Dec 2019 09:16:50 -0600 Subject: [release] making releases fast again In-Reply-To: References: Message-ID: <808012c9-7f06-743b-8a52-da0db22c046c@gmail.com> On 12/20/2019 4:04 AM, Thierry Carrez wrote: > - directly approve stable releases after a cursory semver check, not > waiting for stable-maint-core approval. Rather than go this drastic, how about just not waiting for stable-maint-core (tonyb or myself) to review a stable branch release proposal but if you're going to decentralize stable core to be per-project rather than stable-maint-core, then do like the non-stable release request PTL/liaison thing and ack once one of the per-project stable maint cores signs off on the release or is the person requesting the stable branch release? IOW, don't go from super high standard stable policy release review check to no check, but push the burden to per-project stable cores since they want the responsibility and this is part of the deal. -- Thanks, Matt From Albert.Braden at synopsys.com Fri Dec 20 17:40:41 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 20 Dec 2019 17:40:41 +0000 Subject: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? Message-ID: Running Rocky with a single cell; we have VMs that do not show up in "openstack server list -all-projects" I can see them with "openstack server show" or with "openstack server list -all-projects -host" or by sourcing the credentials for their project and then "openstack server list" If there is a limit on the number of results, that would explain it. Is anyone else seeing this? Here's an example: UUID: d3302d8d-3747-4665-b044-863c8d8a6855 Hostname: us01bctest-centos-57241-5 IP: 10.195.72.66 Project: vgcloud Network: vg-network root at us01odc-p02-ctrl1:~# source admin-openrc root at us01odc-p02-ctrl1:~# openstack server list --all-projects|grep 10.195.72.66 root at us01odc-p02-ctrl1:~# source vg-openrc root at us01odc-p02-ctrl1:~# openstack server list|grep 10.195.72.66 | d3302d8d-3747-4665-b044-863c8d8a6855 | us01bctest-centos-57241-5 | ACTIVE | vg-network=10.195.72.66 | QSC-P-CentOS6.6-19P1-v4 | s1.16cx120g | root at us01odc-p02-ctrl1:~# source admin-openrc root at us01odc-p02-ctrl1:~# openstack server show d3302d8d-3747-4665-b044-863c8d8a6855 +-------------------------------------+----------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+----------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | us01-p02-hv017 | | OS-EXT-SRV-ATTR:hypervisor_hostname | us01-p02-hv017.internal.synopsys.com | | OS-EXT-SRV-ATTR:instance_name | instance-000002b6 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-11-02T01:20:12.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | vg-network=10.195.72.66 | | config_drive | | | created | 2019-11-02T01:15:45Z | | flavor | s1.16cx120g (3cfd8fbc-55da-4c1d-a717-ed86d3a0219f) | | hostId | bf015a3f08917dfc7d7b4d9e429cbe714fcdd11ab788086062fe0eab | | id | d3302d8d-3747-4665-b044-863c8d8a6855 | | image | QSC-P-CentOS6.6-19P1-v4 (91497018-7b42-4448-aa2a-4c08fc9621ac) | | key_name | None | | name | us01bctest-centos-57241-5 | | progress | 0 | | project_id | 0b591e0864d04e6b8b6afa18a5a4a638 | | properties | BU='SCE', Farm='bctest', FarmProject='bnormal', Owner='joelg', Workspace='us01bctest-centos', project='BCTest' | | security_groups | name='default' | | status | ACTIVE | | updated | 2019-11-02T01:20:12Z | | user_id | 0df75a295783f2f7c3843642f3a170f500bbeb4c69806bfd53e166b772617bf4 | | volumes_attached | | +-------------------------------------+----------------------------------------------------------------------------------------------------------------+ root at us01odc-p02-ctrl1:~# openstack server list --all-projects --host us01-p02-hv017|grep 10.195.72.66 | d3302d8d-3747-4665-b044-863c8d8a6855 | us01bctest-centos-57241-5 | ACTIVE | vg-network=10.195.72.66 | QSC-P-CentOS6.6-19P1-v4 | s1.16cx120g | root at us01odc-p02-ctrl1:~# openstack server list --all-projects|grep 10.195.72.66 root at us01odc-p02-ctrl1:~# -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Fri Dec 20 17:47:08 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 20 Dec 2019 11:47:08 -0600 Subject: State of the Gate (placement?) In-Reply-To: References: <20191031211535.vk7rtiq3pvsb6j2t@skaplons-mac> <8a6f54cb-76a7-4522-8a16-93822c4cdcb5@www.fastmail.com> Message-ID: <727395bb-a1ab-5a62-0ca2-e0f87d867737@gmail.com> On 11/1/2019 10:22 AM, Matt Riedemann wrote: > On 11/1/2019 9:55 AM, Clark Boylan wrote: >> OVH controls the disk IOPs that we get pretty aggressively as well. >> Possible it is an IO thing? > > Yeah, so looking at the dstat output in that graph (thanks for pointing > out that site, really nice) we basically have 0 I/O from 16:53 to 16:55, > so uh, that's probably not good. Dredging this back up after discussing with Clark in IRC today. I've posted a patch to disable c-bak and etcd3 in grenade jobs [1]. The hope is this will alleviate some pressure on OVH nodes where iops are restricted. We aren't even effectively doing anything with c-bak and etcd3 in grenade jobs so this should not be controversial. [1] https://review.opendev.org/#/c/700214/ -- Thanks, Matt From Albert.Braden at synopsys.com Fri Dec 20 17:54:27 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 20 Dec 2019 17:54:27 +0000 Subject: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? In-Reply-To: References: Message-ID: Turns out it is max_limit in /etc/nova/nova.conf. From: Albert Braden Sent: Friday, December 20, 2019 9:41 AM To: openstack-discuss at lists.openstack.org Subject: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? Running Rocky with a single cell; we have VMs that do not show up in "openstack server list -all-projects" I can see them with "openstack server show" or with "openstack server list -all-projects -host" or by sourcing the credentials for their project and then "openstack server list" If there is a limit on the number of results, that would explain it. Is anyone else seeing this? Here's an example: UUID: d3302d8d-3747-4665-b044-863c8d8a6855 Hostname: us01bctest-centos-57241-5 IP: 10.195.72.66 Project: vgcloud Network: vg-network root at us01odc-p02-ctrl1:~# source admin-openrc root at us01odc-p02-ctrl1:~# openstack server list --all-projects|grep 10.195.72.66 root at us01odc-p02-ctrl1:~# source vg-openrc root at us01odc-p02-ctrl1:~# openstack server list|grep 10.195.72.66 | d3302d8d-3747-4665-b044-863c8d8a6855 | us01bctest-centos-57241-5 | ACTIVE | vg-network=10.195.72.66 | QSC-P-CentOS6.6-19P1-v4 | s1.16cx120g | root at us01odc-p02-ctrl1:~# source admin-openrc root at us01odc-p02-ctrl1:~# openstack server show d3302d8d-3747-4665-b044-863c8d8a6855 +-------------------------------------+----------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+----------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | us01-p02-hv017 | | OS-EXT-SRV-ATTR:hypervisor_hostname | us01-p02-hv017.internal.synopsys.com | | OS-EXT-SRV-ATTR:instance_name | instance-000002b6 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-11-02T01:20:12.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | vg-network=10.195.72.66 | | config_drive | | | created | 2019-11-02T01:15:45Z | | flavor | s1.16cx120g (3cfd8fbc-55da-4c1d-a717-ed86d3a0219f) | | hostId | bf015a3f08917dfc7d7b4d9e429cbe714fcdd11ab788086062fe0eab | | id | d3302d8d-3747-4665-b044-863c8d8a6855 | | image | QSC-P-CentOS6.6-19P1-v4 (91497018-7b42-4448-aa2a-4c08fc9621ac) | | key_name | None | | name | us01bctest-centos-57241-5 | | progress | 0 | | project_id | 0b591e0864d04e6b8b6afa18a5a4a638 | | properties | BU='SCE', Farm='bctest', FarmProject='bnormal', Owner='joelg', Workspace='us01bctest-centos', project='BCTest' | | security_groups | name='default' | | status | ACTIVE | | updated | 2019-11-02T01:20:12Z | | user_id | 0df75a295783f2f7c3843642f3a170f500bbeb4c69806bfd53e166b772617bf4 | | volumes_attached | | +-------------------------------------+----------------------------------------------------------------------------------------------------------------+ root at us01odc-p02-ctrl1:~# openstack server list --all-projects --host us01-p02-hv017|grep 10.195.72.66 | d3302d8d-3747-4665-b044-863c8d8a6855 | us01bctest-centos-57241-5 | ACTIVE | vg-network=10.195.72.66 | QSC-P-CentOS6.6-19P1-v4 | s1.16cx120g | root at us01odc-p02-ctrl1:~# openstack server list --all-projects|grep 10.195.72.66 root at us01odc-p02-ctrl1:~# -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Fri Dec 20 18:05:39 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 20 Dec 2019 12:05:39 -0600 Subject: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? In-Reply-To: References: Message-ID: <8a20e354-9e15-8083-52c3-682e78e8e49d@gmail.com> On 12/20/2019 11:54 AM, Albert Braden wrote: > Turns out it is max_limit in /etc/nova/nova.conf. Yeah: https://docs.openstack.org/nova/latest/configuration/config.html#api.max_limit See: https://docs.openstack.org/api-guide/compute/paginated_collections.html -- Thanks, Matt From Albert.Braden at synopsys.com Fri Dec 20 20:28:36 2019 From: Albert.Braden at synopsys.com (Albert Braden) Date: Fri, 20 Dec 2019 20:28:36 +0000 Subject: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? In-Reply-To: <8a20e354-9e15-8083-52c3-682e78e8e49d@gmail.com> References: <8a20e354-9e15-8083-52c3-682e78e8e49d@gmail.com> Message-ID: That's very useful, thank you! Is it possible to pull the value of max_limit from the Nova API, or does my script have to grep it from the config file? -----Original Message----- From: Matt Riedemann Sent: Friday, December 20, 2019 10:06 AM To: openstack-discuss at lists.openstack.org Subject: Re: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? On 12/20/2019 11:54 AM, Albert Braden wrote: > Turns out it is max_limit in /etc/nova/nova.conf. Yeah: https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_nova_latest_configuration_config.html-23api.max-5Flimit&d=DwIC-g&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=fw4LQMotcUaOT7JswRsLNAebebvAh1OraMveMnAAg6E&s=ody0k39ys8bbjMl6E5BKbSje7S-yx760EtWiiOdDHMc&e= See: https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.openstack.org_api-2Dguide_compute_paginated-5Fcollections.html&d=DwIC-g&c=DPL6_X_6JkXFx7AXWqB0tg&r=XrJBXYlVPpvOXkMqGPz6KucRW_ils95ZMrEmlTflPm8&m=fw4LQMotcUaOT7JswRsLNAebebvAh1OraMveMnAAg6E&s=n3XKVn2O-C2KBTN0ELtRSrar8WSq57Q74SnTTj6XSuA&e= -- Thanks, Matt From aj at suse.com Fri Dec 20 20:38:20 2019 From: aj at suse.com (Andreas Jaeger) Date: Fri, 20 Dec 2019 21:38:20 +0100 Subject: [fuel][fuel-ccp] Retire Fuel and fuel-ccp repositories In-Reply-To: References: Message-ID: Part of fuel was also the openstack/shotgun repository, I'll retire that one now formally as well, it was marked as retired but not done properly, Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From mriedemos at gmail.com Fri Dec 20 20:41:22 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 20 Dec 2019 14:41:22 -0600 Subject: Is there a limit on the number of VMs returned by "openstack server list --all-projects" ? In-Reply-To: References: <8a20e354-9e15-8083-52c3-682e78e8e49d@gmail.com> Message-ID: On 12/20/2019 2:28 PM, Albert Braden wrote: > Is it possible to pull the value of max_limit from the Nova API, or does my script have to grep it from the config file? You'd have to get it from config. The end user of the cloud hitting the REST API should know as little about how the actual cloud is configured as possible unless it's available in some discoverable way like the version document or if some services have things like capabilities APIs. 403 responses for what you're allowed to do by policy is another discoverability thing but doesn't really help you here. -- Thanks, Matt From mark at stackhpc.com Fri Dec 20 20:52:01 2019 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 20 Dec 2019 20:52:01 +0000 Subject: [release] making releases fast again (was: decentralising release approvals) In-Reply-To: References: Message-ID: On Fri, 20 Dec 2019, 10:06 Thierry Carrez, wrote: > Mark Goddard wrote: > > [...] > > As kolla PTL and ironic release liaison I've proposed a number of > > release patches recently. Generally the release team is good at churning > > through these, but sometimes patches can hang around for a while. > > Usually a ping on IRC will get things moving again within a day or so > > (thanks in particular to Sean who has been very responsive). > > I agree we've seen an increase in processing delay lately, and I'd like > to correct that. There are generally three things that would cause a > perceptible delay in release processing... > > 1- wait for two release managers +2 > > This is something we put in place some time ago, as we had a lot of new > members and thought that would be a good way to onboard them. Lately it > created delays as a lot of those were not as active. > 2- stable releases > > Two subcases in there... Eitherthe deliverable is under stable policy > and there are *significant* delays there as we have to pause to give a > chance to stable-maint-core people to voice an opinion. Or the > deliverable is not under stable policy, but we do a manual check on the > changes, as a way to educate the requester on semver. > > 3- waiting for PTL/release liaison to approve > > That can take a long time, but the release management team is not really > at fault there. > > Could you describe where you've seen "sometimes patches can hang around > for a while"? I suspect they belong in the (2) category? > I hadn't realised there was a requirement for the stable team to review stable patches. That could explain some of my experience. It could also be due to kolla being cycle-trailing, we often make releases at unusual times. > > > [...] > > I have a few questions for the release team about these reviews. > > > > * What manual checks do you do beyond those that are currently automated? > > See https://releases.openstack.org/reference/reviewer_guide.html > > > * Could the above checks be automated? > > We aggressively automate everything that can be. Like I'm currently > working to automate the check that the release was approved by the PTL > or release liaison. > > > * What issues have you caught that were not caught by CI jobs? > > It's generally semver violations, or timing issues (like requesting a > release during a freeze). Sometimes it's corner cases not handled (yet) > by automation, like incompatibility between the release version asked > and the deliverable release model. You can look at the history of > releases for examples. > > > Hopefully I haven't offended anyone here. There's often more involved > > with these things than you first suspect. > > Decentralizing would be a lot of work to create new systems and > processes... and I don't think we can automate everything. It's > unreasonable to expect everyone to know the release process by heart and > respect timing and freezes. And releases are the only thing we produce > that we can't undo. > > I would rather eliminate the issue by making sure release processing is > back to fast. So here is my proposal: > > - go back to single release manager approval > This seems like it should make a big difference - reducing the load on reviewers and the requirements for approval should reduce time in flight. > - directly approve stable releases after a cursory semver check, not > waiting for stable-maint-core approval. > That should make sure all releases are processed within a couple of > days, which I think is a good trade-off between retaining some releases > for 10+ days and not having a chance to catch odd cases before releases > at all. > > Thoughts? > Thanks for the detailed response. I tend to prefer models where teams can be self-sufficient using shared tooling and policies, but I'm also missing some context and history, and don't have to clean up when things go wrong. Ultimately, you've proposed some simple changes which should improve the situation, so that's a good result in my view. > -- > Thierry Carrez (ttx) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at stackhpc.com Fri Dec 20 20:53:40 2019 From: mark at stackhpc.com (Mark Goddard) Date: Fri, 20 Dec 2019 20:53:40 +0000 Subject: [release] making releases fast again In-Reply-To: <808012c9-7f06-743b-8a52-da0db22c046c@gmail.com> References: <808012c9-7f06-743b-8a52-da0db22c046c@gmail.com> Message-ID: On Fri, 20 Dec 2019 at 15:17, Matt Riedemann wrote: > > On 12/20/2019 4:04 AM, Thierry Carrez wrote: > > - directly approve stable releases after a cursory semver check, not > > waiting for stable-maint-core approval. > > Rather than go this drastic, how about just not waiting for > stable-maint-core (tonyb or myself) to review a stable branch release > proposal but if you're going to decentralize stable core to be > per-project rather than stable-maint-core, then do like the non-stable > release request PTL/liaison thing and ack once one of the per-project > stable maint cores signs off on the release or is the person requesting > the stable branch release? > > IOW, don't go from super high standard stable policy release review > check to no check, but push the burden to per-project stable cores since > they want the responsibility and this is part of the deal. Are there many cases where the PTL/release liaison (who must approve a release) isn't also in the per-project stable team? > > -- > > Thanks, > > Matt > From mriedemos at gmail.com Fri Dec 20 21:25:53 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 20 Dec 2019 15:25:53 -0600 Subject: State of the Gate (placement?) In-Reply-To: <727395bb-a1ab-5a62-0ca2-e0f87d867737@gmail.com> References: <20191031211535.vk7rtiq3pvsb6j2t@skaplons-mac> <8a6f54cb-76a7-4522-8a16-93822c4cdcb5@www.fastmail.com> <727395bb-a1ab-5a62-0ca2-e0f87d867737@gmail.com> Message-ID: <20b22b84-cb56-385c-0c6b-101305d23627@gmail.com> On 12/20/2019 11:47 AM, Matt Riedemann wrote: > Dredging this back up after discussing with Clark in IRC today. I've > posted a patch to disable c-bak and etcd3 in grenade jobs [1]. The hope > is this will alleviate some pressure on OVH nodes where iops are > restricted. We aren't even effectively doing anything with c-bak and > etcd3 in grenade jobs so this should not be controversial. > > [1] https://review.opendev.org/#/c/700214/ Actually doing it in the grenade repo doesn't work because of how devstack-gate generates the enabled services list for legacy jobs so I've got a series in devstack-gate that delivers the goods. https://review.opendev.org/#/c/700233/ -- Thanks, Matt From mriedemos at gmail.com Fri Dec 20 21:27:43 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Fri, 20 Dec 2019 15:27:43 -0600 Subject: [release] making releases fast again In-Reply-To: References: <808012c9-7f06-743b-8a52-da0db22c046c@gmail.com> Message-ID: <46a68702-4dc9-232d-5ed0-471c36f49210@gmail.com> On 12/20/2019 2:53 PM, Mark Goddard wrote: > Are there many cases where the PTL/release liaison (who must approve a > release) isn't also in the per-project stable team? Many? I don't know about many, but yes there are or have been quite a few projects where the PTL has never done a stable branch review before so they aren't on the stable branch core team. -- Thanks, Matt From amotoki at gmail.com Sat Dec 21 06:00:09 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Sat, 21 Dec 2019 15:00:09 +0900 Subject: [horizon] team meeting cancelled in the holiday season Message-ID: Hi, Horizon team cancel the team meetings at Dec 25, 2019 and Jan 1, 2020. See you all in new year and enjoy holidays! Thanks, Akihiro Motoki (irc: amotoki) From flux.adam at gmail.com Sat Dec 21 08:44:55 2019 From: flux.adam at gmail.com (Adam Harwell) Date: Sat, 21 Dec 2019 00:44:55 -0800 Subject: [octavia] Holiday Meeting Cancellations! Message-ID: Hey all! Octavia meetings will be cancelled for the next two weeks (Dec 25 and Jan 1), and will resume on Jan 8. See y'all then, and have a great holiday season! --Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From anlin.kong at gmail.com Sat Dec 21 10:29:02 2019 From: anlin.kong at gmail.com (Lingxian Kong) Date: Sat, 21 Dec 2019 23:29:02 +1300 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> Message-ID: On Fri, Dec 20, 2019 at 11:50 PM Thierry Carrez wrote: > > Happy to help with the infrastructure side of things. > Hi ttx, could you please help to create a project in opendev using the existing Gnocchi repo, or guide me how to do that? - Best regards, Lingxian Kong Catalyst Cloud -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry at openstack.org Sat Dec 21 14:05:23 2019 From: thierry at openstack.org (Thierry Carrez) Date: Sat, 21 Dec 2019 15:05:23 +0100 Subject: [Telemetry][TC]Telemetry status In-Reply-To: References: <35341fa4-3db9-0807-1136-9f6dff41b2c2@openstack.org> <81f6092d-08c1-aca7-e293-0e2a0c9c8559@binero.se> <224956f9-dac3-0faf-cb3c-8c7c57646071@binero.se> <6a689352-49e6-828a-58a4-ff3b9129c3c6@binero.se> <28AF37B6-73C6-4705-B2C4-0F69B779F591@gmail.com> Message-ID: <2dd773c5-aad3-a04f-6ef2-ce95fffc39bf@openstack.org> Lingxian Kong wrote: > On Fri, Dec 20, 2019 at 11:50 PM Thierry Carrez > wrote: > > > Happy to help with the infrastructure side of things. > > Hi ttx, could you please help to create a project in opendev using the > existing Gnocchi repo, or guide me how to do that? Hi Lingxian, I'll be on leave until January 6 and mostly offline, so unfortunately I can't really push that migration until I'm back. If you want to try by yourself, the steps to follow are documented at: https://docs.openstack.org/infra/manual/creators.html One early decision to make is whether the project is adopted by an openstack project team (in which case it can be back in the "openstack/" namespace) or if we'd maintain it as a separate project. You can also ask for help in #openstack-infra. -- Thierry Carrez (ttx) From mthode at mthode.org Sat Dec 21 21:24:50 2019 From: mthode at mthode.org (Matthew Thode) Date: Sat, 21 Dec 2019 15:24:50 -0600 Subject: [horizon][requirements] django-3 blocked by usage of django-babel Message-ID: <20191221212450.64vy6j5sbfyqpql5@mthode.org> it looks like django-babel is unmaintained since 2017, so perhaps it's time to look for a replacement. Looking through github issues, it looks like enmerkar is a possible replacement. If that's fine with horizon / django consumers we can start by adding enmerkar to global-requirements and upper-constraints. Thanks, -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From mthode at mthode.org Sun Dec 22 17:53:08 2019 From: mthode at mthode.org (Matthew Thode) Date: Sun, 22 Dec 2019 11:53:08 -0600 Subject: [cliff][docs][requirements] new cliff versions causes docs to fail to build Message-ID: <20191222175308.juzyu6grndfcf2ez@mthode.org> Looks like some things changed in the new version that we depended upon and are now causing failures. Exception occurred: File "/home/zuul/src/opendev.org/openstack/python-openstackclient/.tox/docs/lib/python3.6/site-packages/cliff/sphinxext.py", line 245, in _load_app if not issubclass(cliff_app_class, app.App): TypeError: issubclass() arg 1 must be a class -- Matthew Thode -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From amotoki at gmail.com Mon Dec 23 07:09:01 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Mon, 23 Dec 2019 16:09:01 +0900 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: <20191221212450.64vy6j5sbfyqpql5@mthode.org> References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: I am aware of the issue. While switching to enmerkar looks like the straightforward way, I am trying to explore a way to continue django-babel to minimize the impact to consumers and packagers. When we switch enmerkar, we need to clean up babel-django.cfg in horizon [1] and all horizon plugin repos, plus the infra manual. babel-django.cfg specifies an extractor from django-babel explicitly. We need to drop the extract entry first (as the "django" extractor is registered via an entrypoint in the recent django-babel and enmerkar). BTW, how did you notice this? [1] https://opendev.org/openstack/horizon/src/branch/master/babel-django.cfg On Sun, Dec 22, 2019 at 6:28 AM Matthew Thode wrote: > > it looks like django-babel is unmaintained since 2017, so perhaps it's > time to look for a replacement. Looking through github issues, it looks > like enmerkar is a possible replacement. If that's fine with horizon / > django consumers we can start by adding enmerkar to global-requirements > and upper-constraints. > > Thanks, > > -- > Matthew Thode From amotoki at gmail.com Mon Dec 23 08:59:11 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Mon, 23 Dec 2019 17:59:11 +0900 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: The subject of this thread looks confusing to me. "django-3 blocked by usage of django-babel" Horizon team has not adopted Django 3. An issue I noticed is that django-babel is not compatible with Django 2.2 and translation message extraction in the master branch is now broken. If some test tries to install Django 3, it would be due to some misconfiguration. Thanks, Akihiro On Mon, Dec 23, 2019 at 4:09 PM Akihiro Motoki wrote: > > I am aware of the issue. > > While switching to enmerkar looks like the straightforward way, > I am trying to explore a way to continue django-babel to minimize the > impact to consumers and packagers. > > When we switch enmerkar, we need to clean up babel-django.cfg in horizon [1] > and all horizon plugin repos, plus the infra manual. babel-django.cfg specifies > an extractor from django-babel explicitly. We need to drop the extract > entry first > (as the "django" extractor is registered via an entrypoint in the > recent django-babel > and enmerkar). > > BTW, how did you notice this? > > [1] https://opendev.org/openstack/horizon/src/branch/master/babel-django.cfg > > > > On Sun, Dec 22, 2019 at 6:28 AM Matthew Thode wrote: > > > > it looks like django-babel is unmaintained since 2017, so perhaps it's > > time to look for a replacement. Looking through github issues, it looks > > like enmerkar is a possible replacement. If that's fine with horizon / > > django consumers we can start by adding enmerkar to global-requirements > > and upper-constraints. > > > > Thanks, > > > > -- > > Matthew Thode From mdulko at redhat.com Mon Dec 23 09:40:17 2019 From: mdulko at redhat.com (=?UTF-8?Q?Micha=C5=82?= Dulko) Date: Mon, 23 Dec 2019 10:40:17 +0100 Subject: [kuryr] Multi-net spec 1.1 released Message-ID: Hi, I just wanted to give a heads up that version 1.1 of Multi-Network CRD Specification was just released. It might be worth to implement support for it in Kuryr at some point if there's something we could benefit from. Thanks, Michał From rfolco at redhat.com Mon Dec 23 12:11:15 2019 From: rfolco at redhat.com (Rafael Folco) Date: Mon, 23 Dec 2019 10:11:15 -0200 Subject: [tripleo] TripleO CI Summary: Sprint 40 Message-ID: Greetings, The TripleO CI team has just completed Sprint 40 / Unified Sprint 19 (Nov 21 thru Dec 18). The following is a summary of completed work during this sprint cycle: - Created a PoC for 3rd-party testing jobs for podman and ceph-ansible. The 3rd-party jobs are now running and being triggered from github pull requests. - Closed-out work on the design enhancements to the promotion server. Tested the new promoter code and fixed issues related to manifests and quay.io. - Working w/ quay support re: performance and api issues. - Implemented component pipeline [3] running daily and promoting separated from the integration jobs. Job that promotes the component still under testing and fixing bugs. Component jobs have been also created in downstream. - Addressed issues and technical debt tasks in TripleO CI realm, including issues in the zuul reproducer. Next sprint starts Jan 3rd. The planned work for the next sprint [1] are still going to be defined, and may include component pipeline work and build out CentOS8 promotion and check jobs upstream. The Ruck and Rover for this sprint are TBD. Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes to be tracked in etherpad [2]. Thanks, rfolco [1] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-20 [2] https://etherpad.openstack.org/p/ruckroversprint20 [3] http://dashboard-ci.tripleo.org/d/UDA4H3aZk/component-pipeline?orgId=1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From madongma at cn.ibm.com Mon Dec 23 12:35:47 2019 From: madongma at cn.ibm.com (Dong JV Ma) Date: Mon, 23 Dec 2019 20:35:47 +0800 Subject: IBM z/VM CI is planning to migrate new environment Message-ID: Hello, Due to the hardware issues, the IBM z/VM CI system [1] is planning to migrate to the new hardware in the next few months, the planning time will be in 3 months. We will do following changes with IBM z/VM CI environment: 1. Migrate the IBM z/VM CI to new hardware 2. Upgrade IBM z/VM CI from python2 to python3 During this time, IBM z/VM CI will not stable, please ignore the IBM z/VM CI failures. For any questions feel free to reach out to me(Dong Ma). [1] https://wiki.openstack.org/wiki/ThirdPartySystems/IBM_z/VM_CI Thanks & Best Regards! Dong Ma (马冬) LinuxONE IaaS Infra & CI/CD, China Systems Lab E-mail: madongma at cn.ibm.com IRC: larainema Address: 2/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC -------------- next part -------------- An HTML attachment was scrubbed... URL: From mriedemos at gmail.com Mon Dec 23 14:21:49 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Mon, 23 Dec 2019 08:21:49 -0600 Subject: IBM z/VM CI is planning to migrate new environment In-Reply-To: References: Message-ID: On 12/23/2019 6:35 AM, Dong JV Ma wrote: > Due to the hardware issues, the IBM z/VM CI system [1] is planning to > migrate to the new hardware in the next few months, the planning time > will be in 3 months. > > We will do following changes with IBM z/VM CI environment: > 1. Migrate the IBM z/VM CI to new hardware > 2. Upgrade IBM z/VM CI from python2 to python3 > > During this time, IBM z/VM CI will not stable, please ignore the IBM > z/VM CI failures. For any questions feel free to reach out to me(Dong Ma). > > [1] _https://wiki.openstack.org/wiki/ThirdPartySystems/IBM_z/VM_CI_ Thanks for the heads up. A separate question. Are there any plans to continue doing feature development on the virt driver within nova? The last time an IBM team contributed any features to it was in Rocky [1]. Since then it's basically been in maintenance mode when the driver interface changes. It's not like it's really a drain on resources for the nova team but I'm also assuming no one uses the in-tree driver so its existence is solely to say there is a zVM driver in nova for marketing/sales purposes. [1] https://review.opendev.org/#/c/543344/ -- Thanks, Matt From colleen at gazlene.net Mon Dec 23 17:32:39 2019 From: colleen at gazlene.net (Colleen Murphy) Date: Mon, 23 Dec 2019 09:32:39 -0800 Subject: [ops] Federated Identity Management survey Message-ID: <4a7a0c41-59ce-4aac-839e-0840eeb50348@www.fastmail.com> Hello operators, A researcher from the University of Kent who was influential in the design of keystone's federation implementation has asked the keystone team to gauge adoption of federated identity management in OpenStack deployments. This is something we've neglected to track well in the last few OpenStack user surveys, so I'd like to ask OpenStack operators to please take a few minutes to complete the following survey about your usage of identity federation in your OpenStack deployment (even if you don't use federation): https://uok.typeform.com/to/KuRY0q The results of this survey will benefit not only university research but also the keystone team as it will help us understand where to focus our efforts. Your participation is greatly appreciated. Thanks for your time, Colleen (cmurphy) From mdulko at redhat.com Mon Dec 23 18:44:39 2019 From: mdulko at redhat.com (=?UTF-8?Q?Micha=C5=82?= Dulko) Date: Mon, 23 Dec 2019 19:44:39 +0100 Subject: [kuryr] PTL on vacations Message-ID: <3e79134edd4db3a6fc765e902e6e8ed3c0fadc2a.camel@redhat.com> Hi, I'm starting my vacations now and will be back on January 13th. Please direct any issues to the rest of the core team. Thanks, Michał From dpeacock at redhat.com Mon Dec 23 19:11:57 2019 From: dpeacock at redhat.com (David Peacock) Date: Mon, 23 Dec 2019 14:11:57 -0500 Subject: [heat] Addressing the large patch backlog. In-Reply-To: <2940d9b7-d979-539d-79af-b54e2ee83a9f@redhat.com> References: <2940d9b7-d979-539d-79af-b54e2ee83a9f@redhat.com> Message-ID: Thanks so much for working this Zane, it's much appreciated! :-) On Tue, Dec 3, 2019 at 2:51 PM Zane Bitter wrote: > I added a column for "Core Reviewer comments" as well. > I hijacked this column for my amendments too. > > *_Rebase + Merge_* > > Would you have time to start going through these and rebase or recheck > them (as required), then provide a list somewhere of priority reviews > (i.e. those that rebased easily and passed tests) for cores to go > through? It should be pretty easy to merge a good number of these quite > quickly. > I went through a big blitz of all those rebase + merge patches today; as expected many of them were trivial rebases but a lot were very broken due to their age. I fixed up each and every single one, and even iterated on core review suggestions on a some. They're all in good shape for review / pushing through the merge now. When cores have time it would be great to get those reviewed and hopefully workflowed. > > > *_Research_* > > > > 76 patches are sufficiently complex that they'll need a much closer > > look. Some of these patches are in a seemingly "finished" state, some > > are a way off. Some have unanswered concerns from core review and have > > been left dangling. *If you're the original developer or otherwise > > interested in working these patches through to completion, please do get > > involved.* > > As a first step it'd be great if we could extract from this a list of > stuff that looks promising but hasn't been reviewed by a core. Those > would be the next highest priority to review I'd expect. > Absolutely - that's a great idea! I'll handle this. This is a very rewarding journey, and as I work the last few days before I go on my winter solstice break, it's fantastic for reflection too. Looking forward to continuing this endeavour into the new year. For those who celebrate, enjoy your holidays. For everyone else, keep being excellent! :-) Cheers, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From bence.romsics at gmail.com Mon Dec 23 19:35:35 2019 From: bence.romsics at gmail.com (Bence Romsics) Date: Mon, 23 Dec 2019 20:35:35 +0100 Subject: [neutron] bug deputy report of week 2019-12-16 Message-ID: Hi Neutron Team, Here comes the deputy report of last week: High: * https://bugs.launchpad.net/neutron/+bug/1856600 Unit test jobs are failing with ImportError: cannot import name 'engine' from 'flake8' Gate unblocking fix by Brian merged already. Follow up proposed by Nate: https://review.opendev.org/699243 Medium: * https://bugs.launchpad.net/neutron/+bug/1856523 Sometimes instance can't get public keys due to cirros metadata request failur Occasional loss of metadata response. Follow up bug report for CirrOS by Maciej: https://github.com/cirros-dev/cirros/issues/8 and pull request on github: https://github.com/cirros-dev/cirros/pull/11 * https://bugs.launchpad.net/neutron/+bug/1856675 / https://bugs.launchpad.net/neutron/+bug/1856726 IPv6 Prefix Delegation do not work / IpamBackendMixin._gateway_ip_str is not reading correctly the subnet IP version Fix proposed by Rodolfo: https://review.opendev.org/699465 * https://bugs.launchpad.net/neutron/+bug/1856853 OVSInterfaceDriver.plug_new should catch pyroute2.netlink.exceptions.NetlinkErro Fix proposed by Rodoflo: https://review.opendev.org/699700 * https://bugs.launchpad.net/neutron/+bug/1857021 Wrong order of adding/deleting router ports may end up with not configured router port Fix proposed by Slawek: https://review.opendev.org/700011 Low: * https://bugs.launchpad.net/neutron/+bug/1857035 Add better description for "test_cleanup_network_namespaces_cleans_dhcp_and_l3_namespaces" in case of failure Low hanging fruit improvement to logging. Unassigned. * https://bugs.launchpad.net/neutron/+bug/1857047 openstack port set --no-fixed-ips removed only 1 fixed IP Unassigned. Undecided: * https://bugs.launchpad.net/neutron/+bug/1857016 Possible double row.delete() call in ACL code neutron fix by Terry in the gate: https://review.opendev.org/700007 the same change in neutron-ovn: https://review.opendev.org/700001 Maybe RFE: * https://bugs.launchpad.net/neutron/+bug/1856839 [L3] router processing time increase if there are large large set ports Possbile performance problem reported by Yulong (who is also the assignee). No gerrit change yet. Merry Christmas, Bence From christophe.sauthier at objectif-libre.com Mon Dec 23 21:58:39 2019 From: christophe.sauthier at objectif-libre.com (Christophe Sauthier) Date: Mon, 23 Dec 2019 16:58:39 -0500 Subject: [cloudkitty] Stepping down from PTL In-Reply-To: <6a879c96c9aa82cc31f4ffde7a6b2663@objectif-libre.com> References: <6a879c96c9aa82cc31f4ffde7a6b2663@objectif-libre.com> Message-ID: <2a41826f338d324503f67ae1c08849c5@objectif-libre.com> Thanks a lot Luka for your hard and great work on Cloudkitty. Welcome Justin, I'll be really to help you as much as I can. Cheers Christophe ---- Christophe Sauthier Directeur Général Objectif Libre : Au service de votre Cloud +33 (0) 6 16 98 63 96 | christophe.sauthier at objectif-libre.com https://www.objectif-libre.com | @objectiflibre Recevez la Pause Cloud Et DevOps : https://olib.re/abo-pause Le 2019-12-20 10:13, Luka Peschke a écrit : > Hi all, > > I'm moving to a new position that doesn't involve OpenStack, and > won't leave me the required time to be Cloudkitty's PTL. This is why I > have to step down from the PTL position. jferrieu will take my > position for the end of the U cycle (he's been a major contributor > recently), with the help of huats, who's been the Cloudkitty PTL > before me, and has been around in the community for a long time. > > I've been the PTL for two and a half cycles, and I think that it is a > good thing for the project to take a new lead, with a new vision. > > I'm grateful for my experience within the OpenStack community. > > Thanks, From sundar.nadathur at intel.com Mon Dec 23 23:26:38 2019 From: sundar.nadathur at intel.com (Nadathur, Sundar) Date: Mon, 23 Dec 2019 23:26:38 +0000 Subject: [cyborg] No weekly IRC meeting for next 2 weeks Message-ID: There will be no weekly Cyborg IRC meeting on Dec 26 or Jan 2. Happy Holidays to all. Regards, Sundar -------------- next part -------------- An HTML attachment was scrubbed... URL: From hiroyuki.jo.mt at hco.ntt.co.jp Tue Dec 24 07:51:12 2019 From: hiroyuki.jo.mt at hco.ntt.co.jp (Hiroyuki JO) Date: Tue, 24 Dec 2019 16:51:12 +0900 Subject: [devstack][ceilometer][tacker]ceilometer devstack installation fails Message-ID: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> Hi ceilometer team and all, After a patch[1] was merged, Tacker zuul check fails because ceilometer devstack installation error[2]. Do you have any work aroud of this problem? Addition, why devstack uses pip version 9.0.3 when installing python packages? The cause of our problem is installation error of conluent-kafka. But with latest pip (version 19), I was able to install the package. [1] https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f98f564d54783982 [2] https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a/log/controller/logs/devstacklog.txt.gz Regards, 徐 広幸 (Hiroyuki JO) Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 NTT Network Systems Laboratories From aaronzhu1121 at gmail.com Tue Dec 24 08:12:44 2019 From: aaronzhu1121 at gmail.com (Rong Zhu) Date: Tue, 24 Dec 2019 16:12:44 +0800 Subject: [devstack][ceilometer][tacker]ceilometer devstack installation fails In-Reply-To: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> References: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> Message-ID: Hi Hiroyuki JO, I will check this problem from ceilometer side. Hiroyuki JO 于2019年12月24日 周二15:55写道： > Hi ceilometer team and all, > After a patch[1] was merged, Tacker zuul check fails because ceilometer > devstack installation error[2]. Do you have any work aroud of this > problem? > > Addition, why devstack uses pip version 9.0.3 when installing python > packages? The cause of our problem is installation error of > conluent-kafka. But with latest pip (version 19), I was able to install > the package. > > [1] > > https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f98f564d54783982 > [2] > > https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a/log/controller/logs/devstacklog.txt.gz > > Regards, > 徐 広幸 (Hiroyuki JO) > Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 > NTT Network Systems Laboratories > > > -- Thanks, Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From hiroyuki.jo.mt at hco.ntt.co.jp Wed Dec 25 06:54:02 2019 From: hiroyuki.jo.mt at hco.ntt.co.jp (Hiroyuki JO) Date: Wed, 25 Dec 2019 15:54:02 +0900 Subject: [devstack][ceilometer][tacker]ceilometer devstack installation fails In-Reply-To: References: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> Message-ID: <6a6fc62d274798e0c7fe14e86021553e@hco.ntt.co.jp_1> Thanks Rong. RicoLin also seems to be addressing the problem and submitted a patch[1]. The patch looks work well. [1] https://review.opendev.org/#/c/700490/ 2019-12-24 17:12 に Rong Zhu さんは書きました: > Hi Hiroyuki JO, > > I will check this problem from ceilometer side. > > Hiroyuki JO 于2019年12月24日 周二15:55写道： > >> Hi ceilometer team and all, >> After a patch[1] was merged, Tacker zuul check fails because >> ceilometer >> devstack installation error[2]. Do you have any work aroud of this >> problem? >> >> Addition, why devstack uses pip version 9.0.3 when installing python >> packages? The cause of our problem is installation error of >> conluent-kafka. But with latest pip (version 19), I was able to >> install >> the package. >> >> [1] >> >> https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f98f564d54783982 >> [2] >> >> https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a/log/controller/logs/devstacklog.txt.gz >> >> Regards, >> 徐 広幸 (Hiroyuki JO) >> Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 >> NTT Network Systems Laboratories >> >> >> -- > Thanks, > Rong Zhu -- 徐 広幸 (Hiroyuki JO) Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 NTT Network Systems Laboratories From aaronzhu1121 at gmail.com Wed Dec 25 07:09:31 2019 From: aaronzhu1121 at gmail.com (Rong Zhu) Date: Wed, 25 Dec 2019 15:09:31 +0800 Subject: [devstack][ceilometer][tacker]ceilometer devstack installation fails In-Reply-To: <6a6fc62d274798e0c7fe14e86021553e@hco.ntt.co.jp_1> References: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> <6a6fc62d274798e0c7fe14e86021553e@hco.ntt.co.jp_1> Message-ID: I think this should fix in ceilometer not in devstack, I am now add the dependence in ceilometer bindep.txt. I will check the result, will merge it when it works. Hiroyuki JO 于2019年12月25日 周三14:54写道： > Thanks Rong. RicoLin also seems to be addressing the problem and > submitted a patch[1]. The patch looks work well. > > [1] https://review.opendev.org/#/c/700490/ > > 2019-12-24 17:12 に Rong Zhu さんは書きました: > > Hi Hiroyuki JO, > > > > I will check this problem from ceilometer side. > > > > Hiroyuki JO 于2019年12月24日 周二15:55写道： > > > >> Hi ceilometer team and all, > >> After a patch[1] was merged, Tacker zuul check fails because > >> ceilometer > >> devstack installation error[2]. Do you have any work aroud of this > >> problem? > >> > >> Addition, why devstack uses pip version 9.0.3 when installing python > >> packages? The cause of our problem is installation error of > >> conluent-kafka. But with latest pip (version 19), I was able to > >> install > >> the package. > >> > >> [1] > >> > >> > https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f98f564d54783982 > >> [2] > >> > >> > https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a/log/controller/logs/devstacklog.txt.gz > >> > >> Regards, > >> 徐 広幸 (Hiroyuki JO) > >> Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 > >> NTT Network Systems Laboratories > >> > >> > >> -- > > Thanks, > > Rong Zhu > > -- > 徐 広幸 (Hiroyuki JO) > Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 > NTT Network Systems Laboratories > > -- Thanks, Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaronzhu1121 at gmail.com Wed Dec 25 08:01:21 2019 From: aaronzhu1121 at gmail.com (Rong Zhu) Date: Wed, 25 Dec 2019 16:01:21 +0800 Subject: [devstack][ceilometer][tacker]ceilometer devstack installation fails In-Reply-To: References: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> <6a6fc62d274798e0c7fe14e86021553e@hco.ntt.co.jp_1> Message-ID: Fix this should add extra repository, bindep.txt can't install this package. I think Rico's fix is the right way. Rong Zhu 于2019年12月25日 周三15:09写道： > I think this should fix in ceilometer not in devstack, I am now add the > dependence in ceilometer bindep.txt. I will check the result, will merge it > when it works. > > Hiroyuki JO 于2019年12月25日 周三14:54写道： > >> Thanks Rong. RicoLin also seems to be addressing the problem and >> submitted a patch[1]. The patch looks work well. >> >> [1] https://review.opendev.org/#/c/700490/ >> >> 2019-12-24 17:12 に Rong Zhu さんは書きました: >> > Hi Hiroyuki JO, >> > >> > I will check this problem from ceilometer side. >> > >> > Hiroyuki JO 于2019年12月24日 周二15:55写道： >> > >> >> Hi ceilometer team and all, >> >> After a patch[1] was merged, Tacker zuul check fails because >> >> ceilometer >> >> devstack installation error[2]. Do you have any work aroud of this >> >> problem? >> >> >> >> Addition, why devstack uses pip version 9.0.3 when installing python >> >> packages? The cause of our problem is installation error of >> >> conluent-kafka. But with latest pip (version 19), I was able to >> >> install >> >> the package. >> >> >> >> [1] >> >> >> >> >> https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f98f564d54783982 >> >> [2] >> >> >> >> >> https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a/log/controller/logs/devstacklog.txt.gz >> >> >> >> Regards, >> >> 徐 広幸 (Hiroyuki JO) >> >> Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 >> >> NTT Network Systems Laboratories >> >> >> >> >> >> -- >> > Thanks, >> > Rong Zhu >> >> -- >> 徐 広幸 (Hiroyuki JO) >> Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 >> NTT Network Systems Laboratories >> >> -- > Thanks, > Rong Zhu > -- Thanks, Rong Zhu -------------- next part -------------- An HTML attachment was scrubbed... URL: From eyalb1 at gmail.com Wed Dec 25 11:56:29 2019 From: eyalb1 at gmail.com (Eyal B) Date: Wed, 25 Dec 2019 13:56:29 +0200 Subject: [devstack][ceilometer][tacker]ceilometer devstack installation fails In-Reply-To: References: <98cace3dcf945cdc72fb90c5f882e898@hco.ntt.co.jp_1> <6a6fc62d274798e0c7fe14e86021553e@hco.ntt.co.jp_1> Message-ID: Hi, There is an open bug for confluent-kafka 1.3.0 https://github.com/confluentinc/confluent-kafka-python/issues/751 I submitted a patch https://review.opendev.org/#/c/700512/ to limit to version 1.2.0 if you depend on this patch installation succeeds Eyal On Wed, 25 Dec 2019 at 10:04, Rong Zhu wrote: > Fix this should add extra repository, bindep.txt can't install this > package. I think Rico's fix is the right way. > > Rong Zhu 于2019年12月25日 周三15:09写道： > >> I think this should fix in ceilometer not in devstack, I am now add the >> dependence in ceilometer bindep.txt. I will check the result, will merge it >> when it works. >> >> Hiroyuki JO 于2019年12月25日 周三14:54写道： >> >>> Thanks Rong. RicoLin also seems to be addressing the problem and >>> submitted a patch[1]. The patch looks work well. >>> >>> [1] https://review.opendev.org/#/c/700490/ >>> >>> 2019-12-24 17:12 に Rong Zhu さんは書きました: >>> > Hi Hiroyuki JO, >>> > >>> > I will check this problem from ceilometer side. >>> > >>> > Hiroyuki JO 于2019年12月24日 周二15:55写道： >>> > >>> >> Hi ceilometer team and all, >>> >> After a patch[1] was merged, Tacker zuul check fails because >>> >> ceilometer >>> >> devstack installation error[2]. Do you have any work aroud of this >>> >> problem? >>> >> >>> >> Addition, why devstack uses pip version 9.0.3 when installing python >>> >> packages? The cause of our problem is installation error of >>> >> conluent-kafka. But with latest pip (version 19), I was able to >>> >> install >>> >> the package. >>> >> >>> >> [1] >>> >> >>> >> >>> https://opendev.org/openstack/requirements/commit/c99ab09bc44818b92fe86842f98f564d54783982 >>> >> [2] >>> >> >>> >> >>> https://zuul.opendev.org/t/openstack/build/e6d04b35d16e4447901aba441721115a/log/controller/logs/devstacklog.txt.gz >>> >> >>> >> Regards, >>> >> 徐 広幸 (Hiroyuki JO) >>> >> Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 >>> >> NTT Network Systems Laboratories >>> >> >>> >> >>> >> -- >>> > Thanks, >>> > Rong Zhu >>> >>> -- >>> 徐 広幸 (Hiroyuki JO) >>> Email: hiroyuki.jo.mt at hco.ntt.co.jp / Tel(direct): +81-422-59-7394 >>> NTT Network Systems Laboratories >>> >>> -- >> Thanks, >> Rong Zhu >> > -- > Thanks, > Rong Zhu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zigo at debian.org Wed Dec 25 20:53:11 2019 From: zigo at debian.org (Thomas Goirand) Date: Wed, 25 Dec 2019 21:53:11 +0100 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: On 12/23/19 9:59 AM, Akihiro Motoki wrote: > The subject of this thread looks confusing to me. > "django-3 blocked by usage of django-babel" > > Horizon team has not adopted Django 3. > An issue I noticed is that django-babel is not compatible with Django 2.2 and > translation message extraction in the master branch is now broken. > > If some test tries to install Django 3, it would be due to some > misconfiguration. > > Thanks, > Akihiro Well, it'd be nice to have plans to get Django 3 compat for the end of this cycle, otherwise, we'll get again stuck in downstream distributions, just like we've been stuck in Debian with a broken Horizon for nearly the whole of the last summer (due to Django 2.2 and the new view class instead of method thing breaking all ...). I guess we're all being super tired of Django breaking the world every 2 weeks. At least, I am. But I don't think we have any alternatives... (and no, I do not have time, neither probably the skills, to work on this, sorry...) Cheers, Thomas Goirand (zigo) From ahmed.zaky.abdallah at gmail.com Thu Dec 26 23:14:04 2019 From: ahmed.zaky.abdallah at gmail.com (ahmed.zaky.abdallah at gmail.com) Date: Fri, 27 Dec 2019 00:14:04 +0100 Subject: About the use of security groups with neutron ports Message-ID: <003d01d5bc42$2af8ceb0$80ea6c10$@gmail.com> Hi All, I am trying to wrap my head around something I came across in one of the OpenStack deployments. I am running Telco VNFs one of them is having different VMs using SR-IOV interfaces. On one of my VNFs on Openstack, I defined a wrong IPv6 Gm bearer interface to be exactly the same as the IPv6 Gateway. As I hate re-onboarding, I decided to embark on a journey of changing the IPv6 of the Gm bearer interface manually on the application side, everything went on fine. After two weeks, my customer started complaining about one way RTP flow. The customer was reluctant to blame the operation I carried out because everything worked smooth after my modification. After days of investigation, I remembered that I have port-security enabled and this means AAP “Allowed-Address-Pairs” are defined per vPort (AAP contain the floating IP address of the VM so that the security to allow traffic to and from this VIP). I gave it a try and edited AAP “Allowed-Address-Pairs” to include the correct new IPv6 address. Doing that everything started working fine. The only logical explanation at that time is security group rules are really invoked. Now, I am trying to understand how the iptables are really invoked. I did some digging and it seems like we can control the firewall drivers on two levels: * Nova compute * ML2 plugin I was curious to check nova.conf and it has already the following line: firewall_driver=nova.virt.firewall.NoopFirewallDriver However, checking the ml2 plugin configuration, the following is found: 230 [securitygroup] 231 232 # 233 # From neutron.ml2 234 # 235 236 # Driver for security groups firewall in the L2 agent (string value) 237 #firewall_driver = 238 firewall_driver = openvswitch So, I am jumping to a conclusion that ml2 plugin is the one responsible for enforcing the firewall rules in my case. Have you had a similar experience? Is my assumption correct: If I comment out the ml2 plugin firewall driver then the port security carries no sense at all and security groups won’t be invoked? Cheers, Ahmed -------------- next part -------------- An HTML attachment was scrubbed... URL: From skaplons at redhat.com Fri Dec 27 09:28:11 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Fri, 27 Dec 2019 10:28:11 +0100 Subject: About the use of security groups with neutron ports In-Reply-To: <003d01d5bc42$2af8ceb0$80ea6c10$@gmail.com> References: <003d01d5bc42$2af8ceb0$80ea6c10$@gmail.com> Message-ID: <582E6225-F178-401A-A1D4-A52484B76DD9@redhat.com> Hi, > On 27 Dec 2019, at 00:14, ahmed.zaky.abdallah at gmail.com wrote: > > Hi All, > > I am trying to wrap my head around something I came across in one of the OpenStack deployments. I am running Telco VNFs one of them is having different VMs using SR-IOV interfaces. > > On one of my VNFs on Openstack, I defined a wrong IPv6 Gm bearer interface to be exactly the same as the IPv6 Gateway. As I hate re-onboarding, I decided to embark on a journey of changing the IPv6 of the Gm bearer interface manually on the application side, everything went on fine. > > After two weeks, my customer started complaining about one way RTP flow. The customer was reluctant to blame the operation I carried out because everything worked smooth after my modification. > After days of investigation, I remembered that I have port-security enabled and this means AAP “Allowed-Address-Pairs” are defined per vPort (AAP contain the floating IP address of the VM so that the security to allow traffic to and from this VIP). I gave it a try and edited AAP “Allowed-Address-Pairs” to include the correct new IPv6 address. Doing that everything started working fine. > > The only logical explanation at that time is security group rules are really invoked. > > Now, I am trying to understand how the iptables are really invoked. I did some digging and it seems like we can control the firewall drivers on two levels: > > • Nova compute > • ML2 plugin > > I was curious to check nova.conf and it has already the following line: firewall_driver=nova.virt.firewall.NoopFirewallDriver > > However, checking the ml2 plugin configuration, the following is found: > > 230 [securitygroup] > 231 > 232 # > 233 # From neutron.ml2 > 234 # > 235 > 236 # Driver for security groups firewall in the L2 agent (string value) > 237 #firewall_driver = > 238 firewall_driver = openvswitch > > So, I am jumping to a conclusion that ml2 plugin is the one responsible for enforcing the firewall rules in my case. > > Have you had a similar experience? > Is my assumption correct: If I comment out the ml2 plugin firewall driver then the port security carries no sense at all and security groups won’t be invoked? Firewall_driver config option has to be set to some value. You can set “noop” as firewall_driver to completely disable this feature for all ports. But please remember that You need to set it on agent’s side so it’s on compute nodes, not on neutron-server side. Also, if You want to disable it only for some ports, You can set “port_security_enabled” to False and than SG will not be applied for such port and You will not need to configure any additional IPs in allowed address pairs for this port. > > Cheers, > Ahmed — Slawek Kaplonski Senior software engineer Red Hat From ahmed.zaky.abdallah at gmail.com Fri Dec 27 14:38:03 2019 From: ahmed.zaky.abdallah at gmail.com (ahmed.zaky.abdallah at gmail.com) Date: Fri, 27 Dec 2019 15:38:03 +0100 Subject: About the use of security groups with neutron ports In-Reply-To: <582E6225-F178-401A-A1D4-A52484B76DD9@redhat.com> References: <003d01d5bc42$2af8ceb0$80ea6c10$@gmail.com> <582E6225-F178-401A-A1D4-A52484B76DD9@redhat.com> Message-ID: <000401d5bcc3$3f9ebd30$bedc3790$@gmail.com> Thank you very much, Slawek. In case I have multiple configuration files, how to know which one is currently loaded in neutron? For example, in my environment I have: * ml2_conf.ini * ml2_conf_odl.ini * ml2_conf_sriov.ini * openvswitch_agent.ini * sriov_agent.ini [root at overcloud-controller-0 cbis-admin]# cd /etc/neutron/plugins/ml2/ [root at overcloud-controller-0 ml2]# ls ml2_conf.ini ml2_conf_odl.ini ml2_conf_sriov.ini openvswitch_agent.ini sriov_agent.ini Which one of these is used? Cheers, Ahmed -----Original Message----- From: Slawek Kaplonski Sent: Friday, December 27, 2019 10:28 AM To: ahmed.zaky.abdallah at gmail.com Cc: openstack-discuss at lists.openstack.org Subject: Re: About the use of security groups with neutron ports Hi, > On 27 Dec 2019, at 00:14, ahmed.zaky.abdallah at gmail.com wrote: > > Hi All, > > I am trying to wrap my head around something I came across in one of the OpenStack deployments. I am running Telco VNFs one of them is having different VMs using SR-IOV interfaces. > > On one of my VNFs on Openstack, I defined a wrong IPv6 Gm bearer interface to be exactly the same as the IPv6 Gateway. As I hate re-onboarding, I decided to embark on a journey of changing the IPv6 of the Gm bearer interface manually on the application side, everything went on fine. > > After two weeks, my customer started complaining about one way RTP flow. The customer was reluctant to blame the operation I carried out because everything worked smooth after my modification. > After days of investigation, I remembered that I have port-security enabled and this means AAP “Allowed-Address-Pairs” are defined per vPort (AAP contain the floating IP address of the VM so that the security to allow traffic to and from this VIP). I gave it a try and edited AAP “Allowed-Address-Pairs” to include the correct new IPv6 address. Doing that everything started working fine. > > The only logical explanation at that time is security group rules are really invoked. > > Now, I am trying to understand how the iptables are really invoked. I did some digging and it seems like we can control the firewall drivers on two levels: > > • Nova compute > • ML2 plugin > > I was curious to check nova.conf and it has already the following line: firewall_driver=nova.virt.firewall.NoopFirewallDriver > > However, checking the ml2 plugin configuration, the following is found: > > 230 [securitygroup] > 231 > 232 # > 233 # From neutron.ml2 > 234 # > 235 > 236 # Driver for security groups firewall in the L2 agent (string value) > 237 #firewall_driver = > 238 firewall_driver = openvswitch > > So, I am jumping to a conclusion that ml2 plugin is the one responsible for enforcing the firewall rules in my case. > > Have you had a similar experience? > Is my assumption correct: If I comment out the ml2 plugin firewall driver then the port security carries no sense at all and security groups won’t be invoked? Firewall_driver config option has to be set to some value. You can set “noop” as firewall_driver to completely disable this feature for all ports. But please remember that You need to set it on agent’s side so it’s on compute nodes, not on neutron-server side. Also, if You want to disable it only for some ports, You can set “port_security_enabled” to False and than SG will not be applied for such port and You will not need to configure any additional IPs in allowed address pairs for this port. > > Cheers, > Ahmed — Slawek Kaplonski Senior software engineer Red Hat -------------- next part -------------- An HTML attachment was scrubbed... URL: From amotoki at gmail.com Sat Dec 28 14:59:35 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Sat, 28 Dec 2019 23:59:35 +0900 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: On Thu, Dec 26, 2019 at 5:57 AM Thomas Goirand wrote: > > On 12/23/19 9:59 AM, Akihiro Motoki wrote: > > The subject of this thread looks confusing to me. > > "django-3 blocked by usage of django-babel" > > > > Horizon team has not adopted Django 3. > > An issue I noticed is that django-babel is not compatible with Django 2.2 and > > translation message extraction in the master branch is now broken. > > > > If some test tries to install Django 3, it would be due to some > > misconfiguration. > > > > Thanks, > > Akihiro > > Well, it'd be nice to have plans to get Django 3 compat for the end of > this cycle, otherwise, we'll get again stuck in downstream > distributions, just like we've been stuck in Debian with a broken > Horizon for nearly the whole of the last summer (due to Django 2.2 and > the new view class instead of method thing breaking all ...). > > I guess we're all being super tired of Django breaking the world every 2 > weeks. At least, I am. But I don't think we have any alternatives... > (and no, I do not have time, neither probably the skills, to work on > this, sorry...) Djagno 3 support is not in the list of horizon priorities in Ussuri cycle as it is not an LTS release. It is documented in [1]. The top priorities are found in [2]. I don't think we can do it as a priority considering the resource of the active horizon developers. It completely depends on development resources and is a topic of investment. If someone is responsible to work on it to support both Django 2.2 and 3, I am happy to review it. Thanks, Akihiro [1] https://docs.openstack.org/horizon/latest/contributor/policy.html#django-support [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011442.html > > Cheers, > > Thomas Goirand (zigo) > From amotoki at gmail.com Sat Dec 28 20:24:39 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Sun, 29 Dec 2019 05:24:39 +0900 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: Update on django-babel/enmerkar. To switch django-babel to enmerkar, we need: (1) Cleanup babel configurations in horizon plugins, and (2) Switch django-babel to enmerkar in global-requirements To move (1) forward, I proposed patches to horizon plugins [1]. The babel extractors are registered via entry points so this change makes the maintenance simple regardless of that we switch to enmerkar. Once the cleanup of babel configurations in horizon plugins is done, the cost of switching the babel extractor for django would be low as we can switch the babel extractor for django only by changing it in the horizon repo (+ requirements repo), so I think we can switch enmerkar and if the situation changes we can switch back to django-babel or others. Regarding (2), I proposed patches both to the requirements and horizon repos [2][3]. The current blocking issue is that enmerkar depends on Django>=2.2 but horizon still supports Django 1.11. We plan to drop Django 1.11 support in Ussuri milestone-2, so we need to wait until Django 1.11 support is dropped. (python 2.7 support has been dropped in most horizon plugins so there is no blocking topic for django 1.11 drop.) (side note) django-babel is not a runtime requirement of horizon. It is just used when we extract translation strings and the current user is the infra translation jobs [4], so I think we don't need to care co-installability much. It can be moved to test-requirements.txt in horizon and we can install django-babel/enmerkar explicitly in common_translation_update.sh in [4]. [1] https://review.opendev.org/#/q/topic:babel-config+(status:open+OR+status:merged) [2] https://review.opendev.org/#/c/700727/ (requirements repo) [3] https://review.opendev.org/#/c/700728/ (horizon repo) [4] https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/roles/prepare-zanata-client/files Thanks, Akihiro Motoki On Mon, Dec 23, 2019 at 4:09 PM Akihiro Motoki wrote: > > I am aware of the issue. > > While switching to enmerkar looks like the straightforward way, > I am trying to explore a way to continue django-babel to minimize the > impact to consumers and packagers. > > When we switch enmerkar, we need to clean up babel-django.cfg in horizon [1] > and all horizon plugin repos, plus the infra manual. babel-django.cfg specifies > an extractor from django-babel explicitly. We need to drop the extract > entry first > (as the "django" extractor is registered via an entrypoint in the > recent django-babel > and enmerkar). > > BTW, how did you notice this? > > [1] https://opendev.org/openstack/horizon/src/branch/master/babel-django.cfg > > > > On Sun, Dec 22, 2019 at 6:28 AM Matthew Thode wrote: > > > > it looks like django-babel is unmaintained since 2017, so perhaps it's > > time to look for a replacement. Looking through github issues, it looks > > like enmerkar is a possible replacement. If that's fine with horizon / > > django consumers we can start by adding enmerkar to global-requirements > > and upper-constraints. > > > > Thanks, > > > > -- > > Matthew Thode From miguel at mlavalle.com Sat Dec 28 23:22:51 2019 From: miguel at mlavalle.com (Miguel Lavalle) Date: Sat, 28 Dec 2019 17:22:51 -0600 Subject: [neutron] Bug deputy report week of December 23rd Message-ID: Hi, I was the bug deputy for the week of December 23rd. As expected for an end of year Holidays week, We had very little activity: Requested more information: - https://bugs.launchpad.net/neutron/+bug/1857422 neutron-keepalived-state-change and keeplived cannot be cleanup for those routers which is deleted during l3-agent died -------------- next part -------------- An HTML attachment was scrubbed... URL: From rony.khan at brilliant.com.bd Sun Dec 29 10:14:38 2019 From: rony.khan at brilliant.com.bd (Md. Farhad Hasan Khan) Date: Sun, 29 Dec 2019 16:14:38 +0600 Subject: Openstack Rocky to Train upgrade Message-ID: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> Hi, Could you please help me how can I upgrade from Openstack Rocky version to Train. Thanks & B'Rgds, Rony -------------- next part -------------- An HTML attachment was scrubbed... URL: From radoslaw.piliszek at gmail.com Sun Dec 29 10:50:09 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Sun, 29 Dec 2019 11:50:09 +0100 Subject: Openstack Rocky to Train upgrade In-Reply-To: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> References: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> Message-ID: Hi Rony, It usually depends on OpenStack distribution you used to deploy Rocky. Please note the recommended approach is to first upgrade to the next release, which is Stein, and then to Train. -yoctozepto niedz., 29 gru 2019 o 11:23 Md. Farhad Hasan Khan napisał(a): > > Hi, > > Could you please help me how can I upgrade from Openstack Rocky version to Train. > > > > Thanks & B’Rgds, > > Rony > > From zigo at debian.org Sun Dec 29 16:57:58 2019 From: zigo at debian.org (Thomas Goirand) Date: Sun, 29 Dec 2019 17:57:58 +0100 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: <30f25e42-4d68-5d5f-69a0-0f4d39145b00@debian.org> On 12/28/19 3:59 PM, Akihiro Motoki wrote: > On Thu, Dec 26, 2019 at 5:57 AM Thomas Goirand wrote: >> >> On 12/23/19 9:59 AM, Akihiro Motoki wrote: >>> The subject of this thread looks confusing to me. >>> "django-3 blocked by usage of django-babel" >>> >>> Horizon team has not adopted Django 3. >>> An issue I noticed is that django-babel is not compatible with Django 2.2 and >>> translation message extraction in the master branch is now broken. >>> >>> If some test tries to install Django 3, it would be due to some >>> misconfiguration. >>> >>> Thanks, >>> Akihiro >> >> Well, it'd be nice to have plans to get Django 3 compat for the end of >> this cycle, otherwise, we'll get again stuck in downstream >> distributions, just like we've been stuck in Debian with a broken >> Horizon for nearly the whole of the last summer (due to Django 2.2 and >> the new view class instead of method thing breaking all ...). >> >> I guess we're all being super tired of Django breaking the world every 2 >> weeks. At least, I am. But I don't think we have any alternatives... >> (and no, I do not have time, neither probably the skills, to work on >> this, sorry...) > Hi Akihiro, > Djagno 3 support is not in the list of horizon priorities in Ussuri > cycle as it is not an LTS release. I often get this type of answer from the Horizon team. Unfortunately, this doesn't help with the situation. When Django 3 will reach Debian Sid, Horizon will likely be broken, regardless of Django 3 being LTS or not. > It is documented in [1]. The top priorities are found in [2]. > I don't think we can do it as a priority considering the resource of > the active horizon developers. > It completely depends on development resources and is a topic of investment. > If someone is responsible to work on it to support both Django 2.2 and > 3, I am happy to review it. Sure! I completely understand. Something like the Linux kernel never breaks userland, and it's been like this for decades. As I wrote, Django is breaking the world every 6 months. Not only these happen, but Django upstream only document it in the release note with "thing Z is removed in this release", when really, they should be saying "if you wrote X, then you should write Y instead to address Z being removed". Then we always got to search the internet for someone who already understood the breakage and fixed it, though when a new release of Django just happened, it's hard to find such example. All of this is annoying, frustrating, and IMO not professional at all from the Django upstream. I'm not sure how we can move things toward a better direction so the madness stops. On top of that, very rarely, there's enough resources in the Horizon team to address the breakage early enough. I often tried to fix a few things, but I lack enough time and skills (both are related, of course: if I can't find time to work on these, I can't learn and get skills). Anyway, thanks for your answer, Cheers, Thomas Goirand (zigo) From radoslaw.piliszek at gmail.com Sun Dec 29 20:41:45 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Sun, 29 Dec 2019 21:41:45 +0100 Subject: [oslo][kolla][requirements][release][infra] Hit by an old, fixed bug Message-ID: Hi Folks, as the subject goes, my installation has been hit by an old bug: https://bugs.launchpad.net/oslo.messaging/+bug/1828841 (bug details not important, linked here for background) I am using Stein, deployed with recent Kolla-built source-based images (with only slight modifications compared to vanilla ones). Kolla's procedure for building source-based images considers upper constraints, which, unfortunately, turned out to be lagging behind a few releases w.r.t. oslo.messaging at least. The fix was in 9.7.0 released on May 21, u-c still point to 9.5.0 from Feb 26 and the latest of Stein is 9.8.0 from Jul 18. It seems oslo.messaging is missing from the automatic updates that bot proposes: https://review.opendev.org/#/q/owner:%22OpenStack+Proposal+Bot%22+project:openstack/requirements+branch:stable/stein Per: https://opendev.org/openstack/releases/src/branch/master/doc/source/reference/reviewer_guide.rst#release-jobs this upper-constraint proposal should be happening for all releases. I would be glad if someone investigated why it happens(/ed) and audited whether other OpenStack projects don't need updating as well to avoid running on old deps when new are awaiting for months. :-) Please note this might apply to other branches as well. PS: for some reason oslo.messaging Stein release notes ( https://docs.openstack.org/releasenotes/oslo.messaging/stein.html ) are stuck at 9.5.0 as well, this could be right (I did not inspect the sources) but I am adding this in PS so you have more things to correlate if they need be. -yoctozepto From joe at topjian.net Mon Dec 30 04:57:22 2019 From: joe at topjian.net (Joe Topjian) Date: Sun, 29 Dec 2019 21:57:22 -0700 Subject: [openstack-ansible] strange execution delays Message-ID: Hello, I don't think this is an OSA issue per-se, but it's happening on infrastructure deployed with OSA. Using both the Rocky and Stein releases of OSA on Ubuntu 18.04, we're running into a strange a strange issue where the run times of Ansible will eventually take longer and longer. Digging into the issue more, we find that it's when Ansible executes a command within a container and the call to "su" triggers the delay. Over time, the delay increases more and more. The same behavior is seen if we SSH into containers. The only workaround we've been able to identify that works is to stop and start the containers. We have not been able to identify a root cause of this, nor any consistency to what containers this will happen on. Has anyone run into this? Thanks, Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From rony.khan at brilliant.com.bd Mon Dec 30 07:47:57 2019 From: rony.khan at brilliant.com.bd (Md. Farhad Hasan Khan) Date: Mon, 30 Dec 2019 13:47:57 +0600 Subject: Openstack Rocky to Train upgrade In-Reply-To: References: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> Message-ID: <0dfe01d5bee5$73ea2290$5bbe67b0$@brilliant.com.bd> Hi Radoslaw, Thanks for your suggestion. To upgrade from Rocky to Stein is there any documents or process to do, or simply YUM update with stein release will work. If you have any docs please provide me. Thanks/Rony -----Original Message----- From: Radosław Piliszek [mailto:radoslaw.piliszek at gmail.com] Sent: Sunday, December 29, 2019 4:50 PM To: rony.khan at brilliant.com.bd Cc: OpenStack Discuss Subject: Re: Openstack Rocky to Train upgrade Hi Rony, It usually depends on OpenStack distribution you used to deploy Rocky. Please note the recommended approach is to first upgrade to the next release, which is Stein, and then to Train. -yoctozepto niedz., 29 gru 2019 o 11:23 Md. Farhad Hasan Khan napisał(a): > > Hi, > > Could you please help me how can I upgrade from Openstack Rocky version to Train. > > > > Thanks & B’Rgds, > > Rony > > From radoslaw.piliszek at gmail.com Mon Dec 30 08:13:31 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 30 Dec 2019 09:13:31 +0100 Subject: Openstack Rocky to Train upgrade In-Reply-To: <0dfe01d5bee5$73ea2290$5bbe67b0$@brilliant.com.bd> References: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> <0dfe01d5bee5$73ea2290$5bbe67b0$@brilliant.com.bd> Message-ID: Hi Rony, If you are not using any deployment tools, then it might be a little painful to do upgrades. Please see https://docs.openstack.org/operations-guide/ops-upgrades.html for general notes. IMHO, the most painful point in your path would be to switch from nova placement to standalone placement: https://docs.openstack.org/placement/train/admin/upgrade-to-stein.html -yoctozepto pon., 30 gru 2019 o 08:48 Md. Farhad Hasan Khan napisał(a): > > Hi Radoslaw, > Thanks for your suggestion. > To upgrade from Rocky to Stein is there any documents or process to do, or simply YUM update with stein release will work. If you have any docs please provide me. > > Thanks/Rony > > > -----Original Message----- > From: Radosław Piliszek [mailto:radoslaw.piliszek at gmail.com] > Sent: Sunday, December 29, 2019 4:50 PM > To: rony.khan at brilliant.com.bd > Cc: OpenStack Discuss > Subject: Re: Openstack Rocky to Train upgrade > > Hi Rony, > > It usually depends on OpenStack distribution you used to deploy Rocky. > Please note the recommended approach is to first upgrade to the next release, which is Stein, and then to Train. > > -yoctozepto > > niedz., 29 gru 2019 o 11:23 Md. Farhad Hasan Khan napisał(a): > > > > Hi, > > > > Could you please help me how can I upgrade from Openstack Rocky version to Train. > > > > > > > > Thanks & B’Rgds, > > > > Rony > > > > > > From rony.khan at brilliant.com.bd Mon Dec 30 10:41:20 2019 From: rony.khan at brilliant.com.bd (Md. Farhad Hasan Khan) Date: Mon, 30 Dec 2019 16:41:20 +0600 Subject: Openstack Rocky to Train upgrade In-Reply-To: References: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> <0dfe01d5bee5$73ea2290$5bbe67b0$@brilliant.com.bd> Message-ID: <0f5f01d5befd$ac88d6b0$059a8410$@brilliant.com.bd> Hi Radoslaw, We did manual installation of every services of openstack Rocky version. Now we are planning to upgrade. That’s why we are looking how can we smoothly upgrade the OSP version. Thanks for your links. Thanks/Rony -----Original Message----- From: Radosław Piliszek [mailto:radoslaw.piliszek at gmail.com] Sent: Monday, December 30, 2019 2:14 PM To: rony.khan at brilliant.com.bd Cc: Radosław Piliszek; OpenStack Discuss Subject: Re: Openstack Rocky to Train upgrade Hi Rony, If you are not using any deployment tools, then it might be a little painful to do upgrades. Please see https://docs.openstack.org/operations-guide/ops-upgrades.html for general notes. IMHO, the most painful point in your path would be to switch from nova placement to standalone placement: https://docs.openstack.org/placement/train/admin/upgrade-to-stein.html -yoctozepto pon., 30 gru 2019 o 08:48 Md. Farhad Hasan Khan napisał(a): > > Hi Radoslaw, > Thanks for your suggestion. > To upgrade from Rocky to Stein is there any documents or process to do, or simply YUM update with stein release will work. If you have any docs please provide me. > > Thanks/Rony > > > -----Original Message----- > From: Radosław Piliszek [mailto:radoslaw.piliszek at gmail.com] > Sent: Sunday, December 29, 2019 4:50 PM > To: rony.khan at brilliant.com.bd > Cc: OpenStack Discuss > Subject: Re: Openstack Rocky to Train upgrade > > Hi Rony, > > It usually depends on OpenStack distribution you used to deploy Rocky. > Please note the recommended approach is to first upgrade to the next release, which is Stein, and then to Train. > > -yoctozepto > > niedz., 29 gru 2019 o 11:23 Md. Farhad Hasan Khan napisał(a): > > > > Hi, > > > > Could you please help me how can I upgrade from Openstack Rocky version to Train. > > > > > > > > Thanks & B’Rgds, > > > > Rony > > > > > > From aj at suse.com Mon Dec 30 13:53:44 2019 From: aj at suse.com (Andreas Jaeger) Date: Mon, 30 Dec 2019 14:53:44 +0100 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: On 28/12/2019 21.24, Akihiro Motoki wrote: > [...] > (side note) > django-babel is not a runtime requirement of horizon. It is just used > when we extract > translation strings and the current user is the infra translation jobs > [4], so I think we don't > need to care co-installability much. It can be moved to > test-requirements.txt in horizon > and we can install django-babel/enmerkar explicitly in > common_translation_update.sh in [4]. How will you support stable branches? As you know, the change needs to work with all branches we have translations on, Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From mriedemos at gmail.com Mon Dec 30 14:32:32 2019 From: mriedemos at gmail.com (Matt Riedemann) Date: Mon, 30 Dec 2019 08:32:32 -0600 Subject: Openstack Rocky to Train upgrade In-Reply-To: <0f5f01d5befd$ac88d6b0$059a8410$@brilliant.com.bd> References: <6e5701d5be30$c73b2f30$55b18d90$@brilliant.com.bd> <0dfe01d5bee5$73ea2290$5bbe67b0$@brilliant.com.bd> <0f5f01d5befd$ac88d6b0$059a8410$@brilliant.com.bd> Message-ID: <76ea4ef3-ceaf-5fc3-b13e-3792a432ccc6@gmail.com> On 12/30/2019 4:41 AM, Md. Farhad Hasan Khan wrote: > We did manual installation of every services of openstack Why? When there are several deployment and configuration management tools available, why deploy everything manually? Was it an educational exercise? I understand that, but not really for a production deployment which needs to eventually be upgraded. -- Thanks, Matt From mnaser at vexxhost.com Mon Dec 30 14:44:23 2019 From: mnaser at vexxhost.com (Mohammed Naser) Date: Mon, 30 Dec 2019 09:44:23 -0500 Subject: [openstack-ansible] strange execution delays In-Reply-To: References: Message-ID: Hi Joe, On Mon, Dec 30, 2019 at 12:00 AM Joe Topjian wrote: > > Hello, > > I don't think this is an OSA issue per-se, but it's happening on infrastructure deployed with OSA. > > Using both the Rocky and Stein releases of OSA on Ubuntu 18.04, we're running into a strange a strange issue where the run times of Ansible will eventually take longer and longer. Digging into the issue more, we find that it's when Ansible executes a command within a container and the call to "su" triggers the delay. Over time, the delay increases more and more. The same behavior is seen if we SSH into containers. > > The only workaround we've been able to identify that works is to stop and start the containers. > > We have not been able to identify a root cause of this, nor any consistency to what containers this will happen on. > > Has anyone run into this? No, this is pretty strange. Do you have any PAM modules that might be hitting some sorts of external API for auditing purposes that may be throttling you? How is systemd-logind feeling? Anything odd in your system logs? > Thanks, > Joe From ignaziocassano at gmail.com Mon Dec 30 14:48:29 2019 From: ignaziocassano at gmail.com (Ignazio Cassano) Date: Mon, 30 Dec 2019 15:48:29 +0100 Subject: [stein][cinder][backup][swift] issue Message-ID: Hello All, I configured openstack stein on centos 7 witch ceph. Cinder works fine and object storage on ceph seems to work fine: I can clreate containers, volume etc ..... I configured cinder backup on swift (but swift is using ceph rados gateway) : backup_driver = cinder.backup.drivers.swift.SwiftBackupDriver swift_catalog_info = object-store:swift:publicURL backup_swift_enable_progress_timer = True #backup_swift_url = http://10.102.184.190:8080/v1/AUTH_ backup_swift_auth_url = http://10.102.184.190:5000/v3 backup_swift_auth = per_user backup_swift_auth_version = 1 backup_swift_user = admin backup_swift_user_domain = default #backup_swift_key = #backup_swift_container = volumebackups backup_swift_object_size = 52428800 #backup_swift_project = #backup_swift_project_domain = backup_swift_retry_attempts = 3 backup_swift_retry_backoff = 2 backup_compression_algorithm = zlib If I run a backup as user admin, it creates a container named "volumebackups". If I run a backup as user demo and I do not specify a container name, it tires to write on volumebackups and gives some errors: ClientException: Container PUT failed: http://10.102.184.190:8080/swift/v1/AUTH_964f343cf5164028a803db91488bdb01/volumebackups 409 Conflict BucketAlreadyExists Does it mean I cannot use the same containers name on differents projects ? My ceph.conf is configured for using keystone: [client.rgw.tst2-osctrl01] rgw_frontends = "civetweb port=10.102.184.190:8080" # Keystone information rgw keystone api version = 3 rgw keystone url = http://10.102.184.190:5000 rgw keystone admin user = admin rgw keystone admin password = password rgw keystone admin domain = default rgw keystone admin project = admin rgw swift account in url = true rgw keystone implicit tenants = true Any help, please ? Best Regards Ignazio -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mcginnis at gmx.com Mon Dec 30 15:01:37 2019 From: sean.mcginnis at gmx.com (Sean McGinnis) Date: Mon, 30 Dec 2019 09:01:37 -0600 Subject: [oslo][kolla][requirements][release][infra] Hit by an old, fixed bug In-Reply-To: References: Message-ID: <20191230150137.GA9057@sm-workstation> On Sun, Dec 29, 2019 at 09:41:45PM +0100, Radosław Piliszek wrote: > Hi Folks, > > as the subject goes, my installation has been hit by an old bug: > https://bugs.launchpad.net/oslo.messaging/+bug/1828841 > (bug details not important, linked here for background) > > I am using Stein, deployed with recent Kolla-built source-based images > (with only slight modifications compared to vanilla ones). > Kolla's procedure for building source-based images considers upper > constraints, which, unfortunately, turned out to be lagging behind a > few releases w.r.t. oslo.messaging at least. > The fix was in 9.7.0 released on May 21, u-c still point to 9.5.0 from > Feb 26 and the latest of Stein is 9.8.0 from Jul 18. > > It seems oslo.messaging is missing from the automatic updates that bot proposes: > https://review.opendev.org/#/q/owner:%22OpenStack+Proposal+Bot%22+project:openstack/requirements+branch:stable/stein > > Per: > https://opendev.org/openstack/releases/src/branch/master/doc/source/reference/reviewer_guide.rst#release-jobs > this upper-constraint proposal should be happening for all releases. > This is normal and what is expected. Requirements are only updated for the branch in which those releases happen. So if there is a release of oslo.messaging for stable/train, only the stable/train upper constraints are updated for that new release. The stable/stein branch will not be affected because that shows what the tested upper constraints were for that branch. The last stable/stein release for oslo.messaging was 9.5.0: https://opendev.org/openstack/releases/src/branch/master/deliverables/stein/oslo.messaging.yaml#L49 And 9.5.0 is what is set in the stable/stein upper-constraints: https://opendev.org/openstack/requirements/src/branch/stable/stein/upper-constraints.txt#L146 To get that raised, whatever necessary bugfixes that are required in oslo.messaging would need to be backported per-cycle until stable/stein (as in, if it was in current master, it would need to be backported and merged to stable/train first, then stable/stein), and once merged a stable release would need to be proposed for that branch's version of the library. Once that stable release is done, that will propose the update to the upper constraint for the given branch. > I would be glad if someone investigated why it happens(/ed) and > audited whether other OpenStack projects don't need updating as well > to avoid running on old deps when new are awaiting for months. :-) > Please note this might apply to other branches as well. > > PS: for some reason oslo.messaging Stein release notes ( > https://docs.openstack.org/releasenotes/oslo.messaging/stein.html ) > are stuck at 9.5.0 as well, this could be right (I did not inspect the > sources) but I am adding this in PS so you have more things to > correlate if they need be. > Again, as expected. The last stable/stein release was 9.5.0, so that is correct that the release notes for stein only show up to that point. From radoslaw.piliszek at gmail.com Mon Dec 30 15:52:49 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Mon, 30 Dec 2019 16:52:49 +0100 Subject: [oslo][kolla][requirements][release][infra] Hit by an old, fixed bug In-Reply-To: <20191230150137.GA9057@sm-workstation> References: <20191230150137.GA9057@sm-workstation> Message-ID: Thanks, Sean! I knew I was missing something really basic! I was under the impression that 9.x is Stein, like it happens with main projects (major=branch). I could not find any doc explaining oslo.messaging versioning, perhaps Oslo could release 9.5.1 off the stein branch? The issue remains that, even though oslo backports bugfixes into their stable branches, kolla (and very possibly other deployment solutions) no longer benefit from them. -yoctozepto pon., 30 gru 2019 o 16:01 Sean McGinnis napisał(a): > > On Sun, Dec 29, 2019 at 09:41:45PM +0100, Radosław Piliszek wrote: > > Hi Folks, > > > > as the subject goes, my installation has been hit by an old bug: > > https://bugs.launchpad.net/oslo.messaging/+bug/1828841 > > (bug details not important, linked here for background) > > > > I am using Stein, deployed with recent Kolla-built source-based images > > (with only slight modifications compared to vanilla ones). > > Kolla's procedure for building source-based images considers upper > > constraints, which, unfortunately, turned out to be lagging behind a > > few releases w.r.t. oslo.messaging at least. > > The fix was in 9.7.0 released on May 21, u-c still point to 9.5.0 from > > Feb 26 and the latest of Stein is 9.8.0 from Jul 18. > > > > It seems oslo.messaging is missing from the automatic updates that bot proposes: > > https://review.opendev.org/#/q/owner:%22OpenStack+Proposal+Bot%22+project:openstack/requirements+branch:stable/stein > > > > Per: > > https://opendev.org/openstack/releases/src/branch/master/doc/source/reference/reviewer_guide.rst#release-jobs > > this upper-constraint proposal should be happening for all releases. > > > > This is normal and what is expected. > > Requirements are only updated for the branch in which those releases happen. So > if there is a release of oslo.messaging for stable/train, only the stable/train > upper constraints are updated for that new release. The stable/stein branch > will not be affected because that shows what the tested upper constraints were > for that branch. > > The last stable/stein release for oslo.messaging was 9.5.0: > > https://opendev.org/openstack/releases/src/branch/master/deliverables/stein/oslo.messaging.yaml#L49 > > And 9.5.0 is what is set in the stable/stein upper-constraints: > > https://opendev.org/openstack/requirements/src/branch/stable/stein/upper-constraints.txt#L146 > > To get that raised, whatever necessary bugfixes that are required in > oslo.messaging would need to be backported per-cycle until stable/stein (as in, > if it was in current master, it would need to be backported and merged to > stable/train first, then stable/stein), and once merged a stable release would > need to be proposed for that branch's version of the library. > > Once that stable release is done, that will propose the update to the upper > constraint for the given branch. > > > I would be glad if someone investigated why it happens(/ed) and > > audited whether other OpenStack projects don't need updating as well > > to avoid running on old deps when new are awaiting for months. :-) > > Please note this might apply to other branches as well. > > > > PS: for some reason oslo.messaging Stein release notes ( > > https://docs.openstack.org/releasenotes/oslo.messaging/stein.html ) > > are stuck at 9.5.0 as well, this could be right (I did not inspect the > > sources) but I am adding this in PS so you have more things to > > correlate if they need be. > > > > Again, as expected. The last stable/stein release was 9.5.0, so that is correct > that the release notes for stein only show up to that point. From skaplons at redhat.com Mon Dec 30 16:08:23 2019 From: skaplons at redhat.com (Slawek Kaplonski) Date: Mon, 30 Dec 2019 17:08:23 +0100 Subject: [neutron] Bug deputy report week of December 23rd In-Reply-To: References: Message-ID: <7A3CBE2C-F20D-40E9-8866-657386F42D4C@redhat.com> Hi, Thx Miguel. Now I will take care of it. > On 29 Dec 2019, at 00:22, Miguel Lavalle wrote: > > Hi, > > I was the bug deputy for the week of December 23rd. As expected for an end of year Holidays week, We had very little activity: > > Requested more information: > > - https://bugs.launchpad.net/neutron/+bug/1857422 neutron-keepalived-state-change and keeplived cannot be cleanup for those routers which is deleted during l3-agent died — Slawek Kaplonski Senior software engineer Red Hat From joe at topjian.net Mon Dec 30 19:51:34 2019 From: joe at topjian.net (Joe Topjian) Date: Mon, 30 Dec 2019 12:51:34 -0700 Subject: [openstack-ansible] strange execution delays In-Reply-To: References: Message-ID: Hi Mohammad, Do you have any PAM modules that might be hitting some sorts of > external API for auditing purposes that may be throttling you? > Not unless OSA would have configured something. The deployment is *very* standard, heavily leveraging default values. DNS of each container is configured to use LXC host for resolution. The host is using the systemd-based resolver, but is pointing to a local, dedicated upstream resolver. I want to point the problem there, but we've run into this issue in two different locations, one of which has an upstream DNS resolver that I'm confident does not throttle requests. But, hey, it's DNS - maybe it's still the cause. > How is systemd-logind feeling? Anything odd in your system logs? > Yes. We have a feeling it's *something* with systemd, but aren't exactly sure what. Affected containers' logs end up with a lot of the following entries: Dec 3 20:30:17 infra1-repo-container-a0f194b3 su[4170]: Successful su for root by root Dec 3 20:30:17 infra1-repo-container-a0f194b3 su[4170]: + ??? root:root Dec 3 20:30:17 infra1-repo-container-a0f194b3 su[4170]: pam_unix(su:session): session opened for user root by (uid=0) Dec 3 20:30:27 infra1-repo-container-a0f194b3 dbus-daemon[47]: [system] Failed to activate service 'org.freedesktop.systemd1': timed out (service_start_timeout=25000ms) Dec 3 20:30:42 infra1-repo-container-a0f194b3 su[4170]: pam_systemd(su:session): Failed to create session: Connection timed out Dec 3 20:30:43 infra1-repo-container-a0f194b3 su[4170]: pam_unix(su:session): session closed for user root But we aren't sure if those timeouts are a symptom of cause. Thanks for your help! Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From amotoki at gmail.com Tue Dec 31 00:10:22 2019 From: amotoki at gmail.com (Akihiro Motoki) Date: Tue, 31 Dec 2019 09:10:22 +0900 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: On Mon, Dec 30, 2019 at 10:58 PM Andreas Jaeger wrote: > > On 28/12/2019 21.24, Akihiro Motoki wrote: > > [...] > > (side note) > > django-babel is not a runtime requirement of horizon. It is just used > > when we extract > > translation strings and the current user is the infra translation jobs > > [4], so I think we don't > > need to care co-installability much. It can be moved to > > test-requirements.txt in horizon > > and we can install django-babel/enmerkar explicitly in > > common_translation_update.sh in [4]. > > How will you support stable branches? > > As you know, the change needs to work with all branches we have > translations on, What in my mind is just to move django-babel/enmerkar to test-requirements.txt in the master (ussuri). project-config is branchless, so I believe it is enough to make the following change in install_horizon(). Doesn't it work? (cd ${HORIZON_DIR} && pip install -c $UPPER_CONSTRAINTS_FILE .) to (cd ${HORIZON_DIR} && pip install -c $UPPER_CONSTRAINTS_FILE . && pip install -c $UPPER_CONSTRAINTS_FILE test-requirements.txt) > > Andreas > -- > Andreas Jaeger aj at suse.com Twitter: jaegerandi > SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg > (HRB 36809, AG Nürnberg) GF: Felix Imendörffer > GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB > From aj at suse.com Tue Dec 31 10:43:56 2019 From: aj at suse.com (Andreas Jaeger) Date: Tue, 31 Dec 2019 11:43:56 +0100 Subject: [horizon][requirements] django-3 blocked by usage of django-babel In-Reply-To: References: <20191221212450.64vy6j5sbfyqpql5@mthode.org> Message-ID: <2a6efb7a-a002-9378-f10b-e9174534c50f@suse.com> On 31/12/2019 01.10, Akihiro Motoki wrote: > On Mon, Dec 30, 2019 at 10:58 PM Andreas Jaeger wrote: >> >> On 28/12/2019 21.24, Akihiro Motoki wrote: >>> [...] >>> (side note) >>> django-babel is not a runtime requirement of horizon. It is just used >>> when we extract >>> translation strings and the current user is the infra translation jobs >>> [4], so I think we don't >>> need to care co-installability much. It can be moved to >>> test-requirements.txt in horizon >>> and we can install django-babel/enmerkar explicitly in >>> common_translation_update.sh in [4]. >> >> How will you support stable branches? >> >> As you know, the change needs to work with all branches we have >> translations on, > > What in my mind is just to move django-babel/enmerkar to > test-requirements.txt in the master (ussuri). > project-config is branchless, so I believe it is enough to make the > following change in install_horizon(). > Doesn't it work? > > (cd ${HORIZON_DIR} && pip install -c $UPPER_CONSTRAINTS_FILE .) > to > (cd ${HORIZON_DIR} && pip install -c $UPPER_CONSTRAINTS_FILE . && > pip install -c $UPPER_CONSTRAINTS_FILE test-requirements.txt) That should work - too simple ;) Andreas -- Andreas Jaeger aj at suse.com Twitter: jaegerandi SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, D 90409 Nürnberg (HRB 36809, AG Nürnberg) GF: Felix Imendörffer GPG fingerprint = EF18 1673 38C4 A372 86B1 E699 5294 24A3 FF91 2ACB From jasonanderson at uchicago.edu Tue Dec 31 15:23:33 2019 From: jasonanderson at uchicago.edu (Jason Anderson) Date: Tue, 31 Dec 2019 15:23:33 +0000 Subject: [neutron] Slow floating IP activation when FWaaS v2 enabled Message-ID: <638b8947-1a8e-6431-b898-3d6ecdb6a259@uchicago.edu> Hi all, We recently started using the FWaaS feature (v2) in Neutron for our deployment. However, we noticed that when this feature is enabled, floating IP activation (the setting up of iptables rules that perform DNAT to the destination instance) takes a long time to happen. The effect is that the user has to wait for a while before the floating IP becomes active on an instance. I was wondering if anybody else has experienced this, and if there are any workarounds. I suspect there may be some queuing happening behind an iptables lock somewhere but I haven't been able to verify anything for sure yet. Thank you, -- Jason Anderson Chameleon DevOps Lead Consortium for Advanced Science and Engineering, The University of Chicago Mathematics & Computer Science Division, Argonne National Laboratory -------------- next part -------------- An HTML attachment was scrubbed... URL: From fungi at yuggoth.org Tue Dec 31 19:11:39 2019 From: fungi at yuggoth.org (Jeremy Stanley) Date: Tue, 31 Dec 2019 19:11:39 +0000 Subject: [oslo][kolla][requirements][release][infra] Hit by an old, fixed bug In-Reply-To: References: <20191230150137.GA9057@sm-workstation> Message-ID: <20191231191139.txzccdjdzfjhyifq@yuggoth.org> On 2019-12-30 16:52:49 +0100 (+0100), Radosław Piliszek wrote: [...] > I could not find any doc explaining oslo.messaging versioning [...] The releases site can be used to identify the most recent releases for each series: https://releases.openstack.org/stein/index.html#library-projects As for how the project is versioned, most libraries (including basically all of Oslo) follow this model: https://releases.openstack.org/reference/release_models.html#cycle-with-intermediary Not sure if that's what you're looking for in regards to documentation explaining versioning for that project, but hopefully it's a start. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: not available URL: From radoslaw.piliszek at gmail.com Tue Dec 31 20:21:16 2019 From: radoslaw.piliszek at gmail.com (=?UTF-8?Q?Rados=C5=82aw_Piliszek?=) Date: Tue, 31 Dec 2019 21:21:16 +0100 Subject: [oslo][kolla][requirements][release][infra] Hit by an old, fixed bug In-Reply-To: <20191231191139.txzccdjdzfjhyifq@yuggoth.org> References: <20191230150137.GA9057@sm-workstation> <20191231191139.txzccdjdzfjhyifq@yuggoth.org> Message-ID: Thanks, already familiar with these. I was looking for something more specific actually, like for when to propose a bugfix release after the cycle deliverable has been released. Sorry for the abundance of tags (incl. [infra]), at first I thought the stagnation was due to some procedure failing, yet it seems simply nobody proposed to rerelease oslo.messaging for Stein. Waiting for someone from Oslo to answer in this thread with more insights. :-) This is important for Kolla as we might be better off sourcing oslo.messaging from git rather than relying on releases which may never happen after the freeze... -yoctozepto wt., 31 gru 2019 o 20:24 Jeremy Stanley napisał(a): > > On 2019-12-30 16:52:49 +0100 (+0100), Radosław Piliszek wrote: > [...] > > I could not find any doc explaining oslo.messaging versioning > [...] > > The releases site can be used to identify the most recent releases > for each series: > > https://releases.openstack.org/stein/index.html#library-projects > > As for how the project is versioned, most libraries (including > basically all of Oslo) follow this model: > > https://releases.openstack.org/reference/release_models.html#cycle-with-intermediary > > Not sure if that's what you're looking for in regards to > documentation explaining versioning for that project, but hopefully > it's a start. > -- > Jeremy Stanley