- openstack-discuss - lists.openstack.org

[ops] ops meetups team meeting 2019-12-3
by Chris Morgan 03 Dec '19

03 Dec '19

Minutes from todays meeting on #openstack-operators Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… London meetup links: https://go.bloomberg.com/attend/invite/openstack-operators-meetup-london202… https://etherpad.openstack.org/p/LON-2020-OPS-TOPICS South Korea meetup proposal: https://etherpad.openstack.org/p/ops-meetup-2nd-korea-2020 Event notification twitter account: https://twitter.com/osopsmeetup Chris Morgan - on behalf of the OpenStack Ops Meetups team -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[neutron] Bug Deputy report 2019-11-25 - 2019-12-02
by Nate Johnston 03 Dec '19

03 Dec '19

Here are my notes as Neutron bug deputy for last week. It was a pretty slow week. Critical: - "Old conjunction left after sg update" * URL: https://bugs.launchpad.net/neutron/+bug/1854131 * Status: Fix merged (yang-li) - "py36 unit test cases fails" * URL: https://bugs.launchpad.net/bugs/1854051 * Gate-Failure * Status: Unassigned, may be a typing issue, will bring it up in Neutron CI meeting. High: - "Neutron fails to create bandwidth providers if CONF.host is set" * URL: https://bugs.launchpad.net/neutron/+bug/1853840 * Status: In progress, assigned to rubasov - "[Functional tests] Timeout exception in list_namespace_pids" * URL: https://bugs.launchpad.net/bugs/1854462 * Gate-Failure * Status: Unassigned, will bring it up in Neutron CI meeting. Medium: - "The /v2.0/ports/{port_id}/bindings APIs are not documented" * URL: https://bugs.launchpad.net/bugs/1853873 * Status: Assigned to mlavalle Low: - "Avoid raising NetworkInterfaceNotFound exception in DHCP agent logs" * URL: https://bugs.launchpad.net/bugs/1854723 * Status: In progress, assigned to ralonsoh - "DhcpLocalProcess "_enable" method should not call 'restart' --> 'enable'" * URL: https://bugs.launchpad.net/neutron/+bug/1854940 * Status: Assigned to ralonsoh Invalid: - "minor versions 14.0.2 & 14.0.3 are not compatible in dvr-ha" * URL: https://bugs.launchpad.net/bugs/1854050 * Status: agent running higher version than neutron server not supported Thanks, Nate

1 0

Re: [openstack-community] Openstack FW rules
by Amy Marrich 03 Dec '19

03 Dec '19

Hi Lior, Forwarding this to the OpenStack discuss list where someone might be able to help you. Thanks, Amy (spotz) On Tue, Dec 3, 2019 at 7:41 AM Lior Fellus <felluslior(a)gmail.com> wrote: > Hi, > i am using OpenStack "Mitaka" and we have 3 controllers and 3 computes on > our environment and all managed by Fuel server. > recently i implemented a Veeam backup Server in our environment and in > order to backup the computes and the controllers i need to open ssh to them > directly because Veeam not support ssh tunnel via Fuel server. > can anyone explain me how ? > > Regards, > _______________________________________________ > Community mailing list > Community(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/community >

1 0

VMs auto start
by Alfredo De Luca 03 Dec '19

03 Dec '19

Hi all. Just a quick one. I can't find the settings for VMs auto start after a computer node reboots. Can you point me to the right docs? Which one is it? And you need to add it on nova scheduler right? Cheers -- *Alfredo*

2 2

[gnocchi][telemetry][ceilometer][cloudkitty] Gnocchi unmaintained
by Radosław Piliszek 03 Dec '19

03 Dec '19

Hello Folks, It looks like gnocchi is "officially" marked as unmaintained: https://github.com/gnocchixyz/gnocchi/issues/1049 Has there been any discussion regarding how it affects OpenStack projects? And/or are there any plans to amend this situation? -yoctozepto

15 28

[requirements][os-ken][neutron] update of tinyrpc from 1.0.3 to 1.0.4 breaks neutron-grenade
by Matthew Thode 03 Dec '19

03 Dec '19

2 2

Magnum: coe cluster can not be deleted (DELETE_FAILED)
by Pawel Konczalski 02 Dec '19

02 Dec '19

Hello, i try to delete a broken magnum (v7.1.1.dev21) cluster but neither the cluster or the stack entries can be deleted. Any idea how to force / fix this? BR Pawel openstack coe cluster list | grep DELETE_FAILED openstack coe cluster delete ae5c347c-3ba8-4d39-9125-0bdadef253b4 openstack coe cluster show ae5c347c-3ba8-4d39-9125-0bdadef253b4 | status | DELETE_FAILED ... |--- | status_reason | Resource DELETE failed: JSONDecodeError: resources.kube_masters.resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0) ... | faults | {'1': 'JSONDecodeError: resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0)', 'kube_masters': 'JSONDecodeError: resources.kube_masters.resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0)', '0': 'JSONDecodeError: resources[0].resources.api_pool_member: Expecting value: line 1 column 1 (char 0)', 'api_pool_member': 'JSONDecodeError: resources.api_pool_member: Expecting value: line 1 column 1 (char 0)', 'etcd_pool_member': 'JSONDecodeError: resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0)'} | openstack stack list | grep DELETE_FAILED openstack stack delete 2a907fc4-7b40-43f6-9254-d4084bd056b8 t Payload DELETE: ResourceGroup "kube_masters" [85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec] Stack "slu-k8s-cluster2-6xiqrpfe7jd6" [2a907fc4-7b40-43f6-9254-d4084bd056b8] 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource Traceback (most recent call last): 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", line 924, in _action_recorder 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource yield 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", line 2034, in delete 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource *action_args) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/scheduler.py", line 346, in wrapper 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource step = next(subtask) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resource.py", line 986, in action_handler_task 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource done = check(handler_data) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 596, in check_delete_complete 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource return self._check_status_complete(self.DELETE) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource File "/var/lib/kolla/venv/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 463, in _check_status_complete 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource action=action) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource ResourceFailure: JSONDecodeError: resources.kube_masters.resources[1].resources.etcd_pool_member: Expecting value: line 1 column 1 (char 0) 2019-12-02 12:43:08.887 22 ERROR heat.engine.resource # openstack stack list --nested | grep 7390a9b1d4be4d75b4bd08ab8107e4ff | 77848f9e-ff0c-4f7a-9fd7-8ed9e979b998 | test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd-1-tpqatosey33u | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | 2019-05-13T15:28:04Z | 2019-12-02T15:30:53Z | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | 2019-05-13T15:28:03Z | 2019-12-02T15:30:53Z | 2a907fc4-7b40-43f6-9254-d4084bd056b8 | | 4c91c53b-42f4-41ca-a97c-49d9261069d6 | test-k8s-cluster2-6xiqrpfe7jd6-kube_masters-mlvsrucewmsd-0-hm2ucynq6hkc | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | 2019-05-13T15:28:03Z | 2019-12-02T15:30:54Z | 85e1c6be-bdc8-41a3-ba0f-5ca65cdea8ec | | 2a907fc4-7b40-43f6-9254-d4084bd056b8 | test-k8s-cluster2-6xiqrpfe7jd6 | 7390a9b1d4be4d75b4bd08ab8107e4ff | DELETE_FAILED | 2019-05-13T15:24:45Z | 2019-12-02T15:30:53Z | None |

2 1

[all] Shanghai PTG Feedback
by Kendall Nelson 02 Dec '19

02 Dec '19

Hello Everyone! Now that we've had a few weeks to catch up after the summit and marinate on everything we accomplished together there, we wanted to take a sec to see if anyone had feedback. Normally we would have held a lunchtime feedback session, but due to venue restrictions, we didn't have a good place to do things like normal so we created the etherpad[1] and are distributing it this way :) Hope to hear everyone's feedback! Thanks! -the Kendalls (diablo_rojo & wendallkaters) [1]https://etherpad.openstack.org/p/PVG-PTG-feedback

1 0

[stein][manila-ui] error
by Ignazio Cassano 02 Dec '19

02 Dec '19

Hello, I just installed openstack stein on centos. Manila works fine my command line but when I click "share" in the dashboard the following error appears: Environment: Request Method: GET Request URL: http://10.102.184.190/dashboard/project/shares/ Django Version: 1.11.20 Python Version: 2.7.5 Installed Applications: ['openstack_dashboard.dashboards.project', 'neutron_lbaas_dashboard', 'heat_dashboard', 'openstack_dashboard.dashboards.admin', 'openstack_dashboard.dashboards.identity', 'openstack_dashboard.dashboards.settings', 'dashboards', 'openstack_dashboard', 'django.contrib.contenttypes', 'django.contrib.auth', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'django_pyscss', 'debreach', 'openstack_dashboard.django_pyscss_fix', 'compressor', 'horizon', 'openstack_auth'] Installed Middleware: ('openstack_auth.middleware.OpenstackAuthMonkeyPatchMiddleware', 'debreach.middleware.RandomCommentMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'horizon.middleware.OperationLogMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'horizon.middleware.HorizonMiddleware', 'horizon.themes.ThemeMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware', 'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware') Traceback: File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py" in inner 41. response = get_response(request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response 187. response = self.process_exception_by_middleware(e, request) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py" in _get_response 185. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 36. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 52. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 36. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 113. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/decorators.py" in dec 84. return view_func(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in view 68. return self.dispatch(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch 88. return handler(request, *args, **kwargs) File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in get 223. handled = self.construct_tables() File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in construct_tables 214. handled = self.handle_table(table) File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in handle_table 123. data = self._get_data_dict() File "/usr/lib/python2.7/site-packages/horizon/tables/views.py" in _get_data_dict 43. data.extend(func()) File "/usr/lib/python2.7/site-packages/horizon/utils/memoized.py" in wrapped 109. value = cache[key] = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/manila_ui/dashboards/project/shares/views.py" in get_shares_data 57. share_nets = manila.share_network_list(self.request) File "/usr/lib/python2.7/site-packages/manila_ui/api/manila.py" in share_network_list 280. return manilaclient(request).share_networks.list(detailed=detailed, Exception Type: AttributeError at /project/shares/ Exception Value: 'NoneType' object has no attribute 'share_networks' Anyone can help, please ?

3 7

[cinder] Anastasiya accepted for Outreachy
by Sofia Enriquez 02 Dec '19

02 Dec '19

Hi Cinder team, I'd like to announce that Anastasiya will be working with us improving the Tempest coverage this round. The internship schedule starts on Dec. 3, 2019, to March 3, 2020. Feel free to reach her on IRC *as anastzhyr* if something comes up. On the other hand, If you have any suggestions for tempest test scenarios or possible ideas, please let me know. Regards, Sofi -- L. Sofía Enriquez she/her Associate Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

6 7

"Transible" - convert your current cloud state to Ansible playbooks
by Sagi Shnaidman 02 Dec '19

02 Dec '19

Hi, all I'd like to present you a new tool I wrote recently, although it's under heavy development, but it can be already used and can be helpful in some of your DevOps/CloudOps tasks. I'd appreciate any feedback on it and any contributions of course :) Transible [1] (TRansformANSIBLE) - converts existing cloud state to ansible playbooks. It's a tool that takes your current cloud resources and represents them as Ansible playbooks. Running these playbooks you *ideally* will get no "changed" tasks and the same cloud configuration. Currently, it works with Openstack servers, security rules, security groups, images, volumes, servers, routers, networks, subnets, users, domains and projects which your cloud includes and defines their config as tasks in Ansible playbooks that are ready for deployment. By running these playbooks you actually can deploy or redeploy your current cloud. What are the possible use cases? 1. Freezing cloud config for further IaaC management If you have a ton of deployed resources and lost your tracking, better to start to manage them according to Infrastructure as Code principles. Creating a configuration of everything you have now allows you to change, delete, add things now via code. 2. Keeping current cloud configuration Sometimes it's required to have a "snapshot" of mandatory resources on the cloud and keep them the same no matter what happens in the cloud, like heavy testing for example. Running these playbooks will ensure you have all you need in the cloud in the required configuration. 3. Moving current infrastructure to another tenant, cloud, provider, whatever If you move to another provider but want to keep all you need and as it's configured right now, you can create playbooks with Transible and use them in the new cloud or provider. 4. In the future, I plan to add translation from one cloud configuration to another and it may help with releasing from vendor lock. Of course, because of various clouds' differences, it'll be limited. Can you come up with more? More details about the tool and how it can be configured for various use cases you can read in my longer articles on Medium [2] or Dev.to [3]. I'd appreciate your feedback here or in issues of Transible project. [4] Thanks [1] https://github.com/sshnaidm/transible [2] https://medium.com/@einarum/https-medium-com-einarum-transible-represents-c… [3] https://dev.to/sshnaidm/transible-represents-cloud-configuration-as-ansible… [4] https://github.com/sshnaidm/transible/issues -- Best regards Sagi Shnaidman

1 0

[mitaka][keystone] Authentication over keycloak server possible?
by Michael Stang 02 Dec '19

02 Dec '19

Hi, we have an OpenStack Mitaka installation running (yes I know it's pretty old ;-) ) at our lab and would like to use the keycloak-server from the central IT for authentication. So I would like to know if it is already possible in mitaka to use this external keycloak server or if this only possible in a later OpenStack version? Maybe anyone know and if yes is there any documentation how to do it? Was searching for it but found not much about it by now... Thanks :) Kind regards Michael Michael Stang Laboringenieur, Dipl. Inf. (FH) Duale Hochschule Baden-Württemberg Mannheim Baden-Wuerttemberg Cooperative State University Mannheim ZeMath Zentrum für mathematisch-naturwissenschaftliches Basiswissen Fachbereich Informatik, Fakultät Technik Coblitzallee 1-9 68163 Mannheim michael.stang(a)dhbw-mannheim.de mailto:michael.stang@dhbw-mannheim.de http://www.mannheim.dhbw.de http://www.dhbw-mannheim.de/

2 2

Open Infrastructure Summit and PTG Shanghai - edge recap
by Ildiko Vancsa 02 Dec '19

02 Dec '19

Hi, I wrote up a quick summary about edge activities at the Open Infrastructure Summit and PTG in Shanghai to give a brief summary to people who were there and to catch up those who couldn’t attend this time. You can access the content here: http://lists.openstack.org/pipermail/edge-computing/2019-December/000664.ht… Please let me know if you have any questions or comments. Thanks, Ildikó (IRC: ildikov on Freenode)

1 0

[magnum][stein] error
by Ignazio Cassano 02 Dec '19

02 Dec '19

Hello, I've juste installed magnum on centos 7 stein, when magnu-api starts it does not log under /var/log/magnum but under /var/log/messages and it reports the following error: magnum-api: error: [Errno 32] Broken pipe Please , help me on this thread. Regards Ignazio

1 0

[charms] IRC meetings and the holiday season
by Frode Nordahl 02 Dec '19

02 Dec '19

Hello all, We are entering the holiday season, and as such we will stop our IRC meeting activity until we enter the next year. The next meeting will be held Monday January 13th 2020. I want to use this opportunity to say thank you to all activity and contributions to the charms project this year, and I look forward to what the next year might bring. Cheers! -- Frode Nordahl

1 0

[all] Nominations for the "V" release name
by Sean McGinnis 02 Dec '19

02 Dec '19

Hey everyone, There is ongoing discussion about changing our release naming process, but for the time being we are going to stick with what we have been doing. That means it's time to start thinking about the "V" release name! The next developer event will take place in Vancouver, BC. The geographic location for this release will be things starting with "V" in the British Columbia province. The nomination period is now open. Please add suitable names to https://wiki.openstack.org/wiki/Release_Naming/V_Proposals. We will accept nominations until December 6, 2019 23:59:59 UTC. A recap of our current naming rules: * Each release name must start with the letter of the ISO basic Latin alphabet following the initial letter of the previous release, starting with the initial release of "Austin". After "Z", the next name should start with "A" again. * The name must be composed only of the 26 characters of the ISO basic Latin alphabet. Names which can be transliterated into this character set are also acceptable. * The name must refer to the physical or human geography of the region encompassing the location of the OpenStack design summit for the corresponding release. The exact boundaries of the geographic region under consideration must be declared before the opening of nominations, as part of the initiation of the selection process. * The name must be a single word with a maximum of 10 characters. Words that describe the feature should not be included, so "Foo City" or "Foo Peak" would both be eligible as "Foo". Names which do not meet these criteria but otherwise sound really cool should be added to a separate section of the wiki page and the TC may make an exception for one or more of them to be considered in the Condorcet poll. The naming official is responsible for presenting the list of exceptional names for consideration to the TC before the poll opens. Additional information about the release naming process can be found here: https://governance.openstack.org/tc/reference/release-naming.html Looking forward to having a name for our next release! Sean

4 7

[neutron] Including networking-ovn-core team into neutron-core
by Slawek Kaplonski 02 Dec '19

02 Dec '19

Hi neutrinos, We recently merged spec [1] and now we are starting to moving networking-ovn code to be in neutron tree. Blueprint to track progress of this is on [2]. As a consequence of this merge of code, I think that we need to also include networking-ovn-core team into neutron-core to give people who are cores on networking-ovn today ability to merge ovn related patches in neutron after this migration will be done. Of course current networking-ovn cores should only approve patches related to ovn driver, and not approve patches related to different areas of neutron code. So if there will be no objections for that until the end of this week, I will include networking-ovn-core group in neutron-core. [1] https://review.opendev.org/#/c/658414/ [2] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge — Slawek Kaplonski Senior software engineer Red Hat

2 2

[blazar] IRC meetings cancelled this week
by Pierre Riteau 02 Dec '19

02 Dec '19

Hello, Due to travel, both Blazar IRC meetings are cancelled this week. Cheers, Pierre

1 0

[neutron] DNS resolution delay after VM launch
by Eric K. Miller 01 Dec '19

01 Dec '19

Hi, I'm trying to resolve an issue that happens about 30% of the time. DVR is being used in this environment. The environment is deployed with Kolla Ansible on CentOS 7 using Stein (the latest Kolla Ansible installer). Physical servers have the latest CentOS 7 patches through November 30th, 2019. This seems to happen on new and existing projects, including newly created subnets, and, so far, it seems to be only related to VM launches. Delaying the time between a subnet creation and the VM launch makes no difference in the behavior. When the problem occurs, after a VM is launched, it takes a longer-than-normal time to boot. This would usually indicate a DNS issue or possible a DHCP issue. After the OS boots (after 60 seconds or so), I can login, but with a delayed login (another 20 seconds or so) due to the DNS issue I'm diagnosing. DHCP works fine - no issues with IP assignment, and the /etc/resolv.conf entries are set to the routers' two private IPs. No DHCP errors in the dmesg output. I can connect to both respective qdhcp network namespaces where dnsmasq is running for the two distributed routers, immediately after the subnet is created (in our test case, it is 192.168.99.0/24), and can resolve names using "nslookup <name> 192.168.99.1" and "nslookup <name> 192.168.99.2" instantly. So, dnsmasq is working properly. A floating IP is assigned to this VM, which is used for the following SSH sessions. After SSH'ing to the VM, I can ping the 192.168.99.1 and 192.168.99.2 addresses, so it does not appear to be a network issue (ICMP works), unless there is a firewall rule either in iptables or OVS that is blocking UDP/TCP port 53 for DNS requests, which would be odd, since the egress security group assigned to this VM is unrestricted. I can also ping external IPs from the VM (Internet-accessible IPs), so the routers are working. I can also perform nslookups against external DNS servers without issue, such as "nslookup <name> 1.1.1.1". If I direct nslookup to the internal DNS servers using "nslookup <name> 192.168.99.1", the lookup times out. We normally use CentOS images, but I tried an Ubuntu 19.04 image and the same problem occurs often. Now, what is ODD - the problem goes away after about 3 minutes! DNS lookups against the internal DNS servers (the router IPs) start working perfectly. So something is taking a while to get configured, but only about 30% of the time. If, during the creation of the subnet, we specify --dns-nameserver options, such as: --dns-nameserver 1.1.1.1 the issue NEVER occurs. So it seems to be limited to DNS traffic between the VM and the dnsmasq service in the qdhcp namespace. All traffic in this environment is switched (we are not routing VXLAN traffic, for example), so there can be no physical network interference at Layers 3 or higher. The problem has also occurred on multiple hosts in the environment (the VM has been launched on various hosts with the same issue). Any ideas how to diagnose this? I'm assuming it has something to do with the iptables or OVS configuration, but if someone knows specifically what could cause this, such as a specific rule that is supposed to be created for DNS traffic to the internal routers, I could dive into the these configurations. Eric

1 0

[Usage][Designate] Salve Bind servers
by Steven Relf 01 Dec '19

01 Dec '19

Hi list.... I'm looking at making use of Designate, and have so farmanaged to get designate configured and working against a single bind9 server. The issue I'm having, and this could be due to notunderstanding bind config well, is how do I configure other bind servers to beslaves of the server used by designate. I keep running into an issue where theslave doesn’t want to do a transfer and errors with "received notify forzone 'domain': not authoritative" I'm assuming this has something to do with the slave notknowing about the zone. So the question, do I have to have all my bind servershooked upto designate, and managed with a mdns service, or am I missingsomething obvious. Rgds Steve. The future has already arrived. It's just not evenlydistributed yet - William Gibson

2 1

[Magnum] Virtual PTG planning
by Feilong Wang 01 Dec '19

01 Dec '19

Hi team, As we discussed on last weekly team meeting, we'd like to have a virtual PTG before the Xmas holiday to plan our work for the U release. The general idea is extending our current weekly meeting time from 1 hour to 2 hours and having 2 sessions with total 4 hours. My current proposal is as below, please reply if you have question or comments. Thanks. Pre discussion/Ideas collection: 20th Nov 9:00AM-10:00AM UTC 1st Session: 27th Nov 9:00AM-11:00AM UTC 2nd Session: 4th Dec 9:00AM-11:00AM UTC -- Cheers & Best regards, Feilong Wang (王飞龙) Head of R&D Catalyst Cloud - Cloud Native New Zealand -------------------------------------------------------------------------- Tel: +64-48032246 Email: flwang(a)catalyst.net.nz Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------

3 4

Openstack - Rocky Install
by doug schmidt 30 Nov '19

30 Nov '19

Hi, I'm having an issue with httpd not starting on my install. I have installed rocky on a windows 10 laptop with virtualbox and centos 7 guests. So far I have 2 controllers and 3 compute nodes. I have followed the install docs for rocky. https://docs.openstack.org/install-guide/openstack-services.html#minimal-de… Minimal services have been installed and configured.keystone, glance, nova, neutron are configured and running. I followed the install for horizon dashboard, and that is where I'm getting httpd.service failing. I have gone over the configuration a few times, but I can not find what the issue is. Any ideas of where to look or what to fix? Thanks for any help with this ------------------------ [root@openstack-cntr1 ~]# systemctl | egrep -i 'openstack|neutron|httpd' ● httpd.service loaded failed failed The Apache HTTP Server neutron-dhcp-agent.service loaded active running OpenStack Neutron DHCP Agent neutron-linuxbridge-agent.service loaded active running OpenStack Neutron Linux Bridge Agent neutron-metadata-agent.service loaded active running OpenStack Neutron Metadata Agent neutron-server.service loaded active running OpenStack Neutron Server openstack-glance-api.service loaded active running OpenStack Image Service (code-named Glance) API server openstack-glance-registry.service loaded active running OpenStack Image Service (code-named Glance) Registry server openstack-nova-api.service loaded active running OpenStack Nova API Server openstack-nova-conductor.service loaded active running OpenStack Nova Conductor Server openstack-nova-consoleauth.service loaded active running OpenStack Nova VNC console auth Server openstack-nova-novncproxy.service loaded active running OpenStack Nova NoVNC Proxy Server openstack-nova-scheduler.service loaded active running OpenStack Nova Scheduler Server [root@openstack-cntr1 ~]# systemctl start httpd.service Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details. [root@openstack-cntr1 ~]# systemctl status httpd.service ● httpd.service - The Apache HTTP Server Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/httpd.service.d └─openstack-dashboard.conf Active: failed (Result: exit-code) since Sat 2019-11-30 09:45:46 EST; 15s ago Docs: man:httpd(8) man:apachectl(8) Process: 12855 ExecStartPre=/usr/bin/python /usr/share/openstack-dashboard/manage.py collectstatic --noinput --clear -v0 (code=exited, status=1/FAILURE) Nov 30 09:45:46 openstack-cntr1 python[12855]: File "/usr/share/openstack-dashboard/openstack_dashboard/settings.py", line 376, in <module> Nov 30 09:45:46 openstack-cntr1 python[12855]: from local.local_settings import * # noqa: F403,H303 Nov 30 09:45:46 openstack-cntr1 python[12855]: File "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py", line 399 Nov 30 09:45:46 openstack-cntr1 python[12855]: 'supported_vnic_types': ['*'], Nov 30 09:45:46 openstack-cntr1 python[12855]: ^ Nov 30 09:45:46 openstack-cntr1 python[12855]: IndentationError: unexpected indent Nov 30 09:45:46 openstack-cntr1 systemd[1]: httpd.service: control process exited, code=exited status=1 Nov 30 09:45:46 openstack-cntr1 systemd[1]: Failed to start The Apache HTTP Server. Nov 30 09:45:46 openstack-cntr1 systemd[1]: Unit httpd.service entered failed state. Nov 30 09:45:46 openstack-cntr1 systemd[1]: httpd.service failed. [root@openstack-cntr1 ~]#

3 2

[requirements][stable] Capping requirements in stable branches
by Akihiro Motoki 30 Nov '19

30 Nov '19

Hi, I have a question on version capping in requirements files in stable branches. When some newer version of dependent library does not work with my project, do we accept a patch to add version capping of the library in stable branches? For example, the horizon team received a patch to novaclient to <16 [1]. novaclient 16.0.0 was released after Train release, so there is no surprise that we don't have the version cap novaclient<16 in Train release. My understanding is that we don't usually cap versions of libraries after the release, but I am sending this mail to check our general guideline. Thanks, Akihiro Motoki (irc: amotoki) [1] https://review.opendev.org/#/c/693000/

4 6

[neutron] security group api slow at scale
by Erik Olof Gunnar Andersson 29 Nov '19

29 Nov '19

I was wondering if there has been any work since Rocky to improve the performance of CRUD operations for security groups. When you have many thousands of rules the performance of simple operations gets exponentially slower. It got a little better after we applied the patches from this bug. https://bugs.launchpad.net/neutron/+bug/1810563 But in one of our environments getting security group rule lists, even when there are only 6-8 rules in that group takes almost 60 seconds. Best Regards, Erik Olof Gunnar Andersson

1 0

[horizon][stein] issue
by Ignazio Cassano 29 Nov '19

29 Nov '19

Hello, I just installed stein on centos 7 and when I have a log of instances clicking "next" on horizon the following messages appears in httpd error log (please help me): [Fri Nov 29 10:52:47.306231 2019] [:error] [pid 27395] INFO openstack_auth.forms Login successful for user "admin" using domain "default", remote address 10.102.184.193. [Fri Nov 29 10:53:04.594482 2019] [:error] [pid 27395] ERROR django.request Internal Server Error: /dashboard/project/instances/ [Fri Nov 29 10:53:04.594503 2019] [:error] [pid 27395] Traceback (most recent call last): [Fri Nov 29 10:53:04.594507 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner [Fri Nov 29 10:53:04.594510 2019] [:error] [pid 27395] response = get_response(request) [Fri Nov 29 10:53:04.594513 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response [Fri Nov 29 10:53:04.594515 2019] [:error] [pid 27395] response = self.process_exception_by_middleware(e, request) [Fri Nov 29 10:53:04.594518 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response [Fri Nov 29 10:53:04.594521 2019] [:error] [pid 27395] response = wrapped_callback(request, *callback_args, **callback_kwargs) [Fri Nov 29 10:53:04.594523 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec [Fri Nov 29 10:53:04.594526 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594538 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 52, in dec [Fri Nov 29 10:53:04.594561 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594563 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 36, in dec [Fri Nov 29 10:53:04.594566 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594568 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 113, in dec [Fri Nov 29 10:53:04.594586 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594589 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 84, in dec [Fri Nov 29 10:53:04.594591 2019] [:error] [pid 27395] return view_func(request, *args, **kwargs) [Fri Nov 29 10:53:04.594594 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view [Fri Nov 29 10:53:04.594597 2019] [:error] [pid 27395] return self.dispatch(request, *args, **kwargs) [Fri Nov 29 10:53:04.594600 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch [Fri Nov 29 10:53:04.594603 2019] [:error] [pid 27395] return handler(request, *args, **kwargs) [Fri Nov 29 10:53:04.594605 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 223, in get [Fri Nov 29 10:53:04.594608 2019] [:error] [pid 27395] handled = self.construct_tables() [Fri Nov 29 10:53:04.594611 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 214, in construct_tables [Fri Nov 29 10:53:04.594613 2019] [:error] [pid 27395] handled = self.handle_table(table) [Fri Nov 29 10:53:04.594616 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 123, in handle_table [Fri Nov 29 10:53:04.594638 2019] [:error] [pid 27395] File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 252, in _get_data_dict [Fri Nov 29 10:53:04.594641 2019] [:error] [pid 27395] self._data = {self.table_class._meta.name: self.get_data()} [Fri Nov 29 10:53:04.594644 2019] [:error] [pid 27395] File "/usr/share/openstack-dashboard/openstack_dashboard/dashboards/project/instances/views.py", line 186, in get_data [Fri Nov 29 10:53:04.594661 2019] [:error] [pid 27395] boot_volume.volume_image_metadata['image_id'] in [Fri Nov 29 10:53:04.594664 2019] [:error] [pid 27395] KeyError: 'image_id

2 3