April 2019 - openstack-discuss - lists.openstack.org

[ptg][sig][upgrades] Train PTG Upgrades SIG session
by James Page 01 May '19

01 May '19

Hi All I've finally found time to create an etherpad for the Upgrades SIG session at the upcoming PTG in Denver (on the train on my way to LHR to catch my flight). https://etherpad.openstack.org/p/upgrade-sig-ptg-train I've added a few proposed topics but if you're at the PTG (or summit) and have anything upgrade related to discuss please add your topic to the etherpad over the next few days - I'll then put together a rough schedule for our half day of upgrades discussion on Thursday morning in room 201. IRC meetings never really got restarted since the last PTG but I know that the promise of getting together to discuss upgrade successes and challenges generally appeals to us all based on prior sessions! Thanks in advance and see you all in Denver! Cheers James

1 1

[Ceilometer]: cpu_util meter not being calculated as expected leading to delay in scaling
by Arvind Kumar 01 May '19

01 May '19

Hi, A design issue is observed in ceilometer service of Openstack. Setup include multiple compute nodes and 3 controller nodes. Meters from each compute node are sent to all the 3 ceilometer instances via RabbitMQ in round robin fashion at an interval of 10 min. After transformation of cumulative cpu meter data, cpu_util is generated by ceilometer instance at controller node and is published to the http address configured in ceilometer pipeline configuration. cpu_util is used by the application to take the decision if scaling of VM needs to be triggered or not. Ceilometer instance calculates cpu_util for a VM from the difference between cumulative cpu usage of VM at two timestamp divided by the timestamp difference. Let's say 1 compute node send the cumulative cpu usage of a VM (C1, C2, C3, C4) at timestamp T1, T2, T3, T4 (difference between any two timestamp is 10 min). Now (C1,T1) & (C4,T4) tuple is received by ceilometer instance 1, (C2,T2) by instance 2, (C3,T3) by instance 3. Here even if CPU usage of VM is increased between T1 & T2, cpu_util is calculated for 30 min duration (T1 & T4) rather than as expected for 10 min. This leads to scaling getting triggered after T4 that too when CPU usage is consistently above the threshold between T1 and T4. Please suggest how could this issue could be resolved. Do we have any solution to bind VM or compute node meter data to specific ceilometer instance for processing? Regards, Arvind. "Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s). are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email. and delete this message and any attachments from your system. Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."

2 2

[Zun] openstack appcontainer create Error
by Alejandro Ruiz Bermejo 01 May '19

01 May '19

Hi, i'm installing Zun in Openstack Queens with Ubuntu 18.04.1 LTS, i already have configured docker and kuyr-libnetwork. I'm following the guide at https://docs.openstack.org/zun/queens/install/index.html. I followed all the steps of the installation at controller node and everything resulted without problems. After finished the installation direction at compute node the *systemctl status zun-compute* have the following errors root@compute /h/team# systemctl status zun-compute ● zun-compute.service - OpenStack Container Service Compute Agent Loaded: loaded (/etc/systemd/system/zun-compute.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2019-04-30 16:46:56 UTC; 4h 26min ago Main PID: 2072 (zun-compute) Tasks: 1 (limit: 4915) CGroup: /system.slice/zun-compute.service └─2072 /usr/bin/python /usr/local/bin/zun-compute Apr 30 16:46:56 compute systemd[1]: Started OpenStack Container Service Compute Agent. Apr 30 16:46:57 compute zun-compute[2072]: 2019-04-30 16:46:57.929 2072 INFO zun.cmd.compute [-] Starting server in PID 2072 Apr 30 16:46:57 compute zun-compute[2072]: 2019-04-30 16:46:57.941 2072 INFO zun.container.driver [-] Loading container driver 'docker.driver.DockerDriver' Apr 30 16:46:58 compute zun-compute[2072]: 2019-04-30 16:46:58.028 2072 INFO zun.container.driver [-] Loading container driver 'docker.driver.DockerDriver' Apr 30 16:48:33 compute zun-compute[2072]: 2019-04-30 16:48:33.645 2072 INFO zun.image.driver [req-7e0b8325-1e09-4410-80f4-af807cbc0420 a16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 default - -] Loading container image driver 'glance' Apr 30 16:48:33 compute zun-compute[2072]: 2019-04-30 16:48:33.911 2072 INFO zun.image.driver [req-7e0b8325-1e09-4410-80f4-af807cbc0420 a16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 default - -] Loading container image driver 'glance' Apr 30 16:48:35 compute zun-compute[2072]: 2019-04-30 16:48:35.455 2072 INFO zun.image.driver [req-7e0b8325-1e09-4410-80f4-af807cbc0420 16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 default - -] Loading container image driver 'glance' Apr 30 16:48:35 compute zun-compute[2072]: 2019-04-30 16:48:35.939 2072 ERROR zun.image.glance.driver [req-7e0b8325-1e09-4410-80f4-af807cbc0420 a16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 default - -] Imae cirros was not found in glance: ImageNotFound: Image cirros could not be found. Apr 30 16:48:35 compute zun-compute[2072]: 2019-04-30 16:48:35.940 2072 INFO zun.image.driver [req-7e0b8325-1e09-4410-80f4-af807cbc0420 a16c6ef0319b4643a4ec8e56a1d025cb 59065d8f970b467aa94ef7b35f1edab5 default - -] Loading container image driver 'docker' Apr 30 16:48:55 compute zun-compute[2072]: 2019-04-30 16:48:55.011 2072 ERROR zun.compute.manager [req-7bfa764a-45b8-4e2f-ac70-84d8bb71b135 - - - - -] Error occurred while calling Docker create API: Docker internal error: 500 Server Error: Internal Server Error ("failed to update store for object typpe *libnetwork.endpointCnt: client: etcd member http://controller:2379 has no leader").: DockerError: Docker internal error: 500 Server Error: Internal Server Error ("failed to update store for object type *libnetwork.endpointtCnt: client: etcd member http://controller:2379 has no leader"). Also *systemctl status docker* show the next output root@compute /h/team# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/docker.service.d └─docker.conf, http-proxy.conf, https-proxy.conf Active: active (running) since Tue 2019-04-30 16:46:25 UTC; 4h 18min ago Docs: https://docs.docker.com Main PID: 1777 (dockerd) Tasks: 21 CGroup: /system.slice/docker.service └─1777 /usr/bin/dockerd --group zun -H tcp://compute:2375 -H unix:///var/run/docker.sock --cluster-store etcd://controller:2379 Apr 30 16:46:20 compute dockerd[1777]: time="2019-04-30T16:46:20.815305836Z" level=warning msg="Your kernel does not support cgroup rt runtime" Apr 30 16:46:20 compute dockerd[1777]: time="2019-04-30T16:46:20.815933695Z" level=info msg="Loading containers: start." Apr 30 16:46:24 compute dockerd[1777]: time="2019-04-30T16:46:24.378526837Z" level=info msg="Default bridge (docker0) is assigned with an IP address 17 Apr 30 16:46:24 compute dockerd[1777]: time="2019-04-30T16:46:24.572558877Z" level=info msg="Loading containers: done." Apr 30 16:46:25 compute dockerd[1777]: time="2019-04-30T16:46:25.198101219Z" level=info msg="Docker daemon" commit=e8ff056 graphdriver(s)=overlay2 vers Apr 30 16:46:25 compute dockerd[1777]: time="2019-04-30T16:46:25.198211373Z" level=info msg="Daemon has completed initialization" Apr 30 16:46:25 compute dockerd[1777]: time="2019-04-30T16:46:25.232286069Z" level=info msg="API listen on /var/run/docker.sock" Apr 30 16:46:25 compute dockerd[1777]: time="2019-04-30T16:46:25.232318790Z" level=info msg="API listen on 10.8.9.58:2375" Apr 30 16:46:25 compute systemd[1]: Started Docker Application Container Engine. Apr 30 16:48:55 compute dockerd[1777]: time="2019-04-30T16:48:55.009820439Z" level=error msg="Handler for POST /v1.26/networks/create returned error: failed to update store for object type *libnetwork.endpointCnt: client: etcd member http://controller:2379 has no leader" When i try to launch an app container as the guide says it shows an Error state and when i run opentack appcontainer show this is the reason of the error status_reason | Docker internal error: 500 Server Error: Internal Server Error ("failed to update store for object type *libnetwork.endpointCnt: client: etcd member http://controller:2379 has no leader") I had some troubles installing Kuryr-libnetwork besides that i didn't had any othet problem during the installation of Zun

2 1

[openstack-dev] [neutron] Neutron team dinner at Denver PTG
by Miguel Lavalle 30 Apr '19

30 Apr '19

Hi Neutrinos, After extensive consultations with the community, we decided that pizza and beer is always a crowd pleaser among a bunch of software engineers. So our team dinner will take place at Mellow Mushroom Pizza Bakers, located at 1201 16th Street, Denver, CO 80202, phone number (720) 328-9114. There is a reservation under "Miguel" at 7pm on Friday May 3rd. Here you will find walking directions from the Convention Center: https://goo.gl/maps/G63j6FmuoyBKjA7RA If you haven't done so already and are planning to attend the team dinner, please register your name at the top of this etherpad: https://etherpad.openstack.org/p/openstack-networking-train-ptg Looking forward to see you there!

1 0

["Neutron"] Create subnet gives ERROR "Body contains invalid data"
by Kelam, Koteswara Rao 30 Apr '19

30 Apr '19

Tried to create a subnet with following curl command $ curl -s $END_POINT/neutron/v2.0/subnets -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: $AUTH_TOKEN" -d '{'subnet': {'name': '8c7993a8-905e-4ad4-be3b-11c692e38317', 'network_id': '8f5f7cff-a320-40ba-9903-0c4e6dd80c0c', 'ip_version': 4, 'cidr': '169.254.3.1/32', 'enable_dhcp': 'false'}}' And got following ERROR: {"NeutronError": {"message": "Body contains invalid data", "type": "HTTPBadRequest", "detail": ""}} But when I tried with DOUBLE quotes in the body it got succeeded. $ curl -s $END_POINT/neutron/v2.0/subnets -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: $AUTH_TOKEN" -d '{"subnet": {"name": "8c7993a8-905e-4ad4-be3b-11c692e38317", "network_id": "8f5f7cff-a320-40ba-9903-0c4e6dd80c0c", "ip_version": 4, "cidr": "169.254.3.1/32", "enable_dhcp": 'false'}}' {"subnet":{"description":"","enable_dhcp":false,"tags":[],"network_id":"8f5f7cff-a320-40ba-9903-0c4e6dd80c0c","tenant_id":"39f81bc21a9f4eb7b87586d983b6ec3d","created_at":"2019-04-29T11:56:57Z","segment_id":null,"dns_nameservers":[],"updated_at":"2019-04-29T11:56:57Z","gateway_ip":"169.254.3.2","ipv6_ra_mode":null,"allocation_pools":[{"start":"169.254.3.1","end":"169.254.3.1"}],"host_routes":[],"revision_number":0,"ip_version":4,"ipv6_address_mode":null,"cidr":"169.254.3.1/32","project_id":"39f81bc21a9f4eb7b87586d983b6ec3d","id":"33de75fb-fa10-460a-9123-577ae62a771a","subnetpool_id":null,"name":"8c7993a8-905e-4ad4-be3b-11c692e38317"}} Single quotes not allowed in requested body? Only double quotes allowed?? Single quotes also used to work for me with older versions on neutron. -Koteswar

4 3

[ptg] [interop] [refstack] Interop Forum and PTG sessions
by Egle Sigler 30 Apr '19

30 Apr '19

Hello Everyone, If you in Denver for the summit this week, please join the interop working group team for our forum session on Tuesday, 4:20 pm – 5:00 pm meeting room 507: https://www.openstack.org/summit/denver-2019/summit-schedule/events/23657/i… PTG: Friday, 9:00 am – 10:45 am, Room 112 Thank you, Egle Sigler

1 0

[Zun] deploying zun error at kuryr installation
by Alejandro Ruiz Bermejo 30 Apr '19

30 Apr '19

Hi i've installed Openstack Queens on Ubuntu 18.04 and i'm following the Zun Installation Guide to deploy a container infrastructure. I'm following the steps to install kuryr-libnetwork (I've already installed docker on the compute node and it's working fine). The kuryr installation process had no errors but when i try to verify the operation as it's said in the guide docker network create --driver kuryr --ipam-driver kuryr \ --subnet 10.10.0.0/16 --gateway=10.10.0.1 test_net I'm Getting this error *Error* response from daemon: legacy plugin: Post http://10.8.9.54:23750/Plugin.Activate: dial tcp 10.8.9.54:23750: connect: connection refused I tryed to find the error wit **journalctl -u kuryr-libnetwork Apr 27 13:11:12 compute systemd[1]: Started Kuryr-libnetwork - Docker network plugin for Neutron. Apr 27 13:11:13 compute kuryr-server[10736]: 2019-04-27 13:11:13.273 10736 *WARNING* keystoneauth.identity.generic.base [-] Failed to discover available identity versions when contacting http://controller:35357. Attempting to parse version from URL.: ConnectFailure: Unable to establish connection to http://controller:35357: HTTPConnectionPool(host='controller', port=35357): MTTPConnectionPool(host='controller', port=35357): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fecef1a8090>: Failed to establish a new connection: [Errno 111] Connection refused',)) Apr 27 13:11:13 compute kuryr-server[10736]: 2019-04-27 13:11:13.274 10736 *CRITICAL* kuryr [-] Unhandled error: DiscoveryFailure: Could not determine a suitable URL for the plugin Apr 27 13:11:13 compute kuryr-server[10736]: 2019-04-27 13:11:13.274 10736 *ERROR* kuryr Traceback (most recent call last): This is the relevant content of my /etc/kuryr/kuryr.conf *[DEFAULT]* bindir = /usr/local/libexec/kuryr *[neutron]* www_authenticate_uri = http://controller:5000 auth_url = http://controller:35357 username = kuryr user_domain_name = default password = 123 project_name = service project_domain_name = default auth_type = password And this is the one at /usr/lib/docker/plugins/kuryr/kuryr.spec: http://conrtoller:23750 Btw compute node have connectivity with controller. I already have worked with the environment and launched kvm VMs on it Can someone help troubleshoot this error?? *UPDATE1* I checked open ports on controller node and 23750 isn't open. I followed the directions at the guide, any recommendation? Also in the compute node when i run systemctl status kuryr-libnetwork this is the output ● kuryr-libnetwork.service - Kuryr-libnetwork - Docker network plugin for Neutron Loaded: loaded (/etc/systemd/system/kuryr-libnetwork.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Sat 2019-04-27 16:06:21 UTC; 2min 39s ago Process: 17450 ExecStart=/usr/local/bin/kuryr-server --config-file /etc/kuryr/kuryr.conf (code=exited, status=1/FAILURE) Main PID: 17450 (code=exited, status=1/FAILURE) Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 134, in get_access Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr self.auth_ref = self.get_auth_ref(session) Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 199, in get_auth_ref Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr self._plugin = self._do_create_plugin(session) Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 194, in _do_create_plugin Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr raise exceptions.DiscoveryFailure('Could not determine a suiable URL ' Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr DiscoveryFailure: Could not determine a suitable URL for the plugin Apr 27 16:06:21 compute kuryr-server[17450]: 2019-04-27 16:06:21.497 17450 ERROR kuryr Apr 27 16:06:21 compute systemd[1]: kuryr-libnetwork.service: Main process exited, code=exited, status=1/FAILURE Apr 27 16:06:21 compute systemd[1]: kuryr-libnetwork.service: Failed with result 'exit-code'. *UPDATE2* I tried the recommendations given by **hongbin034** and confirm that i had a problem with Keystone. when i run openstack token issue after exporting the neutron environment varibles i get the following output Failed to discover available identity versions when contacting http://controller:35357. Attempting to parse version from URL. Could not determine a suitable URL for the plugin I checked and the port 35357 isn't even open at the controller node. But at the controller node installation there isn't any step configuring such port

2 1

[scientific-sig] ROOM CHANGE - SIG session at 3:50pm moved to 4E/4F
by Stig Telfer 30 Apr '19

30 Apr '19

Hi All - To get us a projector for our SIG session, we are moving from 507 to 4E/4F Sorry for the late notice! Cheers, Stig

1 0

[sahara][all] Sahara Train Virtual PTG is SCHEDULED
by Jeremy Freudberg 30 Apr '19

30 Apr '19

Hi all, Sahara will have a virtual PTG on Monday, May 6, beginning at 15:00 UTC. Best guess is that it will run two hours. Come and go as you need to. The etherpad is here: https://etherpad.openstack.org/p/sahara-train-ptg Please take a moment to add some topics, and, if you plan to attend, your name. The session will be hosted by Bluejeans: https://bluejeans.com/6304900378 (Take a moment to make sure your browser and peripherals are configured nicely) All interested parties are welcome. Outcomes will appear on openstack-discuss and on the etherpad itself. Till then, Jeremy

1 0

Third party CI and Zuul at OpenInfra Summit
by corvus＠inaugust.com 30 Apr '19

30 Apr '19

Hi, I'd like to invite any third-party CI operators as well as anyone else running Zuul or interested in running Zuul to join us in a discussion about reusing Zuul jobs in a downstream context at the Zuul working group session on Tuesday: https://www.openstack.org/summit/denver-2019/summit-schedule/events/23736/z… -Jim

1 0