- openstack-discuss - lists.openstack.org

[OCTAVIA] How to monitor amphora
by Gaël THEROND 29 Nov '18

29 Nov '18

Hi guys, As already discussed I had to test Octavia as our corporate LoadBalancer solution and it was a success. Thank to everyone on this list that assisted me and especially Michael Octavia is fully working and without weird nightmare glitches. Now that I validated the technical part of my project, I need to enter more operationals questions such as what’s the best way to monitor and log amphora? I would like to be able to get a monitoring and logging agent installed on the amphora in order to get proper metrics about what’s going on with the LoadBalancer, is it fully cpu loaded? Is it using all network resources available, are the file descriptors near the initial limits? How much xxx HTTP return code the loadBalancer is facing and send those logs to an ELK or something similar. Do you know if that something that I could achieve by adding more elements to the image at DIB time or are there any other better best-practices that I should be aware of ? As Octavia is creating a namespace for each HAProxy process, I am wandering if it’s even possible. Thanks a lot for your hard work guys. Kind regards,

5 8

[loci] Project Technical Leadership Change
by Chris Hoge 29 Nov '18

29 Nov '18

After discussion with the current Loci PTL, team members, and the OpenStack TC at the Berlin Summit, I will be stepping in as interim PTL for the Loci project for the remainder of the development cycle. If you have any questions or concerns about this, please feel free to reach out to me directly. As a reminder, Loci has weekly team meetings on Friday at 1500 UTC in the #openstack-loci channel that we will be resuming this week. Thanks, Chris Hoge

2 3

[Openstack] [Nova] Instance creation problem
by Minjun Hong 29 Nov '18

29 Nov '18

Hi. I'm setting up a Openstack system on the servers of my laboratory. While I try to create an instance, a problem has occurred! Instance creation was failed and it seems that libvirt failed to attaching the vif to the instance. When I create a virtual machine by using virsh tool (libvirt) manually, there was no problem. I add the logs as follows: 1. controller node > "/var/log/nova/nova-conductor.log" > 2018-11-28 21:18:13.033 2657 ERROR nova.scheduler.utils > [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736 > 47270e4fb58045dc88b6f0f736286ffc - default default] [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Error from last host: node1 (node > node1): [u'Traceback (most recent call last):\n', u' File > "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1840, in > _do_build_and_run_instance\n filter_properties, request_spec)\n', u' > File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2120, > in _build_and_run_instance\n instance_uuid=instance.uuid, > reason=six.text_type(e))\n', u"RescheduledException: Build of instance > 9c2d08f3-0680-4709-a64d-ae1729a11304 was re-scheduled: internal error: > libxenlight failed to create new domain 'instance-00000008'\n"] > 2018-11-28 21:18:13.033 2657 WARNING nova.scheduler.utils > [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736 > 47270e4fb58045dc88b6f0f736286ffc - default default] Failed to > compute_task_build_instances: Exceeded maximum number of retries. Exceeded > max scheduling attempts 3 for instance > 9c2d08f3-0680-4709-a64d-ae1729a11304. Last exception: internal error: > libxenlight failed to create new domain 'instance-00000008': > MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max > scheduling attempts 3 for instance 9c2d08f3-0680-4709-a64d-ae1729a11304. > Last exception: internal error: libxenlight failed to create new domain > 'instance-00000008' > 2018-11-28 21:18:13.034 2657 WARNING nova.scheduler.utils > [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736 > 47270e4fb58045dc88b6f0f736286ffc - default default] [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Setting instance to ERROR state.: > MaxRetriesExceeded: Exceeded maximum number of retries. Exceeded max > scheduling attempts 3 for instance 9c2d08f3-0680-4709-a64d-ae1729a11304. > Last exception: internal error: libxenlight failed to create new domain > 'instance-00000008' > 2018-11-28 21:18:13.067 2657 WARNING oslo_config.cfg > [req-291fdb2d-fa94-461c-9f5f-68d340791c77 3367829d9c004653bdc9102443bd4736 > 47270e4fb58045dc88b6f0f736286ffc - default default] Option "url" from group > "neutron" is deprecated for removal (Endpoint lookup uses the service > catalog via common keystoneauth1 Adapter configuration options. In the > current release, "url" will override this behavior, but will be ignored > and/or removed in a future release. To achieve the same result, use the > endpoint_override option instead.). Its value may be silently ignored in > the future. > "/var/log/neutron/neutron-linuxbridge-agent.log" > 2018-11-28 17:41:45.593 2476 INFO > neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] > Interface mappings: {'provider': 'enp1s0f1'} > 2018-11-28 17:41:45.593 2476 INFO > neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] > Bridge mappings: {} > 2018-11-28 17:41:45.624 2476 INFO > neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] > Agent initialized successfully, now running... > 2018-11-28 17:41:45.901 2476 INFO > neutron.plugins.ml2.drivers.agent._common_agent > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] RPC agent_id: > lba0369fa2714a > 2018-11-28 17:41:45.907 2476 INFO neutron.agent.agent_extensions_manager > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Loaded agent > extensions: [] > 2018-11-28 17:41:46.121 2476 INFO > neutron.plugins.ml2.drivers.agent._common_agent > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Linux bridge agent > Agent RPC Daemon Started! > 2018-11-28 17:41:46.122 2476 INFO > neutron.plugins.ml2.drivers.agent._common_agent > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Linux bridge agent > Agent out of sync with plugin! > 2018-11-28 17:41:46.512 2476 INFO > neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Clearing orphaned ARP > spoofing entries for devices [] > 2018-11-28 17:41:47.020 2476 INFO > neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Clearing orphaned ARP > spoofing entries for devices [] > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent [-] Failed reporting > state!: MessagingTimeout: Timed out waiting for a reply to message ID > 20ef587240864120b878559ab821adbf > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call > last): > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", > line 128, in _report_state > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent True) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/neutron/agent/rpc.py", line 93, in > report_state > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent return method(context, > 'report_state', **kwargs) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 174, > in call > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent retry=self.retry) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 131, > in _send > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent timeout=timeout, > retry=retry) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > line 559, in send > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent retry=retry) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > line 548, in _send > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent result = > self._waiter.wait(msg_id, timeout) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > line 440, in wait > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent message = > self.waiters.get(msg_id, timeout=timeout) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent File > "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", > line 328, in get > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent 'to message ID %s' % > msg_id) > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent MessagingTimeout: Timed out > waiting for a reply to message ID 20ef587240864120b878559ab821adbf > 2018-11-28 17:42:45.981 2476 ERROR > neutron.plugins.ml2.drivers.agent._common_agent > 2018-11-28 17:42:45.986 2476 WARNING oslo.service.loopingcall [-] Function > 'neutron.plugins.ml2.drivers.agent._common_agent.CommonAgentLoop._report_state' > run outlasted interval by 30.07 sec > 2018-11-28 17:42:46.055 2476 INFO > neutron.plugins.ml2.drivers.agent._common_agent [-] Linux bridge agent > Agent has just been revived. Doing a full sync. > 2018-11-28 17:42:46.156 2476 INFO > neutron.plugins.ml2.drivers.agent._common_agent > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Linux bridge agent > Agent out of sync with plugin! > 2018-11-28 17:43:40.189 2476 INFO neutron.agent.securitygroups_rpc > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Preparing filters for > devices set(['tap4a09374a-7f']) > 2018-11-28 17:43:40.935 2476 INFO > neutron.plugins.ml2.drivers.agent._common_agent > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Port tap4a09374a-7f > updated. Details: {u'profile': {}, u'network_qos_policy_id': None, > u'qos_policy_id': None, u'allowed_address_pairs': [], u'admin_state_up': > True, u'network_id': u'87fa0d9c-5ed3-4332-8782-0d4139eed7f3', > u'segmentation_id': None, u'mtu': 1500, u'device_owner': u'network:dhcp', > u'physical_network': u'provider', u'mac_address': u'fa:16:3e:ab:0e:84', > u'device': u'tap4a09374a-7f', u'port_security_enabled': False, u'port_id': > u'4a09374a-7fa5-42c2-9430-67a0cd65336c', u'fixed_ips': [{u'subnet_id': > u'e95946a8-070c-42c4-877e-279e6e7acc7e', u'ip_address': u'192.0.10.4'}], > u'network_type': u'flat'} > 2018-11-28 17:43:41.124 2476 INFO > neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect > [req-c447894c-9013-4bec-82e4-8b501414184d - - - - -] Skipping ARP spoofing > rules for port 'tap4a09374a-7f' because it has port security disabled "/var/log/neutron/neutron-server.log" > 2018-11-28 17:30:02.130 15995 INFO neutron.pecan_wsgi.hooks.translation > [req-e7554d70-c84d-46d5-8ff6-a536fb35664c 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] GET failed (client > error): The resource could not be found. > 2018-11-28 17:30:02.130 15995 INFO neutron.wsgi > [req-e7554d70-c84d-46d5-8ff6-a536fb35664c 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] 10.150.21.183 "GET > /v2.0/floatingips?fixed_ip_address=192.0.10.10&port_id=8ab0d544-ec5b-4e69-95f4-1f06f7b53bb4 > HTTP/1.1" status: 404 len: 309 time: 0.0072770 > 2018-11-28 17:30:02.167 15995 INFO neutron.wsgi > [req-a2b5f53b-8992-4178-b0b6-55be9d1a0f32 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] 10.150.21.183 "GET > /v2.0/subnets?id=e95946a8-070c-42c4-877e-279e6e7acc7e HTTP/1.1" status: > 200 len: 822 time: 0.0341990 > 2018-11-28 17:30:02.199 15995 INFO neutron.wsgi > [req-9e1a51e9-9ccc-4226-a78f-0420ff95c147 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] 10.150.21.183 "GET > /v2.0/ports?network_id=87fa0d9c-5ed3-4332-8782-0d4139eed7f3&device_owner=network%3Adhcp > HTTP/1.1" status: 200 len: 1080 time: 0.0300300 > 2018-11-28 17:30:02.584 15995 INFO neutron.notifiers.nova [-] Nova event > response: {u'status': u'completed', u'tag': > u'8ab0d544-ec5b-4e69-95f4-1f06f7b53bb4', u'name': u'network-changed', > u'server_uuid': u'a9afc2d4-f4c9-429b-9773-4de8a3eaefa5', u'code': 200} > 2018-11-28 17:30:02.628 15995 INFO neutron.wsgi > [req-73265cf5-5f0d-4217-b716-caa2fb906abf 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] 10.150.21.183 "GET > /v2.0/ports?tenant_id=47270e4fb58045dc88b6f0f736286ffc&device_id=a9afc2d4-f4c9-429b-9773-4de8a3eaefa5 > HTTP/1.1" status: 200 len: 1062 time: 0.0316660 > 2018-11-28 17:30:02.696 15995 INFO neutron.wsgi > [req-ed53b92c-3033-4b4a-ade4-fdc5a3463e8c 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] 10.150.21.183 "GET > /v2.0/networks?id=87fa0d9c-5ed3-4332-8782-0d4139eed7f3 HTTP/1.1" status: > 200 len: 872 time: 0.0655539 > 2018-11-28 17:30:02.702 15995 WARNING neutron.pecan_wsgi.controllers.root > [req-ccd8c9b8-d2cf-40f7-b53b-5936bd0c9a6d 29a3a16fd2484ee9bed834a3835545af > 5ebe3484974848b182a381127cb35a22 - default default] No controller found > for: floatingips - returning response code 404: PecanNotFound 2. compute node > "/var/log/libvirt/libxl/libxl-driver.log" > 2018-11-28 08:40:31.920+0000: libxl: > libxl_event.c:681:libxl__ev_xswatch_deregister: remove watch for path > @releaseDomain: Bad file descriptor > 2018-11-28 09:57:01.707+0000: libxl: > libxl_exec.c:118:libxl_report_child_exitstatus: /etc/xen/scripts/vif-bridge > online [2536] exited with error status 1 > 2018-11-28 09:57:01.708+0000: libxl: > libxl_device.c:1286:device_hotplug_child_death_cb: script: ip link set > vif1.0 name tape5a239a8-6e failed > 2018-11-28 09:57:01.708+0000: libxl: > libxl_create.c:1522:domcreate_attach_devices: Domain 1:unable to add vif > devices "/var/log/xen/xen-hotplug.log" > RTNETLINK answers: Device or resource busy "/var/log/nova/nova-compute.log" > : libvirtError: internal error: libxenlight failed to create new domain > 'instance-00000008' > 2018-11-28 21:18:11.350 2384 ERROR nova.virt.libvirt.driver > [req-b3e761b7-00fa-4930-bd9e-4330f8440c03 48086750fa13420888601964bb6a9d0d > 5ebe3484974848b182a381127cb35a22 - default default] [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Failed to start libvirt guest: > libvirtError: internal error: libxenlight failed to create new domain > 'instance-00000008' > 2018-11-28 21:18:11.352 2384 INFO os_vif > [req-b3e761b7-00fa-4930-bd9e-4330f8440c03 48086750fa13420888601964bb6a9d0d > 5ebe3484974848b182a381127cb35a22 - default default] Successfully unplugged > vif > VIFBridge(active=False,address=fa:16:3e:6b:e4:b7,bridge_name='brq87fa0d9c-5e',has_traffic_filtering=True,id=484807ca-8c7c-4509-a5f5-ed7e5fd2078f,network=Network(87fa0d9c-5ed3-4332-8782-0d4139eed7f3),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap484807ca-8c') > 2018-11-28 21:18:11.554 2384 INFO nova.virt.libvirt.driver > [req-b3e761b7-00fa-4930-bd9e-4330f8440c03 48086750fa13420888601964bb6a9d0d > 5ebe3484974848b182a381127cb35a22 - default default] [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Deleting instance files > /var/lib/nova/instances/9c2d08f3-0680-4709-a64d-ae1729a11304_del > 2018-11-28 21:18:11.556 2384 INFO nova.virt.libvirt.driver > [req-b3e761b7-00fa-4930-bd9e-4330f8440c03 48086750fa13420888601964bb6a9d0d > 5ebe3484974848b182a381127cb35a22 - default default] [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Deletion of > /var/lib/nova/instances/9c2d08f3-0680-4709-a64d-ae1729a11304_del complete > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager > [req-b3e761b7-00fa-4930-bd9e-4330f8440c03 48086750fa13420888601964bb6a9d0d > 5ebe3484974848b182a381127cb35a22 - default default] [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Instance failed to spawn: > libvirtError: internal error: libxenlight failed to create new domain > 'instance-00000008' > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] Traceback (most recent call last): > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2251, in > _build_resources > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] yield resources > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2031, in > _build_and_run_instance > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] > block_device_info=block_device_info) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3089, > in spawn > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] destroy_disks_on_failure=True) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5614, > in _create_domain_and_network > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] destroy_disks_on_failure) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in > __exit__ > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] self.force_reraise() > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in > force_reraise > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] six.reraise(self.type_, > self.value, self.tb) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5583, > in _create_domain_and_network > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] > post_xml_callback=post_xml_callback) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5502, > in _create_domain > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] guest.launch(pause=pause) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 144, in > launch > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] self._encoded_xml, > errors='ignore') > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in > __exit__ > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] self.force_reraise() > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in > force_reraise > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] six.reraise(self.type_, > self.value, self.tb) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 139, in > launch > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] return > self._domain.createWithFlags(flags) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] result = > proxy_call(self._autowrap, f, *args, **kwargs) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in > proxy_call > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] rv = execute(f, *args, **kwargs) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] six.reraise(c, e, tb) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] rv = meth(*args, **kwargs) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] File > "/usr/lib/python2.7/dist-packages/libvirt.py", line 1092, in createWithFlags > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] if ret == -1: raise libvirtError > ('virDomainCreateWithFlags() failed', dom=self) > 2018-11-28 21:18:11.614 2384 ERROR nova.compute.manager [instance: > 9c2d08f3-0680-4709-a64d-ae1729a11304] libvirtError: internal error: > libxenlight failed to create new domain 'instance-00000008' Anyone help me please. _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack(a)lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

2 1

[TripleO][Edge] Reduce base layer of containers for security and size of images (maintenance) sakes
by Bogdan Dobrelya 29 Nov '18

29 Nov '18

Changing the topic to follow the subject. [tl;dr] it's time to rearchitect container images to stop incluiding config-time only (puppet et al) bits, which are not needed runtime and pose security issues, like CVEs, to maintain daily. Background: 1) For the Distributed Compute Node edge case, there is potentially tens of thousands of a single-compute-node remote edge sites connected over WAN to a single control plane, which is having high latency, like a 100ms or so, and limited bandwith. Reducing the base layer size becomes a decent goal there. See the security background below. 2) For a generic security (Day 2, maintenance) case, when puppet/ruby/systemd/name-it gets a CVE fixed, the base layer has to be updated and all layers on top - to be rebuild, and all of those layers, to be re-fetched for cloud hosts and all containers to be restarted... And all of that because of some fixes that have nothing to OpenStack. By the remote edge sites as well, remember of "tens of thousands", high latency and limited bandwith?.. 3) TripleO CI updates (including puppet*) packages in containers, not in a common base layer of those. So each a CI job has to update puppet* and its dependencies - ruby/systemd as well. Reducing numbers of packages to update for each container makes sense for CI as well. Implementation related: WIP patches [0],[1] for early review, uses a config "pod" approach, does not require to maintain a two sets of config vs runtime images. Future work: a) cronie requires systemd, we'd want to fix that also off the base layer. b) rework to podman pods for docker-puppet.py instead of --volumes-from a side car container (can't be backported for Queens then, which is still nice to have a support for the Edge DCN case, at least downstream only perhaps). Some questions raised on IRC: Q: is having a service be able to configure itself really need to involve a separate pod? A: Highly likely yes, removing not-runtime things is a good idea and pods is an established PaaS paradigm already. That will require some changes in the architecture though (see the topic with WIP patches). Q: that's (fetching a config container) actually more data that about to download otherwise A: It's not, if thinking of Day 2, when have to re-fetch the base layer and top layers, when some unrelated to openstack CVEs got fixed there for ruby/puppet/systemd. Avoid the need to restart service containers because of those minor updates puched is also a nice thing. Q: the best solution here would be using packages on the host, generating the config files on the host. And then having an all-in-one container for all the services which lets them run in an isolated mannner. A: I think for Edge cases, that's a no go as we might want to consider tiny low footprint OS distros like former known Container Linux or Atomic. Also, an all-in-one container looks like an anti-pattern from the world of VMs. [0] https://review.openstack.org/#/q/topic:base-container-reduction [1] https://review.rdoproject.org/r/#/q/topic:base-container-reduction > Here is a related bug [1] and implementation [1] for that. PTAL folks! > > [0] https://bugs.launchpad.net/tripleo/+bug/1804822 > [1] https://review.openstack.org/#/q/topic:base-container-reduction > >> Let's also think of removing puppet-tripleo from the base container. >> It really brings the world-in (and yum updates in CI!) each job and each >> container! >> So if we did so, we should then either install puppet-tripleo and co on >> the host and bind-mount it for the docker-puppet deployment task steps >> (bad idea IMO), OR use the magical --volumes-from <a-side-car-container> >> option to mount volumes from some "puppet-config" sidecar container >> inside each of the containers being launched by docker-puppet tooling. > > On Wed, Oct 31, 2018 at 11:16 AM Harald Jensås <hjensas at redhat.com> > wrote: >> We add this to all images: >> >> https://github.com/openstack/tripleo-common/blob/d35af75b0d8c4683a677660646… >> >> /bin/sh -c yum -y install iproute iscsi-initiator-utils lvm2 python >> socat sudo which openstack-tripleo-common-container-base rsync cronie >> crudini openstack-selinux ansible python-shade puppet-tripleo python2- >> kubernetes && yum clean all && rm -rf /var/cache/yum 276 MB >> >> Is the additional 276 MB reasonable here? >> openstack-selinux <- This package run relabling, does that kind of >> touching the filesystem impact the size due to docker layers? >> >> Also: python2-kubernetes is a fairly large package (18007990) do we use >> that in every image? I don't see any tripleo related repos importing >> from that when searching on Hound? The original commit message[1] >> adding it states it is for future convenience. >> >> On my undercloud we have 101 images, if we are downloading every 18 MB >> per image thats almost 1.8 GB for a package we don't use? (I hope it's >> not like this? With docker layers, we only download that 276 MB >> transaction once? Or?) >> >> >> [1] https://review.openstack.org/527927 > > > > -- > Best regards, > Bogdan Dobrelya, > Irc #bogdando -- Best regards, Bogdan Dobrelya, Irc #bogdando

4 5

[Openstack] [OpenStack][cinder]Does cinder allocate disk to the volumes dynamically?
by Soheil Pourbafrani 29 Nov '18

29 Nov '18

Hi, I was wondering if the Cinder allocates disk to volumes statically or dynamically? _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack(a)lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

2 2

[tripleo] Let's improve upstream docs
by Marios Andreou 29 Nov '18

29 Nov '18

Hi folks, as just mentioned in the tripleo weekly irc meeting [1] some of us are trying to make small weekly improvements to the tripleo docs [2]. We are using this bug [3] for tracking and this effort is a result of some feedback during the recent Berlin summit. The general idea is 1 per week (or more if you can and want) - improvement/removal of stale content/identifying missing sections, or anything else you might care to propose. Please join us if you can, just add "Related-Bug: #1804642" to your commit message thanks [1] https://wiki.openstack.org/wiki/Meetings/TripleO [2] https://docs.openstack.org/tripleo-docs/latest/ [3] https://bugs.launchpad.net/tripleo/+bug/1804642

2 3

[kolla] Meeting 28-nov-2018 canceled
by Eduardo Gonzalez 28 Nov '18

28 Nov '18

Dear Kolla Team, Due to many people wont be able to attend today meeting, the Kolla weekly meeting on Wednesday 28th is canceled. Best regards

1 0

[dev] [nova] API updates week 18-48
by Ghanshyam Mann 28 Nov '18

28 Nov '18

Hi All, Please find the Nova API highlights of this week. Weekly Office Hour: =============== No Office hour (Alex was not online) - I did bug triage during that time. Triaged 5 bugs and updated the API bug report etherpad. Planned Features : ============== Below are the API related features for Stein. Ref - https://etherpad.openstack.org/p/stein-nova-subteam-tracking (feel free to add API item there if you are working or found any). NOTE: sequence order are not the priority, they are listed as per their start date. 1. Handling a down cell - https://blueprints.launchpad.net/nova/+spec/handling-down-cell - https://review.openstack.org/#/q/topic:bp/handling-down-cell+(status:open+O… - Weekly Progress: tssurya has progress on this and discussin with matt and melanie . 2. Servers Ips non-unique network names : - https://blueprints.launchpad.net/nova/+spec/servers-ips-non-unique-network-… - Spec Merged - https://review.openstack.org/#/q/topic:bp/servers-ips-non-unique-network-na… - Weekly Progress: No progress. I will take care of this after api policy improvement spec update. 3. Volume multiattach enhancements: - https://blueprints.launchpad.net/nova/+spec/volume-multiattach-enhancements - https://review.openstack.org/#/q/topic:bp/volume-multiattach-enhancements+(… - Weekly Progress: No progress. 4. Add API ref guideline for body text (takashin) - https://review.openstack.org/#/c/605628/ - Weekly Progress: It has one +2 and need one more +2. 5. Detach and attach boot volumes: - https://blueprints.launchpad.net/nova/+spec/detach-boot-volume - https://review.openstack.org/#/q/topic:bp/detach-boot-volume+(status:open+O… - Weekly Progress: No progress. Specs: 6. Nova API policy updates https://blueprints.launchpad.net/nova/+spec/granular-api-policy Spec: https://review.openstack.org/#/c/547850/ - Weekly Progress: No progress in this week. I will update the spec next week. 7. Nova API cleanup https://blueprints.launchpad.net/nova/+spec/api-consistency-cleanup Spec: https://review.openstack.org/#/c/603969/ - Weekly Progress: Updated the spec with review comment and added 2 more cleanup listed in etherpad 8. Support deleting data volume when destroy instance(Brin Zhang) - https://review.openstack.org/#/c/580336/ - Weekly Progress: No Progress this week. Waiting for spec review. COMPLETED: 1. API Extensions merge work - https://blueprints.launchpad.net/nova/+spec/api-extensions-merge-stein - https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic… - Weekly Progress: This was COMPLETED few weeks back itself. 2. Boot instance specific storage backend - https://blueprints.launchpad.net/nova/+spec/boot-instance-specific-storage-… - https://review.openstack.org/#/q/topic:bp/boot-instance-specific-storage-ba… - Weekly Progress: COMPLETED Bugs: ==== This week Bug Progress: https://etherpad.openstack.org/p/nova-api-weekly-bug-report Critical: 0->0 High importance: 1->1 By Status: New: 5->2 Confirmed/Triage: 32-> 32 In-progress: 36->35 Incomplete: 5->6 ===== Total: 78->75 NOTE- There might be some bug which are not tagged as 'api' or 'api-ref', those are not in above list. Tag such bugs so that we can keep our eyes. -gmann

1 0

[openstack-dev] [Cyborg] IRC meeting still happens at usual time this Wednesday
by Li Liu 28 Nov '18

28 Nov '18

Hi Team, I know there is a polling on the new IRC meeting time, but before we gather more inputs, let's keep using the old time slot for this week (10PM Beijing Time/9AM EST/6AM PST) Agenda for this week's meeting: 1. Status update on NOVA interaction spec 2. Status update on DB scheme spec 3. Status tracking on all the open patches -- Thank you Regards Li __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

1 0

Re: [Openstack] [PackStack][Cinder]Save Volumes in The Compute nodes
by Soheil Pourbafrani 28 Nov '18

28 Nov '18

Thanks! and Does it need to cinder module to be installed on compute nodes? On Sat, Nov 17, 2018 at 1:13 PM Bernd Bausch <berndbausch(a)gmail.com> wrote: > Yes. You need to enable Cinder’s InstanceLocalityFilter, see > > https://docs.openstack.org/cinder/latest/configuration/block-storage/schedu… > . > > Here a tip: > https://ask.openstack.org/en/question/92001/cinder-lvm-volume-local-to-inst… > > Bernd > > On Nov 17, 2018, at 8:38, Soheil Pourbafrani <soheil.ir08(a)gmail.com> > wrote: > > Hi, > I have 8 servers with just local HDD disk. I want to use one server as the > controller and network node and the other (7 servers) as the compute node. > > I was wondering if it's possible to install PackStack that every compute > node to store its volumes in it's HDD local disk? (I guess the Cinder > should be installed on every Compute node alongside other settings) > > Thanks > > _______________________________________________ > Mailing list: > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > Post to : openstack(a)lists.openstack.org > Unsubscribe : > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack(a)lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

6 8

[dev] [tc][all] TC office hours is started now on #openstack-tc
by Ghanshyam Mann 28 Nov '18

28 Nov '18

Hi All, TC office hour is started on #openstack-tc channel. Feel free to reach to us for anything you want discuss/input/feedback/help from TC. - TC

1 0

Any way to reply to archived messages?
by Ben Nemec 28 Nov '18

28 Nov '18

Let's say hypothetically someone procrastinated too long on the mailing list switchover and missed some messages on openstack-discuss, but now wants to reply to them. Is there any way to do that without breaking the threading in people's clients? Perhaps some way to request a message from the archive to be sent to me (err, this hypothetical person ;-). -Ben

2 2

[dev] Extracted placement repo is now used in devstack/grenade jobs on master
by Matt Riedemann 28 Nov '18

28 Nov '18

Now that [1] is merged, the extracted openstack/placement repo is being used in devstack and grenade testing for changes on master. This is part of a series of changes for the extraction [2] from nova but it's likely the one that will have the most impact (if any) to other projects on a daily basis because of our CI system. So this is just an FYI in case something goes off the rails in the near future, hit us in the mailing list or #openstack-placement (or #openstack-nova) IRC channels. [1] https://review.openstack.org/#/c/600162/ [2] https://etherpad.openstack.org/p/BER-placement-extract -- Thanks, Matt

1 0

[all][docs] Making the Contributor Portal More Useful Summary
by Kendall Nelson 28 Nov '18

28 Nov '18

Hello! If you’re interested in starting with the etherpad from this forum session, you can peruse that here[1]. The biggest part of the discussion here was that the page[2] is overloaded and most users don’t make it down the page to other information they might be looking for. It’s trying to be too many things at once and we need to simplify the pages. The plan is to split into one focused on contribution information and another focused more on general community info. If you have opinions on how the split should go and weren’t in the forum session, please let us know! Another idea that was discussed was some sort of wizard/decision tree to help people find the part of the portal they want based on 3ish questions: Know what project you’re looking for? (would open to project section of code and docs contribution type) Brand new to OpenStack? (would point to contributor guide or where we need help) Or Interested in a particular topic? (would open to project groups in code and docs contribution type) The rest of discussion centered on smaller changes to the contribution topics in the page. Adding tooltips, making certain things more obvious (like the think to the actual contributor guide), making sure Docs & i18n get included in the code & docs section since that pulls from the project navigator. If you’re interested in getting involved with this page, let me know! We don’t have this work currently tracked in Launchpad or StoryBoard because the page is maintained by the Foundation, but we are working to change that! -Kendall (diablo_rojo) [1]https://etherpad.openstack.org/p/BER-Contrib-Portal [2]https://www.openstack.org/community/

1 0

[scientific] IRC meeting today 2100 UTC - Handling controlled-access data
by Stig Telfer 28 Nov '18

28 Nov '18

Hi All - We have a Scientific SIG IRC meeting at 2100 UTC (about 45 minute’s time) in IRC channel #openstack-meeting. Everyone is welcome. Today we’d like to gather interested parties for reporting on best practice on handling sensitive data (for example, genomics or medical data, but not limited to that). We’d also like to talk on best practices for intensive AI & ML. The full agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_November_27th_20… Cheers, Stig

1 0

[openstack-dev] [pacemaker] Live migration with VirtualDomain resource-agent
by Murali Annamneni 28 Nov '18

28 Nov '18

Hi, I'm trying to setup pacemaker for live-migration when a compute node hosting the VM goes down. So, I created cluster resource for the compute-instance (demo1) with 'VirtualDomain' resource agent. >>> pcs resource create demo1 VirtualDomain migrateuri="qemu+tcp://<DestinationCompuNode>:16509/system" config="/etc/pacemaker/demo1.xml" migration_transport="tcp" meta allow-migrate="true" op start timeout="120s" op stop timeout="180s" And I manually shutdown the compute node hosting the vm, but, I don't see the migration is happening. Can someone please help me what I am missing here ? Thanks & Regards, Murali __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

1 0

Re: [Openstack-operators] IMPORTANT: We're combining the lists!
by Jeremy Stanley 28 Nov '18

28 Nov '18

REMINDER: The openstack, openstack-dev, openstack-sigs and openstack-operators mailing lists (to which this was sent) are being replaced by a new openstack-discuss(a)lists.openstack.org mailing list. The new list[0] has been open for posts from subscribers since Monday November 19, and the old lists will be configured to no longer accept posts starting on Monday December 3. In the interim, posts to the old lists will also get copied to the new list so it's safe to unsubscribe from them now and not miss any messages. See my previous notice[1] for details. As of the time of this announcement, we have 403 subscribers on openstack-discuss with six days to go before the old lists are closed down for good). I have updated the old list descriptions to indicate the openstack-discuss list is preferred, and added a custom "welcome message" with the same for anyone who subscribes to them over the next week. [0] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [1] http://lists.openstack.org/pipermail/openstack-dev/2018-September/134911.ht… -- Jeremy Stanley _______________________________________________ OpenStack-operators mailing list OpenStack-operators(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

1 0

Re: [Openstack-sigs] IMPORTANT: We're combining the lists!
by Jeremy Stanley 28 Nov '18

28 Nov '18

REMINDER: The openstack, openstack-dev, openstack-sigs and openstack-operators mailing lists (to which this was sent) are being replaced by a new openstack-discuss(a)lists.openstack.org mailing list. The new list[0] has been open for posts from subscribers since Monday November 19, and the old lists will be configured to no longer accept posts starting on Monday December 3. In the interim, posts to the old lists will also get copied to the new list so it's safe to unsubscribe from them now and not miss any messages. See my previous notice[1] for details. As of the time of this announcement, we have 403 subscribers on openstack-discuss with six days to go before the old lists are closed down for good). I have updated the old list descriptions to indicate the openstack-discuss list is preferred, and added a custom "welcome message" with the same for anyone who subscribes to them over the next week. [0] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [1] http://lists.openstack.org/pipermail/openstack-dev/2018-September/134911.ht… -- Jeremy Stanley _______________________________________________ openstack-sigs mailing list openstack-sigs(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-sigs

1 0

Re: [openstack-dev] IMPORTANT: We're combining the lists!
by Jeremy Stanley 28 Nov '18

28 Nov '18

REMINDER: The openstack, openstack-dev, openstack-sigs and openstack-operators mailing lists (to which this was sent) are being replaced by a new openstack-discuss(a)lists.openstack.org mailing list. The new list[0] has been open for posts from subscribers since Monday November 19, and the old lists will be configured to no longer accept posts starting on Monday December 3. In the interim, posts to the old lists will also get copied to the new list so it's safe to unsubscribe from them now and not miss any messages. See my previous notice[1] for details. As of the time of this announcement, we have 403 subscribers on openstack-discuss with six days to go before the old lists are closed down for good). I have updated the old list descriptions to indicate the openstack-discuss list is preferred, and added a custom "welcome message" with the same for anyone who subscribes to them over the next week. [0] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [1] http://lists.openstack.org/pipermail/openstack-dev/2018-September/134911.ht… -- Jeremy Stanley __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

1 0

Re: [Openstack] IMPORTANT: We're combining the lists!
by Jeremy Stanley 28 Nov '18

28 Nov '18

REMINDER: The openstack, openstack-dev, openstack-sigs and openstack-operators mailing lists (to which this was sent) are being replaced by a new openstack-discuss(a)lists.openstack.org mailing list. The new list[0] has been open for posts from subscribers since Monday November 19, and the old lists will be configured to no longer accept posts starting on Monday December 3. In the interim, posts to the old lists will also get copied to the new list so it's safe to unsubscribe from them now and not miss any messages. See my previous notice[1] for details. As of the time of this announcement, we have 403 subscribers on openstack-discuss with six days to go before the old lists are closed down for good). I have updated the old list descriptions to indicate the openstack-discuss list is preferred, and added a custom "welcome message" with the same for anyone who subscribes to them over the next week. [0] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [1] http://lists.openstack.org/pipermail/openstack-dev/2018-September/134911.ht… -- Jeremy Stanley _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack(a)lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

1 0

[Openstack-operators] Nova hypervisor uuid
by Ignazio Cassano 28 Nov '18

28 Nov '18

Hi All, Please anyone know where hypervisor uuid is retrived? Sometime updating kmv nodes with yum update it changes and in nova database 2 uuids are assigned to the same node. regards Ignazio _______________________________________________ OpenStack-operators mailing list OpenStack-operators(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

2 1

Re: [OCTAVIA] How to monitor amphora
by Bedyk, Witold 28 Nov '18

28 Nov '18

Hi everyone, Monasca is the service which offers centralized monitoring and logging. As Joseph has written, you can probably use the monasca-agent and configure the existing system[1] and process[2] checks. If you need Octavia specific metrics, you can instrument your code using monasca-statsd library [3]. Best greetings Witek [1] https://github.com/openstack/monasca-agent/blob/master/docs/Plugins.md#syst… [2] https://github.com/openstack/monasca-agent/blob/master/docs/Plugins.md#proc… [3] https://github.com/openstack/monasca-statsd

1 0

[dev] [tc][all] TC office hours is started now on #openstack-tc
by Ghanshyam Mann 27 Nov '18

27 Nov '18

Hi All, TC office hour is started on #openstack-tc channel. Feel free to reach to us for anything you want discuss/input/feedback/help from TC. - TC

1 0

Re: [OCTAVIA] How to monitor amphora
by Joseph Davis 27 Nov '18

27 Nov '18

Hello Sa Pham and others, Just to throw another option in to your discussion, the monasca-agent would also be able to monitor the cpu, memory, etc. on an amphora. Similar to how the Ceilometer agent -> Gnocchi publishing would work, you could then use Monasca services to track usage and set alarms. And Monasca handles storage in a time series database (Cassandra or InfluxDB). https://github.com/openstack/monasca-agent/blob/master/docs/Agent.md has some details, an you can always ask questions in #openstack-monasca. joseph Adam Harwell wrote: I would also note (though I'm guessing you've heard this from Michael already) that logging and metrics for internal amphora data (like Haproxy logs, agent logs, etc) is on our roadmap to officially expose via some mechanism. We discussed possible ways to do this at the last PTG, but we were just waiting on someone to have time to do a blueprint for review. If you'd like to help get support for this upstream, you're welcome (actually, highly encouraged!) to hop into our IRC channel #openstack-lbaas and we can even help you get started. Hope to see you there! --Adam Harwell (rm_work) On Mon, Nov 26, 2018, 10:19 Michael Johnson <johnsomor(a)gmail.com> wrote: > This is an area we have been talking about recently at the PTGs. > > Adding connection log offloading is on our short term road map. > Someone had signed up for that, but has pivoted to work on other > items. > Due to that I can't say if it will make it for Stein or not. If > someone is interested in picking this up, I can help point you to the > documentation of our discussions at the PTGs. If not, I would expect > it to land late in Stein or early in Train. > > As for other monitoring, CPU, etc. this is area we currently leave to > the operator, but would like to do some work on. As mentioned above > gnocchi appears to be a nice time-series database for metrics > collection. We would need to have some design discussions on what and > how to collect the data. As you mentioned, monitoring agents can be > added to the amphora image at build time if the operator would like to > add a local tool set. > > As with most parts of Octavia, we try to make things "plug-able" so > that operators have a choice of tools. Both the log offloading and > metrics should follow that design principle. > > Michael > > On Sun, Nov 25, 2018 at 10:49 PM Gaël THEROND<gael.therond(a)gmail.com> > wrote: >> Hi! >> >> I’ve taken some time to read your presentation and I have to say thank > you for your wonderful work ! It’s pretty much complete and with code > example which is really cool. >> It’s exactly what I was looking for. >> If I could just suggest one thing, it would be to give operators freedom > to choose which logging and which metrics clients they want to use. >> Doing so would avoid having to rely on gnocchi as you would be able to > choose the clients type at amphora build time and specify clients target at > runtime using the amphora-agent. >> On another hand, it might we’ll be tricky to do so as everything is > namespaced into the amphora instance. >> As suggested using gnocchi to store HAProxy logs may well be the easiest > short path actually, I’m wondering if it would be possible for searchlight > to use that logs at some point. >> Anyway, thanks for the awesome job. >> For now I’ll keep it simple and just use the API for basic status. >> >> Kind regards, >> G. >> >> Le lun. 26 nov. 2018 à 04:23, Sa Pham<saphi070(a)gmail.com> a écrit : >>> yes. I have discussed with the Octavia team. PTL Johnson has suggested > me to use gnocchi to store octavia amphora metric. >>> >>> On Wed, Nov 21, 2018 at 3:16 PM Gaël THEROND<gael.therond(a)gmail.com> > wrote: >>>> Hi! Thanks for this material, I’ll have a look at it. >>>> >>>> As our philosophy at work is to always push upstream all our > patchs/features or bug fixes, I won’t modify and keep the source code on > our own and if we need further features like that I think we will rather > push for a Blueprint with all required commits. >>>> Did you already discussed that topic with the Octavia team? >>>> >>>> Thanks a lot. >>>> >>>> Le mer. 21 nov. 2018 à 09:12, Sa Pham<saphi070(a)gmail.com> a écrit : >>>>> Hi, >>>>> >>>>> In Vietnam Openinfra Days 2018, I have a presentation about > monitoring and logging for Octavia Amphora. We have to customize octavia > source code to do this. I send you my presentation slide: > https://drive.google.com/file/d/1dHXExEKrHDg4Cf3D1fBeulLDW_G-txLr/view?usp=… >>>>> Best, >>>>> >>>>> On Wed, Nov 21, 2018 at 3:05 PM Gaël THEROND<gael.therond(a)gmail.com> > wrote: >>>>>> Hi guys, >>>>>> >>>>>> As already discussed I had to test Octavia as our corporate > LoadBalancer solution and it was a success. >>>>>> Thank to everyone on this list that assisted me and especially > Michael Octavia is fully working and without weird nightmare glitches. >>>>>> Now that I validated the technical part of my project, I need to > enter more operationals questions such as what’s the best way to monitor > and log amphora? >>>>>> I would like to be able to get a monitoring and logging agent > installed on the amphora in order to get proper metrics about what’s going > on with the LoadBalancer, is it fully cpu loaded? Is it using all network > resources available, are the file descriptors near the initial limits? How > much xxx HTTP return code the loadBalancer is facing and send those logs to > an ELK or something similar. >>>>>> Do you know if that something that I could achieve by adding more > elements to the image at DIB time or are there any other better > best-practices that I should be aware of ? >>>>>> As Octavia is creating a namespace for each HAProxy process, I am > wandering if it’s even possible. >>>>>> Thanks a lot for your hard work guys. >>>>>> >>>>>> Kind regards, >>>>>> >>>>> -- >>>>> Sa Pham Dang >>>>> Cloud RnD Team - VCCloud >>>>> Phone/Telegram: 0986.849.582 >>>>> Skype: great_bn >>>>> >>>>> >>> -- >>> Sa Pham Dang >>> Cloud RnD Team - VCCloud >>> Phone/Telegram: 0986.849.582 >>> Skype: great_bn >>> >>>

1 0

[openstack-dev] [cyborg] New time for Cyborg weekly IRC meetings
by Nadathur, Sundar 27 Nov '18

27 Nov '18

Hi, The current time for the weekly Cyborg IRC meeting is 1400 UTC, which is 6 am Pacific and 10pm China time. That is a bad time for most people in the call. Please vote in this doodle for what time you prefer. If you need more options, please respond in this thread. [1] https://doodle.com/poll/eqy3hp8hfqtf2qyn Thanks & Regards, Sundar __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

1 0