[openstack-dev] [Trove] Resource not found when creating db instances.
Matt Fischer
matt at mattfischer.com
Thu Jan 19 04:22:04 UTC 2017
Trove works fine with neutron. I would look deeper into your logs. Do you
have any errors about issues with Rabbit message timeouts? If so your guest
may have issues talking to Rabbit. That seems to be a common issue.
On Wed, Jan 18, 2017 at 8:59 PM, Amrith Kumar <amrith.kumar at gmail.com>
wrote:
> Sorry Wang Sen, why do you say Trove is not ready for Neutron"? It has
> worked with Neutron for some releases now.
>
> This does not appear to be at all related to Neutron.
>
> -amrith
>
> --
> amrith.kumar at gmail.com
> On Jan 18, 2017 10:56 PM, "Wang Sen" <wangsen at linux.vnet.ibm.com> wrote:
>
>> Hi all,
>>
>> I met the resource not found error when I'm creating a database
>> instance. The instance stays on build status and turns to error status
>> after timeout.
>>
>> I know trove is not ready for neuton. Is there a work around for this
>> issue ? Thanks in advance.
>>
>> Below is the detailed information.
>>
>> Error Log
>> =========
>>
>> /var/log/trove/trove-taskmanager.log:
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task [-] Error
>> during Manager.publish_exists_event
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task Traceback
>> (most recent call last):
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py", line
>> 220, in run_periodic_tasks
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> task(self, context)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/trove/taskmanager/manager.py", line
>> 429, in publish_exists_event
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> self.admin_context)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py",
>> line 178, in publish_exist_events
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> notifications = transformer()
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py",
>> line 271, in __call__
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> client=self.nova_client)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py",
>> line 40, in load_mgmt_instances
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> mgmt_servers = client.servers.list(search_opts={'all_tenants': 1})
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 835,
>> in list
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> "servers")
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task resp,
>> body = self.api.client.get(url)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 480, in get
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task return
>> self._cs_request(url, 'GET', **kwargs)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 436, in
>> _cs_request
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> self.authenticate()
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 619, in
>> authenticate
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> self._v2_auth(auth_url)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 684, in
>> _v2_auth
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task return
>> self._authenticate(url, body)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 697, in
>> _authenticate
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
>> **kwargs)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 431, in
>> _time_request
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task resp,
>> body = self.request(url, method, **kwargs)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
>> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 425, in
>> request
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task raise
>> exceptions.from_response(resp, body, url, method)
>> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task NotFound:
>> The resource could not be found. (HTTP 404)
>>
>> Openstack Cluster
>> =================
>>
>> openstack version: Neuton
>> trove version: 2.5.0
>> $ root at kvm-215:~# trove --version
>> 2.5.0
>> $ root at kvm-215:~# openstack --version
>> openstack 3.2.0
>>
>> Controller Node: ubuntu 16.04, 9.181.129.215
>> Compute Node: ubuntu 16.04, 9.181.129.213
>>
>> Manage network: 192.168.1.0/24
>> Provider network: 9.181.129.0/24
>>
>> endpoints:
>>
>> $ openstack endpoint list
>>
>> +----------------------------------+-----------+------------
>> --+--------------+---------+-----------+--------------------
>> --------------------------+
>> | ID | Region | Service Name | Service
>> Type | Enabled | Interface | URL |
>> +----------------------------------+-----------+------------
>> --+--------------+---------+-----------+--------------------
>> --------------------------+
>> | 08fa008d727643428800f10b31e32e3b | RegionOne | glance | image
>> | True | admin | http://controller:9292
>> |
>> | 26cd2828179e4ca58b4cda80127889c2 | RegionOne | cinder | volume
>> | True | admin | http://controller:8776/v1/%(tenant_id)s
>> |
>> | 3148ed89e7e241b287d5dc68960d8052 | RegionOne | cinderv2 |
>> volumev2 | True | admin | http://controller:8776/v2/%(te
>> nant_id)s |
>> | 42d5aac6c0a349c5a167280459b16549 | RegionOne | trove |
>> database | True | public | http://controller:8779/v1.0/%(
>> tenant_id)s |
>> | 4600804165954819944a8c0ff9fe5f7d | RegionOne | trove |
>> database | True | admin | http://controller:8779/v1.0/%(
>> tenant_id)s |
>> | 4b7e6ba4db9d406d9f008a5c71700d94 | RegionOne | nova | compute
>> | True | internal | http://controller:8774/v2.1/%(tenant_id)s
>> |
>> | 4c35a14f4c8a4224aede30f91af25647 | RegionOne | glance | image
>> | True | public | http://controller:9292
>> |
>> | 5a2ea322cadb4a4881db689015dfa182 | RegionOne | neutron | network
>> | True | public | http://controller:9696
>> |
>> | 688688d146ee4d0eb93da67d108dd6a8 | RegionOne | neutron | network
>> | True | internal | http://controller:9696
>> |
>> | 6db0991bd4e9439a87ed08e696dfbf35 | RegionOne | cinderv2 |
>> volumev2 | True | internal | http://controller:8776/v2/%(te
>> nant_id)s |
>> | 721c045a5acb44c8a67545bd880e05aa | RegionOne | cinder | volume
>> | True | internal | http://controller:8776/v1/%(tenant_id)s
>> |
>> | 74261ec81ed642b680b35a101902f9c2 | RegionOne | swift |
>> object-store | True | public | http://controller:8080/v1/AUTH
>> _%(tenant_id)s |
>> | 74f5a84c8bb344bda8f3dce5570001d4 | RegionOne | neutron | network
>> | True | admin | http://controller:9696
>> |
>> | 85a295d376584809ac2c8adde8c59afa | RegionOne | cinder | volume
>> | True | public | http://controller:8776/v1/%(tenant_id)s
>> |
>> | 886c00a92bbf4b6d9244427d4f7a0e8d | RegionOne | swift |
>> object-store | True | admin | http://controller:8080/v1
>> |
>> | 8ccce1c3736a4c0493ffee8f07f016b0 | RegionOne | keystone |
>> identity | True | admin | http://controller:35357/v3/
>> |
>> | b59590340879457cb6311d3044387635 | RegionOne | swift |
>> object-store | True | internal | http://controller:8080/v1/AUTH
>> _%(tenant_id)s |
>> | b80a4a3dda8344ac992644d2c8dc167b | RegionOne | trove |
>> database | True | internal | http://controller:8779/v1.0/%(
>> tenant_id)s |
>> | bcfc25d9ec6d4cbfb10746cbd3caf153 | RegionOne | keystone |
>> identity | True | internal | http://controller:35357/v3/
>> |
>> | bf565acdf1b64ea8a079a7e49f37331a | RegionOne | keystone |
>> identity | True | public | http://controller:5000/v3/
>> |
>> | dbcb99659b9f428aae909f77cb22bd1c | RegionOne | cinderv2 |
>> volumev2 | True | public | http://controller:8776/v2/%(te
>> nant_id)s |
>> | de93f0d04ffa43b0bdc93da396044a06 | RegionOne | nova | compute
>> | True | admin | http://controller:8774/v2.1/%(tenant_id)s
>> |
>> | f9e8afbd6fe747619fd325982930e782 | RegionOne | glance | image
>> | True | internal | http://controller:9292
>> |
>> | fb13a0896e724ac4b972531a3cc46853 | RegionOne | nova | compute
>> | True | public | http://controller:8774/v2.1/%(tenant_id)s
>> |
>> +----------------------------------+-----------+------------
>> --+--------------+---------+-----------+--------------------
>> --------------------------+
>>
>> Trove Configuration
>> ===================
>>
>> trove.conf:
>> [DEFAULT]
>> debug = False
>> auth_strategy = keystone
>> add_addresses = True
>> bind_host = 0.0.0.0
>> bind_port = 8779
>> rpc_backend=rabbit
>> control_exchange = trove
>> db_api_implementation = "trove.db.sqlalchemy.api"
>> trove_auth_url = http://controller:5000/v3
>> nova_compute_url = http://controller:8774/v2.1
>> cinder_url = http://controller:8776/v1
>> swift_url = http://controller:8080/v1/AUTH_
>> notifier_queue_hostname = controller
>> os_region_name = RegionOne
>> network_label_regex = .*
>> trove_volume_support = True
>> block_device_mapping = vdb
>> device_path = /dev/vdb
>> max_accepted_volume_size = 10
>> max_instances_per_tenant = 5
>> max_volumes_per_tenant = 100
>> max_backups_per_tenant = 5
>> volume_time_out=30
>> http_get_rate = 200
>> http_post_rate = 200
>> http_put_rate = 200
>> http_delete_rate = 200
>> http_mgmt_post_rate = 200
>> trove_dns_support = False
>> dns_account_id = 123456
>> dns_auth_url = http://127.0.0.1:5000/v2.0
>> dns_username = user
>> dns_passkey = password
>> dns_ttl = 3600
>> dns_domain_name = 'trove.com.'
>> dns_domain_id = 11111111-1111-1111-1111-111111111111
>> dns_driver = trove.dns.designate.driver.DesignateDriver
>> dns_instance_entry_factory = trove.dns.designate.driver.Des
>> ignateInstanceEntryFactory
>> dns_endpoint_url = http://127.0.0.1/v1/
>> dns_service_type = dns
>> network_driver = trove.network.nova.NovaNetwork
>> default_neutron_networks =
>> taskmanager_queue = taskmanager
>> admin_roles = admin
>> agent_heartbeat_time = 10
>> agent_call_low_timeout = 5
>> agent_call_high_timeout = 150
>> reboot_time_out = 60
>> api_paste_config = /etc/trove/api-paste.ini
>> log_dir = /var/log/trove
>> [database]
>> connection = mysql+pymysql://trove:pass4trove@controller/trove
>> idle_timeout = 3600
>> [profiler]
>> [ssl]
>> [t_host=controller
>> rabbit_userid=openstack
>> rabbit_password=RABBITPASS
>> [mysql]
>> root_on_create = False
>> tcp_ports = 3306
>> volume_support = True
>> device_path = /dev/vdb
>> ignore_users = os_admin, root
>> ignore_dbs = mysql, information_schema, performance_schema
>> [redis]
>> tcp_ports = 6379
>> volume_support = False
>> [cassandra]
>> tcp_ports = 7000, 7001, 9042, 9160
>> volume_support = True
>> device_path = /dev/vdb
>> [couchbase]
>> tcp_ports = 8091, 8092, 4369, 11209-11211, 21100-21199
>> volume_support = True
>> device_path = /dev/vdb
>> [mongodb]
>> tcp_ports = 2500, 27017, 27019
>> volume_support = True
>> device_path = /dev/vdb
>> num_config_servers_per_cluster = 1
>> num_query_routers_per_cluster = 1
>> [vertica]
>> tcp_ports = 5433, 5434, 22, 5444, 5450, 4803
>> udp_ports = 5433, 4803, 4804, 6453
>> volume_support = True
>> device_path = /dev/vdb
>> cluster_support = True
>> cluster_member_count = 3
>> api_strategy = trove.common.strategies.cluster.experimental.vertica.api.
>> VerticaAPIStrategy
>> [cors]
>> [cors.subdomain]
>> [keystone_authtoken]
>> auth_uri = http://controller:5000
>> auth_url = http://controller:35357
>> auth_type = password
>> project_domain_name = default
>> user_domain_name = default
>> project_name = service
>> username = trove
>> password = pass4trove
>> oslo_messaging_rabbit]
>>
>>
>> trove-taskmanager.conf:
>> [DEFAULT]
>> debug = False
>> update_status_on_fail = True
>> rpc_backend=rabbit
>> control_exchange = trove
>> db_api_implementation = trove.db.sqlalchemy.api
>> trove_auth_url = http://controller:5000/v3
>> nova_compute_url = http://controller:8774/v2.1
>> cinder_url = http://controller:8776/v1
>> swift_url = http://controller:8080/v1/AUTH_
>> notifier_queue_hostname = controller
>> os_region_name = RegionOne
>> trove_volume_support = True
>> block_device_mapping = vdb
>> device_path = /dev/vdb
>> mount_point = /var/lib/mysql
>> volume_time_out=30
>> server_delete_time_out=480
>> use_nova_server_config_drive = True
>> nova_proxy_admin_user = admin
>> nova_proxy_admin_pass = frobisher86
>> nova_proxy_admin_tenant_name = service
>> taskmanager_manager=trove.taskmanager.manager.Manager
>> exists_notification_transformer = trove.extensions.mgmt.instance
>> s.models.NovaNotificationTransformer
>> exists_notification_ticks = 30
>> notification_service_id = mysql:2f3ff068-2bfb-4f70-9a9d-a6bb65bc084b
>> trove_dns_support = False
>> dns_account_id = 123456
>> dns_auth_url = http://controller:5000/v2.0
>> dns_username = user
>> dns_passkey = password
>> dns_ttl = 3600
>> dns_domain_name = 'trove.com.'
>> dns_domain_id = 11111111-1111-1111-1111-111111111111
>> dns_driver = trove.dns.designate.driver.DesignateDriver
>> dns_instance_entry_factory = trove.dns.designate.driver.Des
>> ignateInstanceEntryFactory
>> dns_endpoint_url = http://controller/v1/
>> dns_service_type = dns
>> network_driver=trove.network.neutron.NeutronDriver
>> default_neutron_networks =
>> trove_security_groups_support = True
>> trove_security_group_rule_cidr = 0.0.0.0/0
>> agent_heartbeat_time = 10
>> agent_call_low_timeout = 5
>> agent_call_high_timeout = 150
>> agent_replication_snapshot_timeout = 36000
>> use_nova_server_volume = False
>> network_label_regex = .*
>> template_path = /etc/trove/templates/
>> pydev_debug = disabled
>> [database]
>> connection = mysql+pymysql://trove:pass4trove@controller/trove
>> idle_timeout = 3600
>> [profiler]
>> [oslo_messaging_rabbit]
>> rabbit_host=controller
>> rabbit_userid=openstack
>> rabbit_password=pass4rabbit
>> [mysql]
>> icmp = True
>> tcp_ports = 3306
>> volume_support = True
>> device_path = /dev/vdb
>> [redis]
>> tcp_ports = 6379
>> volume_support = False
>> [cassandra]
>> tcp_ports = 7000, 7001, 9042, 9160
>> volume_support = True
>> device_path = /dev/vdb
>> [couchbase]
>> tcp_ports = 8091, 8092, 4369, 11209-11211, 21100-21199
>> volume_support = True
>> device_path = /dev/vdb
>> [mongodb]
>> volume_support = True
>> device_path = /dev/vdb
>> [vertica]
>> tcp_ports = 5433, 5434, 22, 5444, 5450, 4803
>> udp_ports = 5433, 4803, 4804, 6453
>> volume_support = True
>> device_path = /dev/vdb
>> mount_point = /var/lib/vertica
>> taskmanager_strategy = trove.common.strategies.cluste
>> r.experimental.vertica.taskmanager.VerticaTaskManagerStrategy
>>
>>
>> trove-conductor.conf:
>> [DEFAULT]
>> debug = False
>> trove_auth_url = http://controller:5000/v3
>> notifier_queue_hostname = controller
>> log_dir = /var/log/trove
>> conductor_manager = trove.conductor.manager.Manager
>> rpc_backend=rabbit
>> control_exchange = trove
>> [profiler]
>> [database]
>> connection = mysql+pymysql://trove:pass4trove@controller/trove
>> [oslo_messaging_rabbit]
>> rabbit_host=controller
>> rabbit_userid=openstack
>> rabbit_password=RABBITPASS
>>
>>
>> trove-guestagent.conf:
>> [DEFAULT]
>> debug = True
>> log_file = trove-guestagent.log
>> log_dir = /var/log/trove
>> ignore_users = os_admin
>> control_exchange = trove
>>
>> rabbit_host = controller
>> rabbit_userid = openstack
>> rabbit_password = RABBITPASS
>> nova_proxy_admin_user = admin
>> nova_proxy_admin_pass = ADMIN_PASS
>> nova_proxy_admin_tenant_name = service
>> trove_auth_url = http://controller:35357/v3
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170118/5ec04b16/attachment.html>
More information about the OpenStack-dev
mailing list