[openstack-dev] [Trove] Resource not found when creating db instances.
Wang Sen
wangsen at linux.vnet.ibm.com
Thu Jan 19 07:43:26 UTC 2017
Hi Matt,
Thanks for your kindly reply. I mispelled Newton in last email.
I did not meet any problems with RabbitMQ, and the nova instances works well. From
the error log I pasted, it seems something goes wrong when novaclient is trying to
communicate with nova api.
On Wed, Jan 18, 2017 at 09:22:04PM -0700, Matt Fischer wrote:
> Trove works fine with neutron. I would look deeper into your logs. Do you
> have any errors about issues with Rabbit message timeouts? If so your guest
> may have issues talking to Rabbit. That seems to be a common issue.
>
> On Wed, Jan 18, 2017 at 8:59 PM, Amrith Kumar <amrith.kumar at gmail.com>
> wrote:
>
> > Sorry Wang Sen, why do you say Trove is not ready for Neutron"? It has
> > worked with Neutron for some releases now.
> >
> > This does not appear to be at all related to Neutron.
> >
> > -amrith
> >
> > --
> > amrith.kumar at gmail.com
> > On Jan 18, 2017 10:56 PM, "Wang Sen" <wangsen at linux.vnet.ibm.com> wrote:
> >
> >> Hi all,
> >>
> >> I met the resource not found error when I'm creating a database
> >> instance. The instance stays on build status and turns to error status
> >> after timeout.
> >>
> >> I know trove is not ready for neuton. Is there a work around for this
> >> issue ? Thanks in advance.
> >>
> >> Below is the detailed information.
> >>
> >> Error Log
> >> =========
> >>
> >> /var/log/trove/trove-taskmanager.log:
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task [-] Error
> >> during Manager.publish_exists_event
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task Traceback
> >> (most recent call last):
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/oslo_service/periodic_task.py", line
> >> 220, in run_periodic_tasks
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> task(self, context)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/trove/taskmanager/manager.py", line
> >> 429, in publish_exists_event
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> self.admin_context)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py",
> >> line 178, in publish_exist_events
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> notifications = transformer()
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py",
> >> line 271, in __call__
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> client=self.nova_client)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/trove/extensions/mgmt/instances/models.py",
> >> line 40, in load_mgmt_instances
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> mgmt_servers = client.servers.list(search_opts={'all_tenants': 1})
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 835,
> >> in list
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> "servers")
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 249, in _list
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task resp,
> >> body = self.api.client.get(url)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 480, in get
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task return
> >> self._cs_request(url, 'GET', **kwargs)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 436, in
> >> _cs_request
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> self.authenticate()
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 619, in
> >> authenticate
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> self._v2_auth(auth_url)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 684, in
> >> _v2_auth
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task return
> >> self._authenticate(url, body)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 697, in
> >> _authenticate
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task
> >> **kwargs)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 431, in
> >> _time_request
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task resp,
> >> body = self.request(url, method, **kwargs)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task File
> >> "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 425, in
> >> request
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task raise
> >> exceptions.from_response(resp, body, url, method)
> >> 2017-01-19 11:27:31.666 22795 ERROR oslo_service.periodic_task NotFound:
> >> The resource could not be found. (HTTP 404)
> >>
> >> Openstack Cluster
> >> =================
> >>
> >> openstack version: Neuton
> >> trove version: 2.5.0
> >> $ root at kvm-215:~# trove --version
> >> 2.5.0
> >> $ root at kvm-215:~# openstack --version
> >> openstack 3.2.0
> >>
> >> Controller Node: ubuntu 16.04, 9.181.129.215
> >> Compute Node: ubuntu 16.04, 9.181.129.213
> >>
> >> Manage network: 192.168.1.0/24
> >> Provider network: 9.181.129.0/24
> >>
> >> endpoints:
> >>
> >> $ openstack endpoint list
> >>
> >> +----------------------------------+-----------+------------
> >> --+--------------+---------+-----------+--------------------
> >> --------------------------+
> >> | ID | Region | Service Name | Service
> >> Type | Enabled | Interface | URL |
> >> +----------------------------------+-----------+------------
> >> --+--------------+---------+-----------+--------------------
> >> --------------------------+
> >> | 08fa008d727643428800f10b31e32e3b | RegionOne | glance | image
> >> | True | admin | http://controller:9292
> >> |
> >> | 26cd2828179e4ca58b4cda80127889c2 | RegionOne | cinder | volume
> >> | True | admin | http://controller:8776/v1/%(tenant_id)s
> >> |
> >> | 3148ed89e7e241b287d5dc68960d8052 | RegionOne | cinderv2 |
> >> volumev2 | True | admin | http://controller:8776/v2/%(te
> >> nant_id)s |
> >> | 42d5aac6c0a349c5a167280459b16549 | RegionOne | trove |
> >> database | True | public | http://controller:8779/v1.0/%(
> >> tenant_id)s |
> >> | 4600804165954819944a8c0ff9fe5f7d | RegionOne | trove |
> >> database | True | admin | http://controller:8779/v1.0/%(
> >> tenant_id)s |
> >> | 4b7e6ba4db9d406d9f008a5c71700d94 | RegionOne | nova | compute
> >> | True | internal | http://controller:8774/v2.1/%(tenant_id)s
> >> |
> >> | 4c35a14f4c8a4224aede30f91af25647 | RegionOne | glance | image
> >> | True | public | http://controller:9292
> >> |
> >> | 5a2ea322cadb4a4881db689015dfa182 | RegionOne | neutron | network
> >> | True | public | http://controller:9696
> >> |
> >> | 688688d146ee4d0eb93da67d108dd6a8 | RegionOne | neutron | network
> >> | True | internal | http://controller:9696
> >> |
> >> | 6db0991bd4e9439a87ed08e696dfbf35 | RegionOne | cinderv2 |
> >> volumev2 | True | internal | http://controller:8776/v2/%(te
> >> nant_id)s |
> >> | 721c045a5acb44c8a67545bd880e05aa | RegionOne | cinder | volume
> >> | True | internal | http://controller:8776/v1/%(tenant_id)s
> >> |
> >> | 74261ec81ed642b680b35a101902f9c2 | RegionOne | swift |
> >> object-store | True | public | http://controller:8080/v1/AUTH
> >> _%(tenant_id)s |
> >> | 74f5a84c8bb344bda8f3dce5570001d4 | RegionOne | neutron | network
> >> | True | admin | http://controller:9696
> >> |
> >> | 85a295d376584809ac2c8adde8c59afa | RegionOne | cinder | volume
> >> | True | public | http://controller:8776/v1/%(tenant_id)s
> >> |
> >> | 886c00a92bbf4b6d9244427d4f7a0e8d | RegionOne | swift |
> >> object-store | True | admin | http://controller:8080/v1
> >> |
> >> | 8ccce1c3736a4c0493ffee8f07f016b0 | RegionOne | keystone |
> >> identity | True | admin | http://controller:35357/v3/
> >> |
> >> | b59590340879457cb6311d3044387635 | RegionOne | swift |
> >> object-store | True | internal | http://controller:8080/v1/AUTH
> >> _%(tenant_id)s |
> >> | b80a4a3dda8344ac992644d2c8dc167b | RegionOne | trove |
> >> database | True | internal | http://controller:8779/v1.0/%(
> >> tenant_id)s |
> >> | bcfc25d9ec6d4cbfb10746cbd3caf153 | RegionOne | keystone |
> >> identity | True | internal | http://controller:35357/v3/
> >> |
> >> | bf565acdf1b64ea8a079a7e49f37331a | RegionOne | keystone |
> >> identity | True | public | http://controller:5000/v3/
> >> |
> >> | dbcb99659b9f428aae909f77cb22bd1c | RegionOne | cinderv2 |
> >> volumev2 | True | public | http://controller:8776/v2/%(te
> >> nant_id)s |
> >> | de93f0d04ffa43b0bdc93da396044a06 | RegionOne | nova | compute
> >> | True | admin | http://controller:8774/v2.1/%(tenant_id)s
> >> |
> >> | f9e8afbd6fe747619fd325982930e782 | RegionOne | glance | image
> >> | True | internal | http://controller:9292
> >> |
> >> | fb13a0896e724ac4b972531a3cc46853 | RegionOne | nova | compute
> >> | True | public | http://controller:8774/v2.1/%(tenant_id)s
> >> |
> >> +----------------------------------+-----------+------------
> >> --+--------------+---------+-----------+--------------------
> >> --------------------------+
> >>
> >> Trove Configuration
> >> ===================
> >>
> >> trove.conf:
> >> [DEFAULT]
> >> debug = False
> >> auth_strategy = keystone
> >> add_addresses = True
> >> bind_host = 0.0.0.0
> >> bind_port = 8779
> >> rpc_backend=rabbit
> >> control_exchange = trove
> >> db_api_implementation = "trove.db.sqlalchemy.api"
> >> trove_auth_url = http://controller:5000/v3
> >> nova_compute_url = http://controller:8774/v2.1
> >> cinder_url = http://controller:8776/v1
> >> swift_url = http://controller:8080/v1/AUTH_
> >> notifier_queue_hostname = controller
> >> os_region_name = RegionOne
> >> network_label_regex = .*
> >> trove_volume_support = True
> >> block_device_mapping = vdb
> >> device_path = /dev/vdb
> >> max_accepted_volume_size = 10
> >> max_instances_per_tenant = 5
> >> max_volumes_per_tenant = 100
> >> max_backups_per_tenant = 5
> >> volume_time_out=30
> >> http_get_rate = 200
> >> http_post_rate = 200
> >> http_put_rate = 200
> >> http_delete_rate = 200
> >> http_mgmt_post_rate = 200
> >> trove_dns_support = False
> >> dns_account_id = 123456
> >> dns_auth_url = http://127.0.0.1:5000/v2.0
> >> dns_username = user
> >> dns_passkey = password
> >> dns_ttl = 3600
> >> dns_domain_name = 'trove.com.'
> >> dns_domain_id = 11111111-1111-1111-1111-111111111111
> >> dns_driver = trove.dns.designate.driver.DesignateDriver
> >> dns_instance_entry_factory = trove.dns.designate.driver.Des
> >> ignateInstanceEntryFactory
> >> dns_endpoint_url = http://127.0.0.1/v1/
> >> dns_service_type = dns
> >> network_driver = trove.network.nova.NovaNetwork
> >> default_neutron_networks =
> >> taskmanager_queue = taskmanager
> >> admin_roles = admin
> >> agent_heartbeat_time = 10
> >> agent_call_low_timeout = 5
> >> agent_call_high_timeout = 150
> >> reboot_time_out = 60
> >> api_paste_config = /etc/trove/api-paste.ini
> >> log_dir = /var/log/trove
> >> [database]
> >> connection = mysql+pymysql://trove:pass4trove@controller/trove
> >> idle_timeout = 3600
> >> [profiler]
> >> [ssl]
> >> [t_host=controller
> >> rabbit_userid=openstack
> >> rabbit_password=RABBITPASS
> >> [mysql]
> >> root_on_create = False
> >> tcp_ports = 3306
> >> volume_support = True
> >> device_path = /dev/vdb
> >> ignore_users = os_admin, root
> >> ignore_dbs = mysql, information_schema, performance_schema
> >> [redis]
> >> tcp_ports = 6379
> >> volume_support = False
> >> [cassandra]
> >> tcp_ports = 7000, 7001, 9042, 9160
> >> volume_support = True
> >> device_path = /dev/vdb
> >> [couchbase]
> >> tcp_ports = 8091, 8092, 4369, 11209-11211, 21100-21199
> >> volume_support = True
> >> device_path = /dev/vdb
> >> [mongodb]
> >> tcp_ports = 2500, 27017, 27019
> >> volume_support = True
> >> device_path = /dev/vdb
> >> num_config_servers_per_cluster = 1
> >> num_query_routers_per_cluster = 1
> >> [vertica]
> >> tcp_ports = 5433, 5434, 22, 5444, 5450, 4803
> >> udp_ports = 5433, 4803, 4804, 6453
> >> volume_support = True
> >> device_path = /dev/vdb
> >> cluster_support = True
> >> cluster_member_count = 3
> >> api_strategy = trove.common.strategies.cluster.experimental.vertica.api.
> >> VerticaAPIStrategy
> >> [cors]
> >> [cors.subdomain]
> >> [keystone_authtoken]
> >> auth_uri = http://controller:5000
> >> auth_url = http://controller:35357
> >> auth_type = password
> >> project_domain_name = default
> >> user_domain_name = default
> >> project_name = service
> >> username = trove
> >> password = pass4trove
> >> oslo_messaging_rabbit]
> >>
> >>
> >> trove-taskmanager.conf:
> >> [DEFAULT]
> >> debug = False
> >> update_status_on_fail = True
> >> rpc_backend=rabbit
> >> control_exchange = trove
> >> db_api_implementation = trove.db.sqlalchemy.api
> >> trove_auth_url = http://controller:5000/v3
> >> nova_compute_url = http://controller:8774/v2.1
> >> cinder_url = http://controller:8776/v1
> >> swift_url = http://controller:8080/v1/AUTH_
> >> notifier_queue_hostname = controller
> >> os_region_name = RegionOne
> >> trove_volume_support = True
> >> block_device_mapping = vdb
> >> device_path = /dev/vdb
> >> mount_point = /var/lib/mysql
> >> volume_time_out=30
> >> server_delete_time_out=480
> >> use_nova_server_config_drive = True
> >> nova_proxy_admin_user = admin
> >> nova_proxy_admin_pass = frobisher86
> >> nova_proxy_admin_tenant_name = service
> >> taskmanager_manager=trove.taskmanager.manager.Manager
> >> exists_notification_transformer = trove.extensions.mgmt.instance
> >> s.models.NovaNotificationTransformer
> >> exists_notification_ticks = 30
> >> notification_service_id = mysql:2f3ff068-2bfb-4f70-9a9d-a6bb65bc084b
> >> trove_dns_support = False
> >> dns_account_id = 123456
> >> dns_auth_url = http://controller:5000/v2.0
> >> dns_username = user
> >> dns_passkey = password
> >> dns_ttl = 3600
> >> dns_domain_name = 'trove.com.'
> >> dns_domain_id = 11111111-1111-1111-1111-111111111111
> >> dns_driver = trove.dns.designate.driver.DesignateDriver
> >> dns_instance_entry_factory = trove.dns.designate.driver.Des
> >> ignateInstanceEntryFactory
> >> dns_endpoint_url = http://controller/v1/
> >> dns_service_type = dns
> >> network_driver=trove.network.neutron.NeutronDriver
> >> default_neutron_networks =
> >> trove_security_groups_support = True
> >> trove_security_group_rule_cidr = 0.0.0.0/0
> >> agent_heartbeat_time = 10
> >> agent_call_low_timeout = 5
> >> agent_call_high_timeout = 150
> >> agent_replication_snapshot_timeout = 36000
> >> use_nova_server_volume = False
> >> network_label_regex = .*
> >> template_path = /etc/trove/templates/
> >> pydev_debug = disabled
> >> [database]
> >> connection = mysql+pymysql://trove:pass4trove@controller/trove
> >> idle_timeout = 3600
> >> [profiler]
> >> [oslo_messaging_rabbit]
> >> rabbit_host=controller
> >> rabbit_userid=openstack
> >> rabbit_password=pass4rabbit
> >> [mysql]
> >> icmp = True
> >> tcp_ports = 3306
> >> volume_support = True
> >> device_path = /dev/vdb
> >> [redis]
> >> tcp_ports = 6379
> >> volume_support = False
> >> [cassandra]
> >> tcp_ports = 7000, 7001, 9042, 9160
> >> volume_support = True
> >> device_path = /dev/vdb
> >> [couchbase]
> >> tcp_ports = 8091, 8092, 4369, 11209-11211, 21100-21199
> >> volume_support = True
> >> device_path = /dev/vdb
> >> [mongodb]
> >> volume_support = True
> >> device_path = /dev/vdb
> >> [vertica]
> >> tcp_ports = 5433, 5434, 22, 5444, 5450, 4803
> >> udp_ports = 5433, 4803, 4804, 6453
> >> volume_support = True
> >> device_path = /dev/vdb
> >> mount_point = /var/lib/vertica
> >> taskmanager_strategy = trove.common.strategies.cluste
> >> r.experimental.vertica.taskmanager.VerticaTaskManagerStrategy
> >>
> >>
> >> trove-conductor.conf:
> >> [DEFAULT]
> >> debug = False
> >> trove_auth_url = http://controller:5000/v3
> >> notifier_queue_hostname = controller
> >> log_dir = /var/log/trove
> >> conductor_manager = trove.conductor.manager.Manager
> >> rpc_backend=rabbit
> >> control_exchange = trove
> >> [profiler]
> >> [database]
> >> connection = mysql+pymysql://trove:pass4trove@controller/trove
> >> [oslo_messaging_rabbit]
> >> rabbit_host=controller
> >> rabbit_userid=openstack
> >> rabbit_password=RABBITPASS
> >>
> >>
> >> trove-guestagent.conf:
> >> [DEFAULT]
> >> debug = True
> >> log_file = trove-guestagent.log
> >> log_dir = /var/log/trove
> >> ignore_users = os_admin
> >> control_exchange = trove
> >>
> >> rabbit_host = controller
> >> rabbit_userid = openstack
> >> rabbit_password = RABBITPASS
> >> nova_proxy_admin_user = admin
> >> nova_proxy_admin_pass = ADMIN_PASS
> >> nova_proxy_admin_tenant_name = service
> >> trove_auth_url = http://controller:35357/v3
> >>
> >>
> >> ____________________________________________________________
> >> ______________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
> >> e
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
More information about the OpenStack-dev
mailing list