<div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">(adding openstack-discuss mail address)</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Hi Abhinav,</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Typically, there are 2 ports attached to Trove instance, one is mgmt port meant to communicate with rabbitmq but can also be responsible for other mgmt traffic (e.g. docker image pull) after routing table or dns customization, that depends on your business requirements. The other one is the user port mainly for database access. You can find more description here <a href="https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#management-network">https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#management-network</a>.</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">There is only 1 Nova VM for each Trove instance.</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Yes, you are using the right image if you deployed Ussuri Trove.</div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="color:rgb(102,102,102);font-family:monospace,monospace"><br></span></div><div><span style="color:rgb(102,102,102);font-family:monospace,monospace">---</span><br></div><div><font color="#666666" face="monospace, monospace">Lingxian Kong</font></div><div><font color="#666666" face="monospace, monospace">Senior Cloud Engineer (Catalyst Cloud)</font></div><div><font color="#666666" face="monospace, monospace">Trove PTL (OpenStack)</font></div><div><font color="#666666" face="monospace, monospace">OpenStack Cloud Provider Co-Lead (Kubernetes)</font><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 19, 2021 at 6:25 PM Abhinav Tyagi <<a href="mailto:abhinav31796@gmail.com">abhinav31796@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Lingxian,</div><div><br></div><div>Thanks for the timely help. <br></div><div><br></div><div>We checked the logs and found out the trove guest agent instance is not able to connect to rabbimq container. We are using OpenStack-Ansible for deployment.
It's a 3 master setup.
But trove we have deployed manually on one compute physical node. <br></div><div><br></div><div>Could it be because of the management network of trove and containers is not the same? Does trove guest instance should have 2 NIC's? When we create trove database instance, are there 2 instances launched? Are we using the right image?</div><div><br></div><div>Regards,</div><div>Aabhinav Tyagi<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 19, 2021 at 3:08 AM Lingxian Kong <<a href="mailto:anlin.kong@gmail.com" target="_blank">anlin.kong@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">Hi Abhinav,</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">Please check the trove guest agent log according to this guide <a href="https://docs.openstack.org/trove/latest/admin/troubleshooting.html" target="_blank">https://docs.openstack.org/trove/latest/admin/troubleshooting.html</a>.</div><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="color:rgb(102,102,102);font-family:monospace,monospace"><br></span></div><div><span style="color:rgb(102,102,102);font-family:monospace,monospace">---</span><br></div><div><font face="monospace, monospace" color="#666666">Lingxian Kong</font></div><div><font face="monospace, monospace" color="#666666">Senior Cloud Engineer (Catalyst Cloud)</font></div><div><font face="monospace, monospace" color="#666666">Trove PTL (OpenStack)</font></div><div><font face="monospace, monospace" color="#666666">OpenStack Cloud Provider Co-Lead (Kubernetes)</font><br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 19, 2021 at 9:20 AM Abhinav Tyagi <<a href="mailto:abhinav31796@gmail.com" target="_blank">abhinav31796@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>Please help us in using trove. We are using <a href="https://tarballs.opendev.org/openstack/trove/images/trove-ussuri-mysql-ubuntu-xenial.qcow2" target="_blank">https://tarballs.opendev.org/openstack/trove/images/trove-ussuri-mysql-ubuntu-xenial.qcow2</a> image. <br></div><div>We have a multi master openstack setup.</div><div><br></div><div>These are the logs:</div><div><br></div><div>Feb 18 19:03:30 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:03:30.535 1274599 INFO trove.taskmanager.models [-] Waiting for instance 5fe896f6-dc7b-424a-87c2-64f207f422a2 up and running with timeout 900s<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall [-] Dynamic backoff interval looping call 'trove.common.utils.build_polling_task.<locals>.poll_and_check' failed: oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 893.31 seconds<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall Traceback (most recent call last):<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall File "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 154, in _run_loop<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall idle = idle_for_func(result, self._elapsed(watch))<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall File "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 351, in _idle_for<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall % self._error_time)<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall oslo_service.loopingcall.LoopingCallTimeOut: Looping call timed out after 893.31 seconds<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.940 1274599 ERROR oslo.service.loopingcall<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.941 1274599 ERROR trove.taskmanager.models [-] Failed to create instance 5fe896f6-dc7b-424a-87c2-64f207f422a2, error: Polling request timed out..: trove.common.exception.PollTimeOut: Polling request timed out.<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.960 1274599 ERROR trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: trove.common.exception.PollTimeOut: Polling request timed out.<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.970 1274599 DEBUG trove.db.models [-] Saving DBInstance: {'_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x7fb5447bd668>, 'tenant_id': '19aa2578cb444b828549b394eed8ebfb', 'hostname': None, 'shard_id': None, 'server_status': None, 'compute_instance_id': 'b928d7e4-6706-48d2-9fb9-c75e19b9765c', 'type': None, 'deleted': 0, 'id': '5fe896f6-dc7b-424a-87c2-64f207f422a2', 'task_id': 91, 'region_id': 'Bangalore', 'deleted_at': None, 'task_description': 'Build error: guestagent timeout.', 'encrypted_key': '***', 'datastore_version_id': 'b4bbc01c-e785-4944-8a76-6c154c19ddff', 'task_start_time': None, 'created': datetime.datetime(2021, 2, 18, 13, 33, 24), 'configuration_id': None, 'volume_id': '68accc42-34ae-4ac3-93fc-1efb9fed2af9', 'slave_of_id': None, 'updated': datetime.datetime(2021, 2, 18, 13, 48, 33, 969535), 'flavor_id': 'bbf4bad4-e258-45b4-8860-e5a51c86ed6e', 'name': 'trove_instance_15', 'cluster_id': None, 'volume_size': 10, 'errors': {}} save /usr/lib/python3.6/site-packages/trove/db/models.py:65<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.981 1274599 ERROR trove.taskmanager.models [-] Trove instance status: ERROR, Trove instance status description: Build error: guestagent timeout.: trove.common.exception.PollTimeOut: Polling request timed out.<br>Feb 18 19:18:33 <a href="http://bngoscompn01.comviva.com" target="_blank">bngoscompn01.comviva.com</a> trove-taskmanager[1274599]: 2021-02-18 19:18:33.994 1274599 DEBUG trove.db.models [-] Saving DBInstanceFault: {'_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x7fb5448688d0>, 'id': 'f0983744-4cfe-4863-b9ae-6fb33cc8a0d2', 'created': datetime.datetime(2021, 2, 18, 13, 48, 33, 994216), 'deleted': False, 'instance_id': '5fe896f6-dc7b-424a-87c2-64f207f422a2', 'message': 'Polling request timed out.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/trove/common/utils.py", line 207, in wait_for_task\n return polling_task.wait()\n File "/usr/local/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n File "/usr/local/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 309, in switch\n return self.greenlet.switch()\n File "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 154, in _run_loop\n idle = idle_for_func(result, self._elapsed(watch))\n File "/usr/lib/python3.6/site-packages/oslo_service/loopingcall.py", line 351, in _idle_for\n % self._error_time)\noslo_service.loopingcall.LoopingCallTimeOut:\n Looping call timed out after 893.31 seconds\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/usr/lib/python3.6/site-packages/trove/taskmanager/models.py", line 431, in wait_for_instance\n time_out=timeout)\n File "/usr/lib/python3.6/site-packages/trove/common/utils.py", line 222, in poll_until\n return wait_for_task(task)\n File "/usr/lib/python3.6/site-packages/trove/common/utils.py", line 209, in wait_for_task\n raise exception.PollTimeOut\ntrove.common.exception.PollTimeOut: Polling request timed out.\n', 'errors': {}, 'updated': datetime.datetime(2021, 2, 18, 13, 48, 33, 994288)} save /usr/lib/python3.6/site-packages/trove/db/models.py:65<br><br><br></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>