Never mind found this patch - https://review.opendev.org/c/openstack/kolla-ansible/+/904972/4 It works!! # openstack database instance list +-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+ | ID | Name | Datastore | Datastore Version | Status | Operating Status | Public | Addresses | Flavor ID | Size | Role | +-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+ | 3538c3dc-5a2d-49ae- | mydb1 | mysql | 5.7.29 | ACTIVE | HEALTHY | False | [{'address': | 3 | 10 | | | b3cb-bcc3c5927153 | | | | | | | '10.0.0.232', 'type': | | | | | | | | | | | | 'private', 'network': | | | | | | | | | | | | '81ce26ec-1151-4960- | | | | | | | | | | | | b3ca-bf529e6296b2'}] | | | | +-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+ On Wed, Jan 24, 2024 at 11:55 PM Satish Patel <satish.txt@gmail.com> wrote:
Hi W
I made good progress. Now my trove instances are stuck in the BUILD state. When I get on to an instance using ssh and I found the following error logs. I can telnet on RabbitMQ ports, which means I don't have any network firewalling issues. To me the error looks like related to the rabbitMQ queue declaration.
ERROR oslo_service.service amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'trove' in vhost '/': received 'false' but current is 'true'
Here are the full logs: Full logs - https://paste.opendev.org/show/bQdXnEFZ12JzTRTYhZwa/
Any idea what could be wrong?
On Wed, Jan 10, 2024 at 10:17 AM Satish Patel <satish.txt@gmail.com> wrote:
Can you send me that patch? I might try to test and see.
On Wed, Jan 10, 2024 at 9:19 AM chunyang wu <wchy1001@gmail.com> wrote:
There is already a patch to support Cinder and Ceph, but it hasn’t been reviewed yet.
Satish Patel <satish.txt@gmail.com> 于2024年1月10日周三 20:57写道：
Oh really? Any plan for ceph or s3 style backup?
On Wed, Jan 10, 2024 at 2:18 AM W Ch <wchy1001@gmail.com> wrote:
Trove only supports swift as backup backend at present.
2024年1月8日 21:30，Satish Patel <satish.txt@gmail.com> 写道：
We don’t use swift. We have ceph storage. Hope it’s not hard requirement for trove ;)
On Mon, Jan 8, 2024 at 12:36 AM W Ch <wchy1001@gmail.com> wrote:
Typically, we don’t need to restrict outbound traffic, only need to restrict inbound traffic. If you still want to restrict outbound traffic, besides the service you mentioned above, you also need to add rules for swift service.
2024年1月8日 12:03，Satish Patel <satish.txt@gmail.com> 写道：
Thank you W ch,
Question related security group. I saw in patch only icmp & ssh is allowed in security rules. That should be enough? https://review.opendev.org/c/openstack/kolla-ansible/+/863321/12/ansible/rol...
For security reasons I want to restrict outbound traffic from db instances (vm). In that case only RabbitMQ, Keystone & Docker should be enough right?
On Sun, Jan 7, 2024 at 10:33 PM W Ch <wchy1001@gmail.com> wrote:
> yes, you can use this image： > https://tarballs.opendev.org/openstack/trove/images/trove-master-guest-ubunt... > remember don’t use the image that it’s name contains “dev” string. > > What is the use of backup images? > > > Trove uses this image to backup and restore instance. > > 2024年1月8日 11:26，Satish Patel <satish.txt@gmail.com> 写道： > > Very helpful, thank you! > > For my testing environment can I use publicly available images from > here - https://tarballs.opendev.org/openstack/trove/images/ > > What is the use of backup images? > > On Sun, Jan 7, 2024 at 10:15 PM W Ch <wchy1001@gmail.com> wrote: > >> Since Victoria release, Trove started to use docker in >> guest-agent. By default, guest-agent will pull the image(mysql, mariaDB or >> postgres ) from the docker hub to set up the database. >> For more information please read: >> https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#p... >> However, the backup images needs to be built by yourself. You can >> refer to this patch: >> https://review.opendev.org/c/openstack/trove/+/886619/22/doc/source/admin/bu... >> >> 2024年1月8日 10:59，Satish Patel <satish.txt@gmail.com> 写道： >> >> Thank you W Ch, >> >> It makes sense, I am thinking of creating a dbaas vlan interface >> and using a vlan tenant for trove-mgmt-net instead of directly attaching to >> the api network of kolla. >> >> What is the use of docker registry for trove guest agents? >> >> docker_insecure_registries = 10.1.2.49:6400 ? >> >> On Sun, Jan 7, 2024 at 8:39 PM W Ch <wchy1001@gmail.com> wrote: >> >>> HI, yes you are right . the patch [0] depends on the patch[1] >>> >>> In your case, you can create a trove manage network like octavia >>> lb-mgmt-net, and set it as trove mgmt network. >>> >>> Setting mgmt network in kolla for Trove: >>> Vim /etc/kolla/config/trove.conf >>> >>> [DEFAULT] >>> management_networks = 7a93bb03-2797-4529-8d6e-27b1699b7eb1 // >>> management network which is the same as the lb-mgmt-net, >>> nova_keypair = trove-key // nova keypair, the same as the >>> octavia nova key >>> management_security_groups = c8c65b8b-17f5-48e5-b094-867a8ecfce32 >>> // used for management port >>> docker_insecure_registries = 10.1.2.49:6400 // if you are >>> using an insecurity docker registry >>> >>> then, reconfigure your trove deployment // kolla-ansible >>> reconfigure —tags trove, horizon >>> >>> As you mentioned above: >>> >>> Second thing I was thinking why don’t I expose rabbitmq/keystone >>> etc on public ip using ngnix so trove vm can access those public >>> endpoints.. what is the problem in this approach ? >>> >>> This is also feasible for trove deployment, but in this >>> situation, the cloud operator might be hard to ssh into the trove guest >>> vm for debugging. We don’t recommend to use this approach in production. >>> >>> Finally, you don’t need to cherry-pick the above patches. Just set >>> up the resources manually to achieve the same result as patch[1]. >>> >>> [0]: >>> https://review.opendev.org/c/openstack/kolla-ansible/+/863521 >>> [1]: https://review.opendev.org/c/openstack/kolla-ansible/+/863321 >>> >>> 2024年1月8日 04:53，Satish Patel <satish.txt@gmail.com> 写道： >>> >>> Hi, >>> >>> Didn't realize that entire trove deployment depend on this patch >>> set - >>> https://review.opendev.org/c/openstack/kolla-ansible/+/863321 >>> >>> What stopped us to merge this patch because I am desperately >>> looking to implement trove on our cloud infra. Let me know if you need help >>> to test it out. >>> >>> >>> >>> On Sun, Jan 7, 2024 at 1:05 PM Satish Patel <satish.txt@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I have tried following in global.yml and it didn't create any >>>> network. Assuming your patch is missing for that right? Or am I missing >>>> something? >>>> >>>> # Trove (DBaaS) >>>> enable_trove: "yes" >>>> enable_cinder_backup: "no" >>>> enable_cinder_backend_lvm: "no" >>>> docker_disable_ip_forward: false >>>> trove_mgmt_network: >>>> name: trove-mgmt-net >>>> provider_network_type: flat >>>> provider_physical_network: physnet1 >>>> external: True >>>> shared: True >>>> subnet: >>>> name: trove-mgmt-subnet >>>> cidr: "192.168.100.0/24" >>>> gateway_ip: 192.168.100.1 >>>> allocation_pool_start: 192.168.100.150 >>>> allocation_pool_end: 192.168.100.199 >>>> enable_dhcp: yes >>>> >>>> On Sun, Jan 7, 2024 at 11:33 AM Satish Patel < >>>> satish.txt@gmail.com> wrote: >>>> >>>>> Thank you for the information. >>>>> >>>>> Let me understand. You are saying I should create trove-mgmt >>>>> network which attached to all controller services and same network will >>>>> attach to trove instances (guest-agent) right? (Just like Octavia >>>>> lb-mgmt-net) >>>>> >>>>> Where do I tell trove to use trove-mgmt network when you spin up >>>>> instance? (In what config file?) >>>>> >>>>> Do I need your patch for that or without that patch it’s >>>>> possible? >>>>> >>>>> >>>>> Second thing I was thinking why don’t I expose rabbitmq/keystone >>>>> etc on public ip using ngnix so trove vm can access those public >>>>> endpoints.. what is the problem in this approach ? >>>>> >>>>> On Sun, Jan 7, 2024 at 5:16 AM W Ch <wchy1001@gmail.com> wrote: >>>>> >>>>>> Hi, please refer to this patch: >>>>>> https://review.opendev.org/c/openstack/kolla-ansible/+/863521/43/tests/templ... >>>>>> in kolla-ansible. >>>>>> >>>>>> In your production, we recommend to create a provider network >>>>>> (vlan or flat network) to be used as the trove management network. >>>>>> >>>>>> This network should be responsible for accessing rabbitmq, >>>>>> swift, keystone and docker registry. >>>>>> >>>>>> For more information about trove management network, please >>>>>> read this document: >>>>>> https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#m... >>>>>> >>>>>> thanks. >>>>>> >>>>>> 2024年1月6日 22:55，Satish Patel <satish.txt@gmail.com> 写道： >>>>>> >>>>>> Folks, >>>>>> >>>>>> I am trying to find a kolla-ansible doc related to how to >>>>>> implement trove in production with best practices. Especially when a >>>>>> guest-agent (running inside VM) talks to RPC service. >>>>>> >>>>>> How are other folks doing in their environments? In most cases >>>>>> RPC is an internal service and not visible to any guest VMs. Looking for >>>>>> some guidance. >>>>>> >>>>>> >>>>>> >>> >> >