I have a question related to external networks. What If I want to attach a network to a trove which doesn't have a neutron router attached. When I try public network which doesn't have any neutron router and I got error saying - Subnet d243bf1c-01db-408a-b8f1-757c3e28b4bb is not associated with router. Some of my customers use their own router like Pfsense instead of a neutron router. In that case how do I attach a network to trove database instances? On Thu, Jan 25, 2024 at 12:36 AM Satish Patel <satish.txt@gmail.com> wrote:
Never mind found this patch - https://review.opendev.org/c/openstack/kolla-ansible/+/904972/4
It works!!
# openstack database instance list
+-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+ | ID | Name | Datastore | Datastore Version | Status | Operating Status | Public | Addresses | Flavor ID | Size | Role |
+-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+ | 3538c3dc-5a2d-49ae- | mydb1 | mysql | 5.7.29 | ACTIVE | HEALTHY | False | [{'address': | 3 | 10 | | | b3cb-bcc3c5927153 | | | | | | | '10.0.0.232', 'type': | | | | | | | | | | | | 'private', 'network': | | | | | | | | | | | | '81ce26ec-1151-4960- | | | | | | | | | | | | b3ca-bf529e6296b2'}] | | | |
+-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+
On Wed, Jan 24, 2024 at 11:55 PM Satish Patel <satish.txt@gmail.com> wrote:
Hi W
I made good progress. Now my trove instances are stuck in the BUILD state. When I get on to an instance using ssh and I found the following error logs. I can telnet on RabbitMQ ports, which means I don't have any network firewalling issues. To me the error looks like related to the rabbitMQ queue declaration.
ERROR oslo_service.service amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'trove' in vhost '/': received 'false' but current is 'true'
Here are the full logs: Full logs - https://paste.opendev.org/show/bQdXnEFZ12JzTRTYhZwa/
Any idea what could be wrong?
On Wed, Jan 10, 2024 at 10:17 AM Satish Patel <satish.txt@gmail.com> wrote:
Can you send me that patch? I might try to test and see.
On Wed, Jan 10, 2024 at 9:19 AM chunyang wu <wchy1001@gmail.com> wrote:
There is already a patch to support Cinder and Ceph, but it hasn’t been reviewed yet.
Satish Patel <satish.txt@gmail.com> 于2024年1月10日周三 20:57写道：
Oh really? Any plan for ceph or s3 style backup?
On Wed, Jan 10, 2024 at 2:18 AM W Ch <wchy1001@gmail.com> wrote:
Trove only supports swift as backup backend at present.
2024年1月8日 21:30，Satish Patel <satish.txt@gmail.com> 写道：
We don’t use swift. We have ceph storage. Hope it’s not hard requirement for trove ;)
On Mon, Jan 8, 2024 at 12:36 AM W Ch <wchy1001@gmail.com> wrote:
> Typically, we don’t need to restrict outbound traffic, only need to > restrict inbound traffic. If you still want to restrict outbound traffic, > besides the service you mentioned above, you also need to add rules for > swift service. > > > 2024年1月8日 12:03，Satish Patel <satish.txt@gmail.com> 写道： > > Thank you W ch, > > Question related security group. I saw in patch only icmp & ssh is > allowed in security rules. That should be enough? > https://review.opendev.org/c/openstack/kolla-ansible/+/863321/12/ansible/rol... > > > For security reasons I want to restrict outbound traffic from db > instances (vm). In that case only RabbitMQ, Keystone & Docker should be > enough right? > > > On Sun, Jan 7, 2024 at 10:33 PM W Ch <wchy1001@gmail.com> wrote: > >> yes, you can use this image： >> https://tarballs.opendev.org/openstack/trove/images/trove-master-guest-ubunt... >> remember don’t use the image that it’s name contains “dev” string. >> >> What is the use of backup images? >> >> >> Trove uses this image to backup and restore instance. >> >> 2024年1月8日 11:26，Satish Patel <satish.txt@gmail.com> 写道： >> >> Very helpful, thank you! >> >> For my testing environment can I use publicly available images from >> here - https://tarballs.opendev.org/openstack/trove/images/ >> >> What is the use of backup images? >> >> On Sun, Jan 7, 2024 at 10:15 PM W Ch <wchy1001@gmail.com> wrote: >> >>> Since Victoria release, Trove started to use docker in >>> guest-agent. By default, guest-agent will pull the image(mysql, mariaDB or >>> postgres ) from the docker hub to set up the database. >>> For more information please read: >>> https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#p... >>> However, the backup images needs to be built by yourself. You >>> can refer to this patch: >>> https://review.opendev.org/c/openstack/trove/+/886619/22/doc/source/admin/bu... >>> >>> 2024年1月8日 10:59，Satish Patel <satish.txt@gmail.com> 写道： >>> >>> Thank you W Ch, >>> >>> It makes sense, I am thinking of creating a dbaas vlan interface >>> and using a vlan tenant for trove-mgmt-net instead of directly attaching to >>> the api network of kolla. >>> >>> What is the use of docker registry for trove guest agents? >>> >>> docker_insecure_registries = 10.1.2.49:6400 ? >>> >>> On Sun, Jan 7, 2024 at 8:39 PM W Ch <wchy1001@gmail.com> wrote: >>> >>>> HI, yes you are right . the patch [0] depends on the patch[1] >>>> >>>> In your case, you can create a trove manage network like octavia >>>> lb-mgmt-net, and set it as trove mgmt network. >>>> >>>> Setting mgmt network in kolla for Trove: >>>> Vim /etc/kolla/config/trove.conf >>>> >>>> [DEFAULT] >>>> management_networks = 7a93bb03-2797-4529-8d6e-27b1699b7eb1 // >>>> management network which is the same as the lb-mgmt-net, >>>> nova_keypair = trove-key // nova keypair, the same as the >>>> octavia nova key >>>> management_security_groups = c8c65b8b-17f5-48e5-b094-867a8ecfce32 >>>> // used for management port >>>> docker_insecure_registries = 10.1.2.49:6400 // if you are >>>> using an insecurity docker registry >>>> >>>> then, reconfigure your trove deployment // kolla-ansible >>>> reconfigure —tags trove, horizon >>>> >>>> As you mentioned above: >>>> >>>> Second thing I was thinking why don’t I expose rabbitmq/keystone >>>> etc on public ip using ngnix so trove vm can access those public >>>> endpoints.. what is the problem in this approach ? >>>> >>>> This is also feasible for trove deployment, but in this >>>> situation, the cloud operator might be hard to ssh into the trove guest >>>> vm for debugging. We don’t recommend to use this approach in production. >>>> >>>> Finally, you don’t need to cherry-pick the above patches. Just >>>> set up the resources manually to achieve the same result as patch[1]. >>>> >>>> [0]: >>>> https://review.opendev.org/c/openstack/kolla-ansible/+/863521 >>>> [1]: >>>> https://review.opendev.org/c/openstack/kolla-ansible/+/863321 >>>> >>>> 2024年1月8日 04:53，Satish Patel <satish.txt@gmail.com> 写道： >>>> >>>> Hi, >>>> >>>> Didn't realize that entire trove deployment depend on this patch >>>> set - >>>> https://review.opendev.org/c/openstack/kolla-ansible/+/863321 >>>> >>>> What stopped us to merge this patch because I am desperately >>>> looking to implement trove on our cloud infra. Let me know if you need help >>>> to test it out. >>>> >>>> >>>> >>>> On Sun, Jan 7, 2024 at 1:05 PM Satish Patel <satish.txt@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I have tried following in global.yml and it didn't create any >>>>> network. Assuming your patch is missing for that right? Or am I missing >>>>> something? >>>>> >>>>> # Trove (DBaaS) >>>>> enable_trove: "yes" >>>>> enable_cinder_backup: "no" >>>>> enable_cinder_backend_lvm: "no" >>>>> docker_disable_ip_forward: false >>>>> trove_mgmt_network: >>>>> name: trove-mgmt-net >>>>> provider_network_type: flat >>>>> provider_physical_network: physnet1 >>>>> external: True >>>>> shared: True >>>>> subnet: >>>>> name: trove-mgmt-subnet >>>>> cidr: "192.168.100.0/24" >>>>> gateway_ip: 192.168.100.1 >>>>> allocation_pool_start: 192.168.100.150 >>>>> allocation_pool_end: 192.168.100.199 >>>>> enable_dhcp: yes >>>>> >>>>> On Sun, Jan 7, 2024 at 11:33 AM Satish Patel < >>>>> satish.txt@gmail.com> wrote: >>>>> >>>>>> Thank you for the information. >>>>>> >>>>>> Let me understand. You are saying I should create trove-mgmt >>>>>> network which attached to all controller services and same network will >>>>>> attach to trove instances (guest-agent) right? (Just like Octavia >>>>>> lb-mgmt-net) >>>>>> >>>>>> Where do I tell trove to use trove-mgmt network when you spin >>>>>> up instance? (In what config file?) >>>>>> >>>>>> Do I need your patch for that or without that patch it’s >>>>>> possible? >>>>>> >>>>>> >>>>>> Second thing I was thinking why don’t I expose >>>>>> rabbitmq/keystone etc on public ip using ngnix so trove vm can access those >>>>>> public endpoints.. what is the problem in this approach ? >>>>>> >>>>>> On Sun, Jan 7, 2024 at 5:16 AM W Ch <wchy1001@gmail.com> wrote: >>>>>> >>>>>>> Hi, please refer to this patch: >>>>>>> https://review.opendev.org/c/openstack/kolla-ansible/+/863521/43/tests/templ... >>>>>>> in kolla-ansible. >>>>>>> >>>>>>> In your production, we recommend to create a provider network >>>>>>> (vlan or flat network) to be used as the trove management network. >>>>>>> >>>>>>> This network should be responsible for accessing rabbitmq, >>>>>>> swift, keystone and docker registry. >>>>>>> >>>>>>> For more information about trove management network, please >>>>>>> read this document: >>>>>>> https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#m... >>>>>>> >>>>>>> thanks. >>>>>>> >>>>>>> 2024年1月6日 22:55，Satish Patel <satish.txt@gmail.com> 写道： >>>>>>> >>>>>>> Folks, >>>>>>> >>>>>>> I am trying to find a kolla-ansible doc related to how to >>>>>>> implement trove in production with best practices. Especially when a >>>>>>> guest-agent (running inside VM) talks to RPC service. >>>>>>> >>>>>>> How are other folks doing in their environments? In most cases >>>>>>> RPC is an internal service and not visible to any guest VMs. Looking for >>>>>>> some guidance. >>>>>>> >>>>>>> >>>>>>> >>>> >>> >> >