I have a question related to external networks. What If I want to attach a network to a trove which doesn't have a neutron router attached. When I try public network which doesn't have any neutron router and I got error saying - Subnet d243bf1c-01db-408a-b8f1-757c3e28b4bb is not associated with router.

Some of my customers use their own router like Pfsense instead of a neutron router. In that case how do I attach a network to trove database instances? 

On Thu, Jan 25, 2024 at 12:36 AM Satish Patel <satish.txt@gmail.com> wrote:
Never mind found this patch - https://review.opendev.org/c/openstack/kolla-ansible/+/904972/4 

It works!! 

# openstack  database instance list
+-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+
| ID                    | Name  | Datastore | Datastore Version | Status | Operating Status | Public | Addresses              | Flavor ID | Size | Role |
+-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+
| 3538c3dc-5a2d-49ae-   | mydb1 | mysql     | 5.7.29            | ACTIVE | HEALTHY          | False  | [{'address':           | 3         |   10 |      |
| b3cb-bcc3c5927153     |       |           |                   |        |                  |        | '10.0.0.232', 'type':  |           |      |      |
|                       |       |           |                   |        |                  |        | 'private', 'network':  |           |      |      |
|                       |       |           |                   |        |                  |        | '81ce26ec-1151-4960-   |           |      |      |
|                       |       |           |                   |        |                  |        | b3ca-bf529e6296b2'}]   |           |      |      |
+-----------------------+-------+-----------+-------------------+--------+------------------+--------+------------------------+-----------+------+------+

On Wed, Jan 24, 2024 at 11:55 PM Satish Patel <satish.txt@gmail.com> wrote:
Hi W

I made good progress. Now my trove instances are stuck in the BUILD state. When I get on to an instance using ssh and I found the following error logs. I can telnet on RabbitMQ ports, which means I don't have any network firewalling issues. To me the error looks like related to the rabbitMQ queue declaration. 

ERROR oslo_service.service amqp.exceptions.PreconditionFailed: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'trove' in vhost '/': received 'false' but current is 'true'

Here are the full logs: Full logs - https://paste.opendev.org/show/bQdXnEFZ12JzTRTYhZwa/
 
Any idea what could be wrong? 



On Wed, Jan 10, 2024 at 10:17 AM Satish Patel <satish.txt@gmail.com> wrote:
Can you send me that patch? I might try to test and see. 

On Wed, Jan 10, 2024 at 9:19 AM chunyang wu <wchy1001@gmail.com> wrote:
There is already a patch to support Cinder and Ceph, but it hasn’t been reviewed yet.

Satish Patel <satish.txt@gmail.com> 于2024年1月10日周三 20:57写道：
Oh really? Any plan for ceph or s3 style backup? 

On Wed, Jan 10, 2024 at 2:18 AM W Ch <wchy1001@gmail.com> wrote:
Trove only supports swift as backup backend at present.


2024年1月8日 21:30，Satish Patel <satish.txt@gmail.com> 写道：

We don’t use swift. We have ceph storage. Hope it’s not hard requirement for trove ;)

On Mon, Jan 8, 2024 at 12:36 AM W Ch <wchy1001@gmail.com> wrote:
Typically, we don’t need to restrict outbound traffic, only need to restrict inbound traffic. If you still want to restrict outbound traffic,  besides the service you mentioned above, you also need to add rules for swift service.


2024年1月8日 12:03，Satish Patel <satish.txt@gmail.com> 写道：

Thank you W ch, 

Question related security group. I saw in patch only icmp & ssh is allowed in security rules. That should be enough? https://review.opendev.org/c/openstack/kolla-ansible/+/863321/12/ansible/roles/trove/defaults/main.yml#203 

For security reasons I want to restrict outbound traffic from db instances (vm). In that case only RabbitMQ, Keystone & Docker should be enough right? 


On Sun, Jan 7, 2024 at 10:33 PM W Ch <wchy1001@gmail.com> wrote:
yes, you can use this image： https://tarballs.opendev.org/openstack/trove/images/trove-master-guest-ubuntu-jammy.qcow2   remember don’t use the image that it’s name contains “dev” string.

What is the use of backup images? 

Trove uses this image to backup and restore instance.

2024年1月8日 11:26，Satish Patel <satish.txt@gmail.com> 写道：

Very helpful, thank you! 

For my testing environment can I use publicly available images from here - https://tarballs.opendev.org/openstack/trove/images/ 

What is the use of backup images? 

On Sun, Jan 7, 2024 at 10:15 PM W Ch <wchy1001@gmail.com> wrote:
Since Victoria release, Trove started to use docker in guest-agent. By default, guest-agent will pull the image(mysql, mariaDB or postgres ) from the docker hub to set up the database.
However, the backup images needs to be built by yourself.   You can refer to this patch: https://review.opendev.org/c/openstack/trove/+/886619/22/doc/source/admin/building_guest_images.rst#331

2024年1月8日 10:59，Satish Patel <satish.txt@gmail.com> 写道：

Thank you W Ch, 

It makes sense, I am thinking of creating a dbaas vlan interface and using a vlan tenant for trove-mgmt-net instead of directly attaching to the api network of kolla. 

What is the use of docker registry for trove guest agents?

 docker_insecure_registries =  10.1.2.49:6400  ? 

On Sun, Jan 7, 2024 at 8:39 PM W Ch <wchy1001@gmail.com> wrote:
HI,  yes you are right . the patch [0] depends on the patch[1]

In your case, you can create a trove manage network like octavia lb-mgmt-net, and set it as trove mgmt network.

Setting mgmt network in kolla for Trove:
Vim /etc/kolla/config/trove.conf 

[DEFAULT]
management_networks = 7a93bb03-2797-4529-8d6e-27b1699b7eb1    // management network which is the same as the lb-mgmt-net, 
nova_keypair = trove-key    // nova keypair, the same as the octavia nova key
management_security_groups = c8c65b8b-17f5-48e5-b094-867a8ecfce32     // used for management  port 
docker_insecure_registries = 10.1.2.49:6400      // if you are using an insecurity docker registry

then,  reconfigure your trove deployment //   kolla-ansible reconfigure —tags trove, horizon

As you mentioned above:

Second thing I was thinking why don’t I expose rabbitmq/keystone etc on public ip using ngnix so trove vm can access those public endpoints.. what is the problem in this approach ? 

This is also feasible for trove deployment,  but in this situation, the cloud operator might  be hard to ssh into  the trove guest vm  for debugging. We don’t recommend to use this approach in production.

Finally, you don’t need to cherry-pick the above patches. Just set up the resources manually to achieve the same result as patch[1].


2024年1月8日 04:53，Satish Patel <satish.txt@gmail.com> 写道：

Hi,

Didn't realize that entire trove deployment depend on this patch set - https://review.opendev.org/c/openstack/kolla-ansible/+/863321 

What stopped us to merge this patch because I am desperately looking to implement trove on our cloud infra. Let me know if you need help to test it out. 



On Sun, Jan 7, 2024 at 1:05 PM Satish Patel <satish.txt@gmail.com> wrote:
Hi,

I have tried following in global.yml and it didn't create any network. Assuming your patch is missing for that right? Or am I missing something? 

# Trove (DBaaS)
enable_trove: "yes"
enable_cinder_backup: "no"
enable_cinder_backend_lvm: "no"
docker_disable_ip_forward: false
trove_mgmt_network:
   name: trove-mgmt-net
   provider_network_type: flat
   provider_physical_network: physnet1
   external: True
   shared: True
   subnet:
     name: trove-mgmt-subnet
     cidr: "192.168.100.0/24"
     gateway_ip: 192.168.100.1
     allocation_pool_start: 192.168.100.150
     allocation_pool_end: 192.168.100.199
     enable_dhcp: yes

On Sun, Jan 7, 2024 at 11:33 AM Satish Patel <satish.txt@gmail.com> wrote:
Thank you for the information.

Let me understand. You are saying I should create trove-mgmt network which attached to all controller services and same network will attach to trove instances (guest-agent) right?  (Just like Octavia lb-mgmt-net) 

Where do I tell trove to use trove-mgmt network when you spin up instance? (In what config file?)

Do I need your patch for that or without that patch it’s possible? 


Second thing I was thinking why don’t I expose rabbitmq/keystone etc on public ip using ngnix so trove vm can access those public endpoints.. what is the problem in this approach ? 

On Sun, Jan 7, 2024 at 5:16 AM W Ch <wchy1001@gmail.com> wrote:
Hi, please refer to this patch: https://review.opendev.org/c/openstack/kolla-ansible/+/863521/43/tests/templates/globals-default.j2   in kolla-ansible.

In your production, we recommend to create a provider network (vlan or flat network) to be used as the trove management network.

This network should be responsible for accessing  rabbitmq, swift, keystone and docker registry. 

For more information about trove management network, please read this document:  https://docs.openstack.org/trove/latest/admin/run_trove_in_production.html#management-network

thanks.

2024年1月6日 22:55，Satish Patel <satish.txt@gmail.com> 写道：

Folks,

I am trying to find a kolla-ansible doc related to how to implement trove in production with best practices. Especially when a guest-agent (running inside VM) talks to RPC service. 

How are other folks doing in their environments? In most cases RPC is an internal service and not visible to any guest VMs. Looking for some guidance.