Open Stack

Hi!

Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me.

Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read
about in the latest edition of the book "Openstack Trove Essentials" by Alok Shrivastwa and Sunil Sarat that i purchased recently.

For example, as mentioned in the book:

- Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents
  (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database. (page 6)
- The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. (page 10)
- As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. (page 58)
When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function build_guest_image() is performed, involving those needed elements
as minimal requirements:
- ubuntu-minimal (which also invokes ubuntu-common i think)
- cloud-init-datasources
- pip-and-virtualenv
- pip-cache
- guest-agent
- ${guest_os}-docker
- root-passwd
ref:
https://github.com/openstack/trove/blob/master/integration/scripts/functions_qemu
So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation
or by manipulating elements (for example, such as we have a script placed in ubuntu-common element that disables privacy extensions for IPv6 (RFC4941):

/usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041
I am really looking forward to our soon deployment of the Trove project, i see huge potential there!
Best Regards



Bekir Fajkovic
Senior DBA
Mobile: +46 70 019 48 47

www.citynetwork.eu | www.citycloud.com

INNOVATION THROUGH OPEN IT INFRASTRUCTURE
ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED

----- Original Message -----
From: Lingxian Kong (anlin.kong at gmail.com)
Date: 01/24/21 12:02
To: Bekir Fajkovic (bekir.fajkovic at citynetwork.eu)
Cc: openstack-discuss (openstack-discuss at lists.openstack.org)
Subject: Re: Some questions regarding OpenStack Trove Victoria release

Hi Bekir,
I'm very happy to answer your questions. See me comments in line.

On Sat, Jan 23, 2021 at 8:24 AM Bekir Fajkovic <bekir.fajkovic at citynetwork.eu> wrote:


The questions:
------------------------

Having the fact that Trove Victoria release provides Docker containers as a new way of database instance provisioning, i am wondering how far the project
is developed in terms of covering the different types of databases. What i can see by mainly parsing the code provided on Github, those seem to be 
officially released:

- MySQL
- MariaDB
- PostgreSQL

and the rest of the planned database types are in "experimental" phase. And also, regarding certain types of databases (for example MySQL, version 5.7 and 8.0) only certain 
versions of the datastores seems to be supported, but not all.


MySQL 5.7.x is supported but 8.0 is in experimental.
 

On the other hand, nothing regarding datastore versions supported for MariaDB and PostgreSQL seems to be
mentioned somewhere. Could someone please confirm that as well as give some more details about it?


MariaDB 10.4.x and PostgreSQL 12.4 are supported. The other versions need to be fully tested.



I successfully managed to create certain versions of datastores in my devstack environment, belonging to those 3 database types mentioned above (and based on
trovestack-generated dev guest image that is by default delivered with devstack installation), but not without some undesirable events. For example, i am able to register 
PostgreSQL datastore version 12 and instantiate a database instance of that version but not version 13 and above, where i get some hostname-related errors etc.

Yes, because PostgreSQL 13 has never been tested.


Also, a question regarding the building of the production-ready guest image. As mentioned, Trovestack script is provided as a possible way of producing the images (by omitting
dev option the Trove Guest Agent binaries are deployed into the instantiated VM). How does an image produced this way looks like? From where the base image is fetched,
is it a "cloud based image" with cloud-init in it, are the automatic security and software patching features disabled in produced image, so that we do not get unexpected service 
interruptions when the OS suddenly decides to start updating itself etc..

If you look at trovestack script implementation, you can see it's calling disk-image-create script from diskimage-builder, and there are some elements[2] defined in trove repo[3] for building the image.
[1]: https://docs.openstack.org/diskimage-builder/latest
[2]: https://docs.openstack.org/diskimage-builder/latest/elements.html
[3]: https://github.com/openstack/trove/tree/master/integration/scripts/files/elements


Regarding the Trove Guest Agent service - i read in some Trove books previously that there are dedicated agents for each and every database type, is it the same situation
in Victoria release, or is there an "universal" Guest Agent covering all the database types nowadays? Where is the code that adapts the Agent commands towards the database 
instances placed inside the project?


Trove never has dedicated agents for each and every database type, it's using the same trove-guestagent but with different configurations for different datastores.



The backups - as i can see there seem to be some kind of dedicated docker-backup images involved in each database type. Could someone explain the internals of backup mechanisms
inside Trove Victoria release in more details?


The backup container image is created to help trove-guestagent to achieve datastore-agnostic (so we only need to maintain a universal guest image), we shift the backup and restore functionalities and needed tools to a dedicated container. The implementation can be found here[4].
[4]: https://github.com/openstack/trove/tree/master/backup


---
Lingxian Kong
Senior Software Engineer
Catalyst Cloud
www.catalystcloud.nz 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210126/9e8970ea/attachment.html>