Some questions regarding OpenStack Trove Victoria release
Hello! My name is Bekir Fajkovic and i work at City Network cloud service provider mostly as a DBA but i am also involved in many other kind of activities inside the Company. We at City Network are currently in a process of deployment of OpenStack Trove project (latest official release) and are thus preparing a dedicated region inside our OpenStack hosting environment to host it as a beta version internally, to begin with, where we are going to involve some of our customers to evaluate the service after, hopefully, successful installation and configuration. To be totally honest, while there is a whole lot of a certain kind of documentation provided covering many aspects, some other aspects inside the project do not seem to be documented in depth or at all (which of course is totally understandable), so i am forced to ask some questions this way and i hope You find them not to be too annoying to anyone. The questions: ------------------------ Having the fact that Trove Victoria release provides Docker containers as a new way of database instance provisioning, i am wondering how far the project is developed in terms of covering the different types of databases. What i can see by mainly parsing the code provided on Github, those seem to be officially released: - MySQL - MariaDB - PostgreSQL and the rest of the planned database types are in "experimental" phase. And also, regarding certain types of databases (for example MySQL, version 5.7 and 8.0) only certain versions of the datastores seems to be supported, but not all. On the other hand, nothing regarding datastore versions supported for MariaDB and PostgreSQL seems to be mentioned somewhere. Could someone please confirm that as well as give some more details about it? I successfully managed to create certain versions of datastores in my devstack environment, belonging to those 3 database types mentioned above (and based on trovestack-generated dev guest image that is by default delivered with devstack installation), but not without some undesirable events. For example, i am able to register PostgreSQL datastore version 12 and instantiate a database instance of that version but not version 13 and above, where i get some hostname-related errors etc. Also, a question regarding the building of the production-ready guest image. As mentioned, Trovestack script is provided as a possible way of producing the images (by omitting dev option the Trove Guest Agent binaries are deployed into the instantiated VM). How does an image produced this way looks like? From where the base image is fetched, is it a "cloud based image" with cloud-init in it, are the automatic security and software patching features disabled in produced image, so that we do not get unexpected service interruptions when the OS suddenly decides to start updating itself etc.. Regarding the Trove Guest Agent service - i read in some Trove books previously that there are dedicated agents for each and every database type, is it the same situation in Victoria release, or is there an "universal" Guest Agent covering all the database types nowadays? Where is the code that adapts the Agent commands towards the database instances placed inside the project? The backups - as i can see there seem to be some kind of dedicated docker-backup images involved in each database type. Could someone explain the internals of backup mechanisms inside Trove Victoria release in more details? So, that would be all for the moment and although You probably consider my questions being as stupid as possible, i still dared to ask them :) I hope You will be able to provide the answers to at least some of them! Thanks in advance! Best Regards Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
Hi Bekir, I'm very happy to answer your questions. See me comments in line. On Sat, Jan 23, 2021 at 8:24 AM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
The questions: ------------------------
Having the fact that Trove Victoria release provides Docker containers as a new way of database instance provisioning, i am wondering how far the project is developed in terms of covering the different types of databases. What i can see by mainly parsing the code provided on Github, those seem to be officially released:
- MySQL - MariaDB - PostgreSQL
and the rest of the planned database types are in "experimental" phase. And also, regarding certain types of databases (for example MySQL, version 5.7 and 8.0) only certain versions of the datastores seems to be supported, but not all.
MySQL 5.7.x is supported but 8.0 is in experimental.
On the other hand, nothing regarding datastore versions supported for MariaDB and PostgreSQL seems to be mentioned somewhere. Could someone please confirm that as well as give some more details about it?
MariaDB 10.4.x and PostgreSQL 12.4 are supported. The other versions need to be fully tested.
I successfully managed to create certain versions of datastores in my devstack environment, belonging to those 3 database types mentioned above (and based on trovestack-generated dev guest image that is by default delivered with devstack installation), but not without some undesirable events. For example, i am able to register PostgreSQL datastore version 12 and instantiate a database instance of that version but not version 13 and above, where i get some hostname-related errors etc.
Yes, because PostgreSQL 13 has never been tested.
Also, a question regarding the building of the production-ready guest image. As mentioned, Trovestack script is provided as a possible way of producing the images (by omitting dev option the Trove Guest Agent binaries are deployed into the instantiated VM). How does an image produced this way looks like? From where the base image is fetched, is it a "cloud based image" with cloud-init in it, are the automatic security and software patching features disabled in produced image, so that we do not get unexpected service interruptions when the OS suddenly decides to start updating itself etc..
If you look at trovestack script implementation, you can see it's calling disk-image-create script from diskimage-builder, and there are some elements[2] defined in trove repo[3] for building the image. [1]: https://docs.openstack.org/diskimage-builder/latest [2]: https://docs.openstack.org/diskimage-builder/latest/elements.html [3]: https://github.com/openstack/trove/tree/master/integration/scripts/files/ele...
Regarding the Trove Guest Agent service - i read in some Trove books previously that there are dedicated agents for each and every database type, is it the same situation in Victoria release, or is there an "universal" Guest Agent covering all the database types nowadays? Where is the code that adapts the Agent commands towards the database instances placed inside the project?
Trove never has dedicated agents for each and every database type, it's using the same trove-guestagent but with different configurations for different datastores.
The backups - as i can see there seem to be some kind of dedicated docker-backup images involved in each database type. Could someone explain the internals of backup mechanisms inside Trove Victoria release in more details?
The backup container image is created to help trove-guestagent to achieve datastore-agnostic (so we only need to maintain a universal guest image), we shift the backup and restore functionalities and needed tools to a dedicated container. The implementation can be found here[4]. [4]: https://github.com/openstack/trove/tree/master/backup --- Lingxian Kong Senior Software Engineer Catalyst Cloud www.catalystcloud.nz
Hi! Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me. Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "Openstack Trove Essentials" by Alok Shrivastwa and Sunil Sarat that i purchased recently. For example, as mentioned in the book: - Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database. (page 6) - The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. (page 10) - As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. (page 58) When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function build_guest_image() is performed, involving those needed elements as minimal requirements: - ubuntu-minimal (which also invokes ubuntu-common i think) - cloud-init-datasources - pip-and-virtualenv - pip-cache - guest-agent - ${guest_os}-docker - root-passwd ref: https://github.com/openstack/trove/blob/master/integration/scripts/functions... So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in ubuntu-common element that disables privacy extensions for IPv6 (RFC4941): /usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041 I am really looking forward to our soon deployment of the Trove project, i see huge potential there! Best Regards Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/24/21 12:02 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Some questions regarding OpenStack Trove Victoria release Hi Bekir, I'm very happy to answer your questions. See me comments in line. On Sat, Jan 23, 2021 at 8:24 AM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: The questions: ------------------------ Having the fact that Trove Victoria release provides Docker containers as a new way of database instance provisioning, i am wondering how far the project is developed in terms of covering the different types of databases. What i can see by mainly parsing the code provided on Github, those seem to be officially released: - MySQL - MariaDB - PostgreSQL and the rest of the planned database types are in "experimental" phase. And also, regarding certain types of databases (for example MySQL, version 5.7 and 8.0) only certain versions of the datastores seems to be supported, but not all. MySQL 5.7.x is supported but 8.0 is in experimental. On the other hand, nothing regarding datastore versions supported for MariaDB and PostgreSQL seems to be mentioned somewhere. Could someone please confirm that as well as give some more details about it? MariaDB 10.4.x and PostgreSQL 12.4 are supported. The other versions need to be fully tested. I successfully managed to create certain versions of datastores in my devstack environment, belonging to those 3 database types mentioned above (and based on trovestack-generated dev guest image that is by default delivered with devstack installation), but not without some undesirable events. For example, i am able to register PostgreSQL datastore version 12 and instantiate a database instance of that version but not version 13 and above, where i get some hostname-related errors etc. Yes, because PostgreSQL 13 has never been tested. Also, a question regarding the building of the production-ready guest image. As mentioned, Trovestack script is provided as a possible way of producing the images (by omitting dev option the Trove Guest Agent binaries are deployed into the instantiated VM). How does an image produced this way looks like? From where the base image is fetched, is it a "cloud based image" with cloud-init in it, are the automatic security and software patching features disabled in produced image, so that we do not get unexpected service interruptions when the OS suddenly decides to start updating itself etc.. If you look at trovestack script implementation, you can see it's calling disk-image-create script from diskimage-builder, and there are some elements[2] defined in trove repo[3] for building the image. [1]: https://docs.openstack.org/diskimage-builder/latest [2]: https://docs.openstack.org/diskimage-builder/latest/elements.html [3]: https://github.com/openstack/trove/tree/master/integration/scripts/files/ele... Regarding the Trove Guest Agent service - i read in some Trove books previously that there are dedicated agents for each and every database type, is it the same situation in Victoria release, or is there an "universal" Guest Agent covering all the database types nowadays? Where is the code that adapts the Agent commands towards the database instances placed inside the project? Trove never has dedicated agents for each and every database type, it's using the same trove-guestagent but with different configurations for different datastores. The backups - as i can see there seem to be some kind of dedicated docker-backup images involved in each database type. Could someone explain the internals of backup mechanisms inside Trove Victoria release in more details? The backup container image is created to help trove-guestagent to achieve datastore-agnostic (so we only need to maintain a universal guest image), we shift the backup and restore functionalities and needed tools to a dedicated container. The implementation can be found here[4]. [4]: https://github.com/openstack/trove/tree/master/backup --- Lingxian Kong Senior Software Engineer Catalyst Cloud www.catalystcloud.nz
On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hi!
Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me.
Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "*Openstack Trove Essentials*" by Alok Shrivastwa and Sunil Sarat that i purchased recently.
For example, as mentioned in the book:
*- Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database.* (page 6) *- The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. *(page 10) *- As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. *(page 58)
Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different".
When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function *build_guest_image() *is performed, involving those needed elements as minimal requirements:
- *ubuntu-minimal *(which also invokes *ubuntu-common* i think) - *cloud-init-datasources* - *pip-and-virtualenv* - *pip-cache* - *guest-agent* - ${guest_os}-*docker* - *root-passwd*
ref:
https://github.com/openstack/trove/blob/master/integration/scripts/functions...
So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in *ubuntu-common* element that disables privacy extensions for IPv6 (RFC4941):
/usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041
You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove.
I am really looking forward to our soon deployment of the Trove project, i see huge potential there!
Good luck and please let me know if you have any other questions. --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) <http://www.catalystcloud.nz>
Hello again! One additional question here: Regarding the status of MSSQL provisioning inside the Trove project, is this abandoned or is it going to be provided in the future? Do You have any developers involved in development of this particular datastore type right now and how easy it would be to add it to the Trove Victoria concept? I guess most of the work should be done on modifying the Guest Agent to adapt to MSSQL, probably this part of the project: https://github.com/openstack/trove/tree/master/trove/guestagent If there is no ongoing development work towards integration of MSSQL into Trove right now, maybe we at City Network could contribute with that particular work by allocating suitable developer resources? Best Regards Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/26/21 10:51 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[2]: Some questions regarding OpenStack Trove Victoria release On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: Hi! Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me. Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "Openstack Trove Essentials" by Alok Shrivastwa and Sunil Sarat that i purchased recently. For example, as mentioned in the book: - Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database. (page 6) - The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. (page 10) - As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. (page 58) Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different". When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function build_guest_image() is performed, involving those needed elements as minimal requirements: - ubuntu-minimal (which also invokes ubuntu-common i think) - cloud-init-datasources - pip-and-virtualenv - pip-cache - guest-agent - ${guest_os}-docker - root-passwd ref: https://github.com/openstack/trove/blob/master/integration/scripts/functions... So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in ubuntu-common element that disables privacy extensions for IPv6 (RFC4941): /usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041 You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove. I am really looking forward to our soon deployment of the Trove project, i see huge potential there! Good luck and please let me know if you have any other questions. --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes)
Hello again, Bekir. There is no one currently working (or ever worked) on SQL Server support in Trove, but I don't think it is difficult to add. My only concern is, given the SQL Server is an enterprise database platform, how is it going to be tested in the upstream CI? --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Tue, Feb 16, 2021 at 10:27 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hello again!
One additional question here:
Regarding the status of MSSQL provisioning inside the Trove project, is this abandoned or is it going to be provided in the future? Do You have any developers involved in development of this particular datastore type right now and how easy it would be to add it to the Trove Victoria concept?
I guess most of the work should be done on modifying the Guest Agent to adapt to MSSQL, probably this part of the project:
https://github.com/openstack/trove/tree/master/trove/guestagent
If there is no ongoing development work towards integration of MSSQL into Trove right now, maybe we at City Network could contribute with that particular work by allocating suitable developer resources?
Best Regards
*Bekir Fajkovic* Senior DBA Mobile: +46 70 019 48 47
www.citynetwork.eu | www.citycloud.com
INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
----- Original Message ----- ------------------------------ From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/26/21 10:51 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[2]: Some questions regarding OpenStack Trove Victoria release
On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hi!
Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me.
Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "*Openstack Trove Essentials*" by Alok Shrivastwa and Sunil Sarat that i purchased recently.
For example, as mentioned in the book:
*- Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database.* (page 6) *- The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. *(page 10) *- As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. *(page 58)
Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different".
When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function *build_guest_image() *is performed, involving those needed elements as minimal requirements:
- *ubuntu-minimal *(which also invokes *ubuntu-common* i think) - *cloud-init-datasources* - *pip-and-virtualenv* - *pip-cache* - *guest-agent* - ${guest_os}-*docker* - *root-passwd*
ref:
https://github.com/openstack/trove/blob/master/integration/scripts/functions...
So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in *ubuntu-common* element that disables privacy extensions for IPv6 (RFC4941):
/usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041
You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove.
I am really looking forward to our soon deployment of the Trove project, i see huge potential there!
Good luck and please let me know if you have any other questions.
--- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) <http://www.catalystcloud.nz>
Hello! Yes, i am aware of the fact that MSSQL was previously provisioned only through a certain forks derived from the Trove project, prior to Victoria release, that's true. I assume You are aiming at licensing and cost issues in regards of testing and usage in general of the mssql as an enterprise platform? My vision is like this - there are already docker images ready for deployment of mssql 2017 and 2019 versions (all editions, Developer - Express - Standard - Enterprise - EnterpriseCore) and these are available for docker deployment through environment variables (MSSQL_PID in particular) where the product id is the edition chosen. If choosing a commercial edition, Microsoft assumes the SPLA or other license agreement already exists for the customer, so the pricing for the commercial instances should easily be figured out by the cloud hosting platform and existing license agreements with Microsoft. It is at least so if You try to deploy mssql on, say, Ubuntu today. After installation, user must execute a script that configures certain things and also deploys the suitable Edition chosen (and the license model as well). No need fot any license key registration is needed in this kind of scenario. When it comes to testing, Developer Edition of the product would be quite suitable since it is free of charge. Express Edition is free of charge as well but Developer Edition provides full functionality. Pure functionality testing can be carried out by, say, at least City Network, each time new things are deployed and improved for that particular datastore type. Honestly, although not open source and although it is highly commercial product, a huge army of the customers are expecting mostly that particular database type to be provided. :) Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 02/16/21 11:33 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[4]: Some questions regarding OpenStack Trove Victoria release Hello again, Bekir. There is no one currently working (or ever worked) on SQL Server support in Trove, but I don't think it is difficult to add. My only concern is, given the SQL Server is an enterprise database platform, how is it going to be tested in the upstream CI? --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Tue, Feb 16, 2021 at 10:27 PM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: Hello again! One additional question here: Regarding the status of MSSQL provisioning inside the Trove project, is this abandoned or is it going to be provided in the future? Do You have any developers involved in development of this particular datastore type right now and how easy it would be to add it to the Trove Victoria concept? I guess most of the work should be done on modifying the Guest Agent to adapt to MSSQL, probably this part of the project: https://github.com/openstack/trove/tree/master/trove/guestagent If there is no ongoing development work towards integration of MSSQL into Trove right now, maybe we at City Network could contribute with that particular work by allocating suitable developer resources? Best Regards Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/26/21 10:51 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[2]: Some questions regarding OpenStack Trove Victoria release On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: Hi! Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me. Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "Openstack Trove Essentials" by Alok Shrivastwa and Sunil Sarat that i purchased recently. For example, as mentioned in the book: - Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database. (page 6) - The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. (page 10) - As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. (page 58) Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different". When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function build_guest_image() is performed, involving those needed elements as minimal requirements: - ubuntu-minimal (which also invokes ubuntu-common i think) - cloud-init-datasources - pip-and-virtualenv - pip-cache - guest-agent - ${guest_os}-docker - root-passwd ref: https://github.com/openstack/trove/blob/master/integration/scripts/functions... So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in ubuntu-common element that disables privacy extensions for IPv6 (RFC4941): /usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041 You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove. I am really looking forward to our soon deployment of the Trove project, i see huge potential there! Good luck and please let me know if you have any other questions. --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes)
Yeah, if there is a way to test SQL Server integration for Trove in upstream CI, either using Developer Edition or 3rd party CI facility, I don't see there is any reason to block the contribution --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Wed, Feb 17, 2021 at 12:08 AM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hello!
Yes, i am aware of the fact that MSSQL was previously provisioned only through a certain forks derived from the Trove project, prior to Victoria release, that's true.
I assume You are aiming at licensing and cost issues in regards of testing and usage in general of the mssql as an enterprise platform?
My vision is like this - there are already docker images ready for deployment of mssql 2017 and 2019 versions (all editions, Developer - Express - Standard - Enterprise - EnterpriseCore) and these are available for docker deployment through environment variables (MSSQL_PID in particular) where the product id is the edition chosen. If choosing a commercial edition, Microsoft assumes the SPLA or other license agreement already exists for the customer, so the pricing for the commercial instances should easily be figured out by the cloud hosting platform and existing license agreements with Microsoft. It is at least so if You try to deploy mssql on, say, Ubuntu today. After installation, user must execute a script that configures certain things and also deploys the suitable Edition chosen (and the license model as well). No need fot any license key registration is needed in this kind of scenario.
When it comes to testing, Developer Edition of the product would be quite suitable since it is free of charge. Express Edition is free of charge as well but Developer Edition provides full functionality. Pure functionality testing can be carried out by, say, at least City Network, each time new things are deployed and improved for that particular datastore type.
Honestly, although not open source and although it is highly commercial product, a huge army of the customers are expecting mostly that particular database type to be provided.
:)
*Bekir Fajkovic* Senior DBA Mobile: +46 70 019 48 47
www.citynetwork.eu | www.citycloud.com
INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
----- Original Message ----- ------------------------------ From: Lingxian Kong (anlin.kong@gmail.com) Date: 02/16/21 11:33 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[4]: Some questions regarding OpenStack Trove Victoria release
Hello again, Bekir.
There is no one currently working (or ever worked) on SQL Server support in Trove, but I don't think it is difficult to add. My only concern is, given the SQL Server is an enterprise database platform, how is it going to be tested in the upstream CI?
--- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes)
On Tue, Feb 16, 2021 at 10:27 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hello again!
One additional question here:
Regarding the status of MSSQL provisioning inside the Trove project, is this abandoned or is it going to be provided in the future? Do You have any developers involved in development of this particular datastore type right now and how easy it would be to add it to the Trove Victoria concept?
I guess most of the work should be done on modifying the Guest Agent to adapt to MSSQL, probably this part of the project:
https://github.com/openstack/trove/tree/master/trove/guestagent
If there is no ongoing development work towards integration of MSSQL into Trove right now, maybe we at City Network could contribute with that particular work by allocating suitable developer resources?
Best Regards
*Bekir Fajkovic* Senior DBA Mobile: +46 70 019 48 47
www.citynetwork.eu | www.citycloud.com
INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
----- Original Message ----- ------------------------------ From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/26/21 10:51 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[2]: Some questions regarding OpenStack Trove Victoria release
On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hi!
Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me.
Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "*Openstack Trove Essentials*" by Alok Shrivastwa and Sunil Sarat that i purchased recently.
For example, as mentioned in the book:
*- Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database.* (page 6) *- The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. *(page 10) *- As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. *(page 58)
Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different".
When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function *build_guest_image() *is performed, involving those needed elements as minimal requirements:
- *ubuntu-minimal *(which also invokes *ubuntu-common* i think) - *cloud-init-datasources* - *pip-and-virtualenv* - *pip-cache* - *guest-agent* - ${guest_os}-*docker* - *root-passwd*
ref:
https://github.com/openstack/trove/blob/master/integration/scripts/functions...
So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in *ubuntu-common* element that disables privacy extensions for IPv6 (RFC4941):
/usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041
You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove.
I am really looking forward to our soon deployment of the Trove project, i see huge potential there!
Good luck and please let me know if you have any other questions.
--- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) <http://www.catalystcloud.nz>
OK. I guess the current contributors inside the Trove community are quite busy with their own ongoing projects, and if that is the situation then i may suggest that i try to allocate some of our internal resources with considerable python skills to contribute at least with mssql part inside Trove. Also, please tell me if You have other suggestions then so, regarding the approach of mssql contribution inside the project, and i will take that into consideration. Our desire is that we try to add that particular type of database inside the project as soon as possible, provided it all becomes approved. I assume it is important for us to go through the instructions describing the contribution mechanisms and maybe also establish conventional routines for testing. I am quite experienced in work with mssql as a senior dba in that area and i know the concepts of the product, how the product woks and what should be tested primarily. To begin with, perhaps a review should be made in order to figure out how the database engine should be adapted into the current Trove DBaaS concept, given it's specific way of applying the configuration parameters and such, then also input parameter to docker container that are crucial to provide during instance creation (collation, edition) etc. I would be more than happy if we could manage to start with all this in the near future. When it comes to one of the other questions that i raised in our previous conversation - to intentionally disable all kinds of automatic OS upgrades in the host VM in order to avoid the risks of unexpected disruptions in service delivery, my idea is to add a post install script inside 'ubuntu-guest' element that would disable the automatic updates. For example, to add a suitable executable bash script under: /trove/integration/scripts/files/elements/ubuntu-guest/post-install.d containing, for example similar code: -------------------------------------------------------------------------------------------- #!/bin/bash set -e set -o xtrace sed -i 's/"1"/"0"/g' /etc/apt/apt.conf.d/20auto-upgrades -------------------------------------------------------------------------------------------- that could be applied during instantiation of the VM from the guest image. That might possibly do the magic, or what is Your opinion? Then, applying the unattended updates could be carried out in more controllable fashion, instead letting the OS decide. This is particularly important in transaction intensive systems and similar.. Best Regards Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 02/17/21 03:19 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[6]: Some questions regarding OpenStack Trove Victoria release Yeah, if there is a way to test SQL Server integration for Trove in upstream CI, either using Developer Edition or 3rd party CI facility, I don't see there is any reason to block the contribution --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Wed, Feb 17, 2021 at 12:08 AM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: Hello! Yes, i am aware of the fact that MSSQL was previously provisioned only through a certain forks derived from the Trove project, prior to Victoria release, that's true. I assume You are aiming at licensing and cost issues in regards of testing and usage in general of the mssql as an enterprise platform? My vision is like this - there are already docker images ready for deployment of mssql 2017 and 2019 versions (all editions, Developer - Express - Standard - Enterprise - EnterpriseCore) and these are available for docker deployment through environment variables (MSSQL_PID in particular) where the product id is the edition chosen. If choosing a commercial edition, Microsoft assumes the SPLA or other license agreement already exists for the customer, so the pricing for the commercial instances should easily be figured out by the cloud hosting platform and existing license agreements with Microsoft. It is at least so if You try to deploy mssql on, say, Ubuntu today. After installation, user must execute a script that configures certain things and also deploys the suitable Edition chosen (and the license model as well). No need fot any license key registration is needed in this kind of scenario. When it comes to testing, Developer Edition of the product would be quite suitable since it is free of charge. Express Edition is free of charge as well but Developer Edition provides full functionality. Pure functionality testing can be carried out by, say, at least City Network, each time new things are deployed and improved for that particular datastore type. Honestly, although not open source and although it is highly commercial product, a huge army of the customers are expecting mostly that particular database type to be provided. :) Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 02/16/21 11:33 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[4]: Some questions regarding OpenStack Trove Victoria release Hello again, Bekir. There is no one currently working (or ever worked) on SQL Server support in Trove, but I don't think it is difficult to add. My only concern is, given the SQL Server is an enterprise database platform, how is it going to be tested in the upstream CI? --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Tue, Feb 16, 2021 at 10:27 PM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: Hello again! One additional question here: Regarding the status of MSSQL provisioning inside the Trove project, is this abandoned or is it going to be provided in the future? Do You have any developers involved in development of this particular datastore type right now and how easy it would be to add it to the Trove Victoria concept? I guess most of the work should be done on modifying the Guest Agent to adapt to MSSQL, probably this part of the project: https://github.com/openstack/trove/tree/master/trove/guestagent If there is no ongoing development work towards integration of MSSQL into Trove right now, maybe we at City Network could contribute with that particular work by allocating suitable developer resources? Best Regards Bekir Fajkovic Senior DBA Mobile: +46 70 019 48 47 www.citynetwork.eu | www.citycloud.com INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED ----- Original Message ----- From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/26/21 10:51 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[2]: Some questions regarding OpenStack Trove Victoria release On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic <bekir.fajkovic@citynetwork.eu> wrote: Hi! Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me. Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "Openstack Trove Essentials" by Alok Shrivastwa and Sunil Sarat that i purchased recently. For example, as mentioned in the book: - Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database. (page 6) - The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. (page 10) - As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. (page 58) Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different". When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function build_guest_image() is performed, involving those needed elements as minimal requirements: - ubuntu-minimal (which also invokes ubuntu-common i think) - cloud-init-datasources - pip-and-virtualenv - pip-cache - guest-agent - ${guest_os}-docker - root-passwd ref: https://github.com/openstack/trove/blob/master/integration/scripts/functions... So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in ubuntu-common element that disables privacy extensions for IPv6 (RFC4941): /usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041 You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove. I am really looking forward to our soon deployment of the Trove project, i see huge potential there! Good luck and please let me know if you have any other questions. --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes)
Hi Bekir, Thanks again for your interest in trove contribution. For SQL Server support, please just submit PoC code (including CI configuration) to verify your ideas and get familiar with the the upstream contribution process and the CI pipeline mechanism. For the particular issue of auto-upgrade, I agree with the fix. Do you mind creating a story here https://storyboard.openstack.org/#!/project/openstack/trove and submitting a patch for that? --- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) On Fri, Feb 19, 2021 at 4:04 AM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
OK.
I guess the current contributors inside the Trove community are quite busy with their own ongoing projects, and if that is the situation then i may suggest that i try to allocate some of our internal resources with considerable python skills to contribute at least with mssql part inside Trove. Also, please tell me if You have other suggestions then so, regarding the approach of mssql contribution inside the project, and i will take that into consideration. Our desire is that we try to add that particular type of database inside the project as soon as possible, provided it all becomes approved. I assume it is important for us to go through the instructions describing the contribution mechanisms and maybe also establish conventional routines for testing. I am quite experienced in work with mssql as a senior dba in that area and i know the concepts of the product, how the product woks and what should be tested primarily. To begin with, perhaps a review should be made in order to figure out how the database engine should be adapted into the current Trove DBaaS concept, given it's specific way of applying the configuration parameters and such, then also input parameter to docker container that are crucial to provide during instance creation (collation, edition) etc. I would be more than happy if we could manage to start with all this in the near future.
When it comes to one of the other questions that i raised in our previous conversation - to intentionally disable all kinds of automatic OS upgrades in the host VM in order to avoid the risks of unexpected disruptions in service delivery, my idea is to add a post install script inside 'ubuntu-guest' element that would disable the automatic updates.
For example, to add a suitable executable bash script under:
*/trove/integration/scripts/files/elements/ubuntu-guest/post-install.d* containing, for example similar code:
-------------------------------------------------------------------------------------------- #!/bin/bash set -e set -o xtrace sed -i 's/"1"/"0"/g' /etc/apt/apt.conf.d/20auto-upgrades
--------------------------------------------------------------------------------------------
that could be applied during instantiation of the VM from the guest image.
That might possibly do the magic, or what is Your opinion? Then, applying the unattended updates could be carried out in more controllable fashion, instead letting the OS decide. This is particularly important in transaction intensive systems and similar..
Best Regards
*Bekir Fajkovic* Senior DBA Mobile: +46 70 019 48 47
www.citynetwork.eu | www.citycloud.com
INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
----- Original Message ----- ------------------------------ From: Lingxian Kong (anlin.kong@gmail.com) Date: 02/17/21 03:19 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[6]: Some questions regarding OpenStack Trove Victoria release
Yeah, if there is a way to test SQL Server integration for Trove in upstream CI, either using Developer Edition or 3rd party CI facility, I don't see there is any reason to block the contribution
--- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes)
On Wed, Feb 17, 2021 at 12:08 AM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hello!
Yes, i am aware of the fact that MSSQL was previously provisioned only through a certain forks derived from the Trove project, prior to Victoria release, that's true.
I assume You are aiming at licensing and cost issues in regards of testing and usage in general of the mssql as an enterprise platform?
My vision is like this - there are already docker images ready for deployment of mssql 2017 and 2019 versions (all editions, Developer - Express - Standard - Enterprise - EnterpriseCore) and these are available for docker deployment through environment variables (MSSQL_PID in particular) where the product id is the edition chosen. If choosing a commercial edition, Microsoft assumes the SPLA or other license agreement already exists for the customer, so the pricing for the commercial instances should easily be figured out by the cloud hosting platform and existing license agreements with Microsoft. It is at least so if You try to deploy mssql on, say, Ubuntu today. After installation, user must execute a script that configures certain things and also deploys the suitable Edition chosen (and the license model as well). No need fot any license key registration is needed in this kind of scenario.
When it comes to testing, Developer Edition of the product would be quite suitable since it is free of charge. Express Edition is free of charge as well but Developer Edition provides full functionality. Pure functionality testing can be carried out by, say, at least City Network, each time new things are deployed and improved for that particular datastore type.
Honestly, although not open source and although it is highly commercial product, a huge army of the customers are expecting mostly that particular database type to be provided.
:)
*Bekir Fajkovic* Senior DBA Mobile: +46 70 019 48 47
www.citynetwork.eu | www.citycloud.com
INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
----- Original Message ----- ------------------------------ From: Lingxian Kong (anlin.kong@gmail.com) Date: 02/16/21 11:33 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[4]: Some questions regarding OpenStack Trove Victoria release
Hello again, Bekir.
There is no one currently working (or ever worked) on SQL Server support in Trove, but I don't think it is difficult to add. My only concern is, given the SQL Server is an enterprise database platform, how is it going to be tested in the upstream CI?
--- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes)
On Tue, Feb 16, 2021 at 10:27 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hello again!
One additional question here:
Regarding the status of MSSQL provisioning inside the Trove project, is this abandoned or is it going to be provided in the future? Do You have any developers involved in development of this particular datastore type right now and how easy it would be to add it to the Trove Victoria concept?
I guess most of the work should be done on modifying the Guest Agent to adapt to MSSQL, probably this part of the project:
https://github.com/openstack/trove/tree/master/trove/guestagent
If there is no ongoing development work towards integration of MSSQL into Trove right now, maybe we at City Network could contribute with that particular work by allocating suitable developer resources?
Best Regards
*Bekir Fajkovic* Senior DBA Mobile: +46 70 019 48 47
www.citynetwork.eu | www.citycloud.com
INNOVATION THROUGH OPEN IT INFRASTRUCTURE ISO 9001, 14001, 27001, 27015 & 27018 CERTIFIED
----- Original Message ----- ------------------------------ From: Lingxian Kong (anlin.kong@gmail.com) Date: 01/26/21 10:51 To: Bekir Fajkovic (bekir.fajkovic@citynetwork.eu) Cc: openstack-discuss (openstack-discuss@lists.openstack.org) Subject: Re: Re[2]: Some questions regarding OpenStack Trove Victoria release
On Tue, Jan 26, 2021 at 8:43 PM Bekir Fajkovic < bekir.fajkovic@citynetwork.eu> wrote:
Hi!
Thanks a lot for the kind answers and explanations, now the picture of the concept and the current development situation overall is much clearer to me.
Regarding the question about different types of Guest Agents acquired, depending on the database type, that i asked, it is mainly based on the information i read about in the latest edition of the book "*Openstack Trove Essentials*" by Alok Shrivastwa and Sunil Sarat that i purchased recently.
For example, as mentioned in the book:
*- Let's also look at the different types of guest agents that are required depending on the database engine that needs to be supported. The different guest agents (for example, the MySQL and PostgreSQL guest agents) may even have different capabilities depending on what is supported on the particular database.* (page 6) *- The Guest Agent code is different for every datastore that needs to be supported and the Guest Agent for that particular datastore is installed on the corresponding image of the datastore version. *(page 10) *- As we have already seen in the previous chapters, the guest agent is different for different database engines, and hence the correct version of the guest agent needs to be installed on the system. *(page 58)
Some of those have been changed and not the case any more. After database containerization, there is no database related stuff installed in the guest image. However, it's correct that different datastores are implemented as different drivers, so you can say "The Guest Agent code is different".
When it comes to guest image creation, i found now the places in the code that are used, as well as the acquired elements. A call to the function *build_guest_image() *is performed, involving those needed elements as minimal requirements:
- *ubuntu-minimal *(which also invokes *ubuntu-common* i think) - *cloud-init-datasources* - *pip-and-virtualenv* - *pip-cache* - *guest-agent* - ${guest_os}-*docker* - *root-passwd*
ref:
https://github.com/openstack/trove/blob/master/integration/scripts/functions...
So, when it comes to my question regarding the disabling of the automatic updates, it should be doable in a couple of ways. Either by executing a script placed in UserData during guest VM creation and initialisation or by manipulating elements (for example, such as we have a script placed in *ubuntu-common* element that disables privacy extensions for IPv6 (RFC4941):
/usr/local/lib/python3.6/dist-packages/diskimage_builder/elements/ubuntu-common/install.d/80-disable-rfc3041
You are right, but the recommended way is either those changes could be contributed back to the upstream if they are common feature requests and could benefit the others, or they are implemented in a separate element so there is little chance that conflict may happen when upgrading trove.
I am really looking forward to our soon deployment of the Trove project, i see huge potential there!
Good luck and please let me know if you have any other questions.
--- Lingxian Kong Senior Cloud Engineer (Catalyst Cloud) Trove PTL (OpenStack) OpenStack Cloud Provider Co-Lead (Kubernetes) <http://www.catalystcloud.nz>
On 2021-02-16 23:32:53 +1300 (+1300), Lingxian Kong wrote:
There is no one currently working (or ever worked) on SQL Server support in Trove, but I don't think it is difficult to add. My only concern is, given the SQL Server is an enterprise database platform, how is it going to be tested in the upstream CI? [...]
The way this is typically handled in other OpenStack projects is that someone with a vested interest in continued MSSQLS support in Trove would set up a "third-party CI system" which runs tests against an environment they maintain in their own network and reports back on proposed changes for openstack/trove or related repositories. The biggest challenge is finding someone with *enough* interest in MSSQLS support to commit to building and operating that CI system. -- Jeremy Stanley
participants (3)
-
Bekir Fajkovic
-
Jeremy Stanley
-
Lingxian Kong