<div dir="ltr"><div style="font-family:courier new,monospace" class="gmail_default">The email thread below is an one that I have been having with Luke and some others at IBM who have been working on Trove.</div><div style="font-family:courier new,monospace" class="gmail_default"><br></div><div style="font-family:courier new,monospace" class="gmail_default">I feel that these kind(s) of conversations are better had in the mailing list, and in replying to this email, I let the participants know that absent any objections, I would like to continue this conversation on the ML. Hearing nothing in the past two days, I am forwarding this to the ML.</div><div style="font-family:courier new,monospace" class="gmail_default"><br></div><div style="font-family:courier new,monospace" class="gmail_default">-amrith<br></div><div style="font-family:courier new,monospace" class="gmail_default"></div><br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div><div class="m_5781699149525160981m_9044739750522956958gmail-m_383450970643444482gmail_signature"></div></div>
<br><div class="gmail_quote"><span class="">On Thu, Aug 17, 2017 at 4:24 PM, Luke Browning <span dir="ltr"><<a href="mailto:LukeBrowning@us.ibm.com" target="_blank">LukeBrowning@us.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">Hi Amrith,</font><br><br><font size="2">I understand that the Trove project will be going into maintenance mode soon. I have a number of Trove related enhancements and bug fixes that I would like to submit to the community, but I don't know if they would be considered given that the project is going into maintenance mode. Please elaborate on what you mean by maintenance mode.</font></p></div></blockquote><div><br></div></span><div>See: <a href="https://review.openstack.org/#/c/488947/" target="_blank">https://review.openstack.org/#<wbr>/c/488947/</a><br></div><div><br></div><div>It is not a foregone conclusion that the project will go into maintenance mode. Thierry and I (for example) are not convinced that this is the right course of action, but if there are no offers to contribute to the project, it may be a decision by default.<br></div><div><br></div><div>Do you subscribe to the OpenStack-dev mailing list. Please also see <a href="http://openstack.markmail.org/thread/4p6gp263fei4mbru" target="_blank">http://openstack.markmail.org/<wbr>thread/4p6gp263fei4mbru</a></div><div><br></div><div>Finally, for a description of what maintenance mode means, please refer: <a href="https://governance.openstack.org/tc/reference/tags/status_maintenance-mode.html" target="_blank">https://governance.openstack.o<wbr>rg/tc/reference/tags/status_ma<wbr>intenance-mode.html</a><br></div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><br><font size="2">Would Trove be updated to support new OpenStack versions?</font></p></div></blockquote><div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">Would support be provided for Trove gate testing?</font><br><font size="2"></font></p></div></blockquote><div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">Would elements be enhanced to support Xenial and newer versions of databases? </font><br><font size="2"></font></p></div></blockquote><div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">Is there a time limit associated with maintenance mode? For example, would it remain in effect for a year or two after the new stuff is released? </font><br><br></p></div></blockquote></span><div><div>For the four questions above, I direct your attention to the definition of the maintenance-mode tag (<a href="https://governance.openstack.org/tc/reference/tags/status_maintenance-mode.html" target="_blank">https://governance.openstack.<wbr>org/tc/reference/tags/status_m<wbr>aintenance-mode.html</a>) and remind you that support for different versions from the community, for gate testing and for elements is dependent on people participating (and their employers allowing them to do this). At this point, I have no one who has stepped up to do this in any significant way. There are a couple of people from China Mobile who want to help but who are largely in the weeds, trying to fix this and that bug but have no idea what Trove is all about. IBM has a core reviewer on the project (Mariam is copied on this email). I am happy that she's able to devote what time she can to the project but absent competent reviewers, whether your changes ever get merged or not remains an interesting question. Since you will be contributing elements and propose to contribute a tool, will you be (or will IBM be) offering to support it?<br></div><br></div><span class=""><div> <font size="2">Let me provide some back ground on my code changes, so you will have a better idea of what might be submitted in a patch. </font><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><font size="2">I have written a fully automated command that creates a virtual guest image based on Ubuntu 16.04 containing a user specified version of a database. The user's selection is validated against an internal list of databases built into the tool. Once validated, the tool creates the image, registers it with Glance, and then creates a Trove Datastore definition for it. This is done in a single command invocation that typically takes about 25 minutes to complete. For some of the databases, it takes considerably longer (2 hours for mongodb) as I have to compile source code. The guest image that is produced is complete from a Trove perspective. It includes the Cloud specific trove-guestagent.conf file and SSH public keys for the DBA and OpenStack controller nodes. </font><br><br></p></div></blockquote></span><div><div>What mechanism does it use to develop the image? is it diskimage-builder or some other?</div><div><br></div><div><br></div><div>Is there something the matter with the existing tool that exists, it already works, it does more than just build an image, and it is already integrated in the gate. Why not use that?</div> </div><div><div></div><div>This approach is nice (having a full guest image with the guest agent and all that but as you observe, it takes about 2 hours per build and from a developers perspective, this is a pain.</div><div><br></div><div>I assume that the tool is therefore for production use cases.</div><div><br></div><div>Could the elements that you have developed be used with the existing tool or is your tool a complete replacement for the one that already exists?</div></div><div><br></div><div><div>Is there some reason that this tool was not developed in the open, using the normal open development process?</div><div><br></div><div>There are ways to accelerate the compiling of source code and such through the use of image layers (if you are using DIB). I have elements and code that will build most of the trove elements in minutes and I'm waiting to see whether there is any point in contributing them to the upstream or not, at this point. I could show you how to do that, and it would be awesome to have you build your tool into the build process somehow. That would be a win-win for all ...</div><br></div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">I also have commands that setup Nova flavors on a per database basis. These flavors can be customized by the user before they are instantiated in the cloud.</font><br></p></div></blockquote></span><div><div>The existing command (trovestack) already does this.</div> </div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><font size="2">I wouldn't expect to drop these commands to the Trove community. Only the changes to the Trove-guestagent and elements enabling databases. </font><br><br><font size="2">Here's the list of databases that are currently supported for ppc64le. In some cases, I have had to add Xenial support. </font><br><br><font size="2">mariadb 10.1 --- package comes from the community</font><br><font size="2">mongodb 3.4 community edition --- source code from the community is compiled as there is no ppc64le package</font><br><font size="2">mongodb 3.4 enterprise edition --- package comes from community</font><br><font size="2">mysql 5.7 --- package comes from Ubuntu 16.04. percona-xtrabackup-24 is compiled as there is no power support in community</font><br><font size="2">postgresql 9.5 --- package comes from Ubuntu 16.04 </font><br><font size="2">postgresql 9.6 --- package comes from community</font><br><font size="2">redis 3.0 --- package comes from Ubuntu 16.04</font><br><font size="2">redis 3.2 --- source code from the community is compiled</font></p></div></blockquote></span><div><div>Why not just contribute the elements to the existing project and have the current tool use them?</div> </div><div><div>What of other databases currently supported? galera cluster (pxc), vertica, db2, percona, couchdb, and couchbase?</div><br></div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><br><font size="2">My element related code changes add environment variables that control the database version, the source of the database software (distro, community, enterprise), and logging options. In this way, the logic in the element is more generalized and can be made to support multiple versions. I also added an environment variable that specifies the name of the source package to be downloaded and compiled when that is applicable. I also have environment variables to enable database and trove guest agent logging. </font></p></div></blockquote></span><div><div>Which is nice except that your guest agent code may or may not know how to deal with that.</div> </div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><br><font size="2">My trove-guestagent code changes are mostly about not reconfiguring the database with configuration data remotely passed to it from the task manager. My code changes are isolated to the prepare step. This logic looks something like: if database version x, add or remove configuration parameter. Note there is relatively very little of this code, ~150 LOCs, as database configuration parameters rarely change. Note there is already some of this DB version specific logic in the trove guest agent, so I am just adding more of it. But I believe a better way is not to alter the database configuration data when the database is fully configured in a Trove compliant manner via elements. This would be implemented through a new command argument to the Trove Guest agent which says ignore configuration changes. It would be set by the elements as well.</font></p></div></blockquote></span><div><div>Hard to say whether this is a good approach or not with out more detail.</div> </div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><br><font size="2">I haven't looked into the scale out configuration data yet, but this can be handled differently as required. </font><br><br></p></div></blockquote></span><div><div>Not sure what you mean.</div><div><br></div><div></div> </div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">So, are these the sorts of changes that I could add in a maintenance release? </font><br><br></p></div></blockquote></span><div><div>Maintenance release of what, trove? Are you asking whether these changes can be made in a stable branch? if yes, which branch? See <a href="https://docs.openstack.org/project-team-guide/stable-branches.html" target="_blank">https://docs.openstack.org/pro<wbr>ject-team-guide/stable-branche<wbr>s.html</a> and the section "Appropriate Fixes".</div> </div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><font size="2">1) conditionally disable the dynamic installation and configuration of database software in the trove-guest agent via a new command argument </font><br><font size="2">2) add environment variables to trove elements enabling multiple versions of databases to be installed and configured</font><br><font size="2">3) add environment variables to trove elements to enable database and trove-guest agent logging</font><br><font size="2">4) add xenial support to elements</font><br><font size="2">5) compiling source code in elements for missing stack elements for ppc64le </font><br><br><font size="2">These changes are implemented at: </font><a href="https://github.com/open-power-ref-design-toolkit/os-services/tree/master/osa/dbaas/dbimage-builder" target="_blank"><font size="2">https://github.com/open-power-<wbr>ref-design-toolkit/os-services<wbr>/tree/master/osa/dbaas/dbimage<wbr>-builder</font></a></p></div></blockquote></span><div><div>OK, so this is a whole new tool, sure you can propose it but how do we make sure that it gets integrated into the current testing/gate? If you want to drop this then I'd also have to assume that there would be some code that would make it part of the current gate, and ideally make it generate the images for the current gate. But for that, the fact that you install the guest agent onto the image is a serious drag.</div><div><br></div><div><br></div><div>Also, why this wasn't proposed as a blueprint or spec in the upstream so it could be developed in the open, I don't quite understand. After all, you are now showing up with a complete tool and asking that it be included which largely runs counter to the whole notion of open development, at least as I understand it.</div> </div><div><div></div><div>Finally, once committed, do you (or IBM) commit to maintaining this code and fixing bugs in it should they arise, and upgrade/update it from time to time?<br></div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><p><br><br><font size="2">Thanks, Luke Browning</font><br><br>
</p></div>
</blockquote></div><br></div></div>
</blockquote></div><br></div></div>