<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Nov 2, 2015 at 10:49 AM, Paul Belanger <span dir="ltr"><<a href="mailto:pabelanger@redhat.com" target="_blank">pabelanger@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class=""><div class="h5">On Mon, Nov 02, 2015 at 10:27:26AM -0800, Clark Boylan wrote:<br>
> On Mon, Nov 2, 2015, at 07:58 AM, Paul Belanger wrote:<br>
> > Greetings,<br>
> ><br>
> > I'd like to start a thread talking about the effort to upgrade our<br>
> > version of<br>
> > puppet-mysql to a newer / latest version. I know there has been some talk<br>
> > on this already, would somebody mind adding some information?<br>
> ><br>
> > I have heard 3 things:<br>
> ><br>
> > 1. Remove database support from current modules, and only use<br>
> > sql_connection<br>
> > strings.<br>
> > 2. Move everything to trove<br>
> > 3. Setup mysql cluster<br>
> I think this is mostly orthogonal to updating the puppet-mysql module<br>
> because 2 + 1ish (use sql_connection) is basically what we do everywhere<br>
> but Jenkins slave image builds, <a href="http://paste.openstack.org" rel="noreferrer" target="_blank">paste.openstack.org</a>, and<br>
> <a href="http://wiki.openstack.org" rel="noreferrer" target="_blank">wiki.openstack.org</a>.<br>
> ><br>
> > Again, I don't know if there are true or not, just things I have seen<br>
> > people<br>
> > talk about.<br>
> ><br>
> > The obviously part that is missing, is _how_ we are going to do the<br>
> > upgrade. I<br>
> > know some people (clarkb?) already have some ideas on that.<br>
> ><br>
> > The reason for me asking, 2 weeks ago I offered to help the infra-cloud<br>
> > move<br>
> > forward and upgrading puppet-mysql was one of the items discussed. So,<br>
> > here I<br>
> > am, offering to do the grunt work, just need some understanding on what<br>
> > people<br>
> > want to do.<br>
> I was hoping that there was a forward and backward compatible<br>
> intermediate step we could make where old and new configs were supported<br>
> but I am told that isn't possible. As a result we will likely need to<br>
> update puppet-mysql, then all three of the above uses of puppet-mysql<br>
> semi atomically to keep everything working.<br>
><br>
> Testing that image builds work before the update is simple as we can<br>
> just run<br>
> openstack-infra/project-config/nodepool/scripts/prepare_node_bare.sh to<br>
> see if that works. Paste.o.o is simple and has gone from local Drizzle<br>
> to Trove to local MySQL without much fuss so I doubt it will have much<br>
> trouble but a test deployment is easy if we are worried.<br>
><br>
> The tricky one is going to be <a href="http://wiki.openstack.org" rel="noreferrer" target="_blank">wiki.openstack.org</a> and this is maybe where<br>
> the above list is relevant to the discussion. We could move it to an off<br>
> host database hosted by trove and not worry about updating puppet-mysql<br>
> in that module at all. Or we can test a deployment of wiki using the<br>
> newer mysql module before prod deployment. In either case we should<br>
> probably announce a wiki downtime prior to the upgrade, stop apache/php,<br>
> perform a database backup, switch to trove/run puppet with newer module,<br>
> restart apache/php.<br>
><br>
</div></div>So one of the big things I see, is once we bump puppet-mysql to the newer /<br>
latest version, all our current nodes will start consuming it (unless we stop<br>
puppet on each server).<br>
<br>
One thought I had was some like this:<br>
<br>
1. Create /opt/puppet/modules/old, install current puppet-mysql module into<br>
it.<br>
2. Remove /etc/puppet/modules/mysql and append --modulepath to now include<br>
/opt/puppet/modules/old<br>
3. Add dynamic logic to build modulepath for puppet-mysql based on flag and<br>
force all nodes to it.<br>
4. install puppet-mysql latest into /etc/puppet/modules.<br>
5. Start migration process, paste.o.o for example.<br>
6. Work through all nodes.<br>
7. Disable dynamic modulepath logic (can be used for other module upgrades).<br>
8. Delete /opt/puppet/modules/old/mysql<br>
<br>
Something like this, would allow us to some how control which nodes start<br>
consuming the newer version of puppet-mysql. Instead of a massive cutover for<br>
all our nodes.<br>
<div class=""><div class="h5"><br></div></div></blockquote><div>A possible simpler solution would be to use the load_module_metadata() function from the stdlib module[1]. This means writing logic into each of the modules that create a mysql::server to check for the module version and do the right thing accordingly, e.g.</div><div><br></div><div> $mysql_module_metadata = load_module_metadata('mysql', true)</div><div> if empty($mysql_module_metadata) { #mysql module is too old to support metadata.json</div><div> class { 'mysql::server':</div><div> config_hash => { 'root_password' => 'secret' }</div><div> }</div><div> } else { # we could check the actual version with $mysql_module_metadata['version'] or just assume it's new</div><div> class { 'mysql::server':</div><div> root_password => 'secret'</div><div> }</div><div> }</div><div><br></div><div>Right now the ability to not fail when metadata.json isn't found isn't released[2] so we'd have to update stdlib to the latest commit in master or pressure Puppet Labs into releasing sooner than they plan to.</div><div><br></div><div>Colleen</div><div><br></div><div>[1] <a href="https://github.com/puppetlabs/puppetlabs-stdlib#load_module_metadata">https://github.com/puppetlabs/puppetlabs-stdlib#load_module_metadata</a> </div><div>[2] <a href="https://github.com/puppetlabs/puppetlabs-stdlib/pull/537">https://github.com/puppetlabs/puppetlabs-stdlib/pull/537</a></div></div></div></div>