<div dir="ltr"><span id="m_-3275140848134288534gmail-docs-internal-guid-cd137465-5a42-1fb9-6faf-c1eff8fc6f2e"><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Hello,</span></p><br><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">On December 13 we came the end of sprint using our new team structure, and here’s the highlights.</span></p><br><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">Sprint Review:</span></p><br><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">The goal on this sprint was to enable into quickstart a way to reproduce upstream jobs, in your personal RDO cloud tenant, making easy to developers to debug and reproduce their code.</span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Unfortunately due the RDO cloud upgrade and upstream infra issues, we weren’t able to finish successfully this sprint, having completed 4 cards, 5 that are waiting to be merged, and 3 blocked.</span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Nevertheless we have reviews [1] and documentation that can be used to reproduce the jobs upstream manually [2], and your feedback is more than welcome to improve it in our future automated script</span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">One can see the results of the sprint via <a href="https://trello.com/c/EWotWxGe/" target="_blank">https://trello.com/c/EWotWxGe/</a></span></p><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">Tripleo CI community meeting</span></p><br><ul style="margin-top:0pt;margin-bottom:0pt"><li>TLS Everywhere Job:<br></li><ul style="margin-top:0pt;margin-bottom:0pt"><li>Harry is working to have a periodic job running TLS</li><li>OVB (most recent hash) has the ability to deploy an extra node alongside the undercloud</li><ul style="margin-top:0pt;margin-bottom:0pt"><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://github.com/cybertron/openstack-virtual-baremetal/commit/c288a1ae973f3b32e9e4481f6604204386cbae9c" style="background-color:transparent;font-family:Arial;text-decoration-line:none" target="_blank">https://github.com/cybertron/<wbr>openstack-virtual-baremetal/<wbr>commit/<wbr>c288a1ae973f3b32e9e4481f660420<wbr>4386cbae9c</a></span></li><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://github.com/cybertron/openstack-virtual-baremetal/commit/2e4dd517d1736f004725ee29ac3e4764af246bab" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://github.com/cybertron/<wbr>openstack-virtual-baremetal/<wbr>commit/<wbr>2e4dd517d1736f004725ee29ac3e47<wbr>64af246bab</a></span></li></ul><li>Patch is live to deploy extra nodes in te-broker</li><ul><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://review.openstack.org/#/c/512899/" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://review.openstack.org/#<wbr>/c/512899/</a></span></li></ul></ul><li>OVB migration</li><ul><li>Blocked due RDO cloud upgrade</li></ul><li>RDO Cloud</li><ul><li>Upgrade process still in progress</li></ul><li>Kubernetes</li><ul><li>Will we see kubernetes in CI? Yes</li><li>There are several scenarios for kubernetes and openshift already</li><li>Kubernetes jobs are in check queue</li><li>Openshift jobs are in experimental</li><li>For more questions, please ping Flavio (flaper87)</li></ul><li>Upgrades related patches:</li><ul><li><a href="https://review.openstack.org/#/c/504822/" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none"><span style="color:rgb(0,0,0);background-color:transparent;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap">https://review.openstack.org/#<wbr>/c/504822/</span></a><span style="background-color:transparent;color:rgb(0,0,0);font-family:Arial;vertical-align:baseline;white-space:pre-wrap"> Support release change during playbook run</span></li><li><a href="https://review.openstack.org/#/c/504939/" target="_blank" style="background-color:transparent;white-space:pre-wrap;font-family:Arial">https://review.openstack.org/#<wbr>/c/504939/</a><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"> Set repo setup release in playbook</span></li></ul></ul><ul style="margin-top:0pt;margin-bottom:0pt"></ul><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">Ruck and Rover</span></p><br><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">What is Ruck and Rover</span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">One person in our team is designated Ruck and another Rover, one is responsible to monitoring the CI, checking for failures, opening bugs, participate on meetings, and this is your focal point to any CI issues. The other person, is responsible to work on these bugs, fix problems and the rest of the team are focused on the sprint. For more information about our structure, check [1]</span></p><br><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">List of bugs that Ruck and Rover were working on:</span></p><br><ul style="margin-top:0pt;margin-bottom:0pt"><li>Undercloud fails with " [ERROR] Could not lock /var/run/os-refresh-config.<wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">lock</span></li><ul style="margin-top:0pt;margin-bottom:0pt"><li>intermittently seen from upstream --> rdophase2.  posted patch to add elastic-recheck query to help id future instances</li><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://bugs.launchpad.net/tripleo/+bug/1669110" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://bugs.launchpad.net/<wbr>tripleo/+bug/1669110</a></span></li><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://review.openstack.org/#/c/527559" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://review.openstack.org/#<wbr>/c/527559</a></span></li></ul><li>Newton: RDO CI gate and phase1 promotion jobs fail with error "publicURL endpoint for workflowv2 service not found" because mistral is disabled</li><ul><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://bugs.launchpad.net/tripleo/+bug/1737502" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://bugs.launchpad.net/<wbr>tripleo/+bug/1737502</a></span></li></ul><li>http_proxy makes quickstart fail</li><ul><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://bugs.launchpad.net/tripleo/+bug/1736499" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://bugs.launchpad.net/<wbr>tripleo/+bug/1736499</a></span></li></ul><li>Need better logging for HA jobs</li><ul style="margin-top:0pt;margin-bottom:0pt"><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://bugs.launchpad.net/tripleo/+bug/1695237" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://bugs.launchpad.net/<wbr>tripleo/+bug/1695237</a></span></li><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://review.openstack.org/#/c/527554/" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://review.openstack.org/#<wbr>/c/527554/</a></span></li></ul><li>Remove mistral tempest tests jenkins-periodic-master-rdo_<wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">trunk-virtbasic-1ctlr_1comp_</span><wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">64gb</span></li><ul><li><span style="background-color:transparent;font-family:Arial;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a href="https://bugs.launchpad.net/tripleo/+bug/1736252" target="_blank" style="background-color:transparent;font-family:Arial;text-decoration-line:none">https://bugs.launchpad.net/<wbr>tripleo/+bug/1736252</a></span></li></ul><li>1737940 CI: neutron.tests.tempest.<wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">scenario.test_qos.QoSTest.</span><wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">test_qos fails in pike promotion jobs</span></li><li>Fix Released - #1737716 TripleO CI jobs fail: can't connect to nodepool hosts</li><li>#1737688 RDO CI quickstart job logs collection of dlrn logs is broken in tripleo-quickstart-extras-<wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">gate-master-tripleo-ci-</span><wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">delorean-full-minimal_</span><wbr style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial"><span style="background-color:transparent;white-space:pre-wrap;color:rgb(0,0,0);font-family:Arial">pacemaker</span></li><li>#1737617 master promoter script looping and failing since 12/9 (should put a 2 hr timeout on the promioter script  - to tech debt for the sprint team)</li><li>#1737568 CI: fs035 OVB job fails to run mysql_init_bundle container  ( still running on RH1 - could fix it there - contact HA people)</li><li>Fix Released - 1737502 Newton: RDO CI gate and phase1 promotion jobs fail with error "publicURL endpoint for workflowv2 service not found" because mistral is disabled</li><li>#1737485 CI: featureset020 (all tempest tests) promotion job fails with timeout on mtu test </li><li>Cron jobs of promotions scripts are commented, please uncomment them when RDO cloud is online and stable</li></ul><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">We also have our new Ruck and Rover for this week:</span></p><br><ul style="margin-top:0pt;margin-bottom:0pt"><li>Ruck</li><ul><li>John Trownbridge - trown|ruck</li></ul><li>Rover</li><ul><li>Ronelle Landy - rlandy|rover</li></ul></ul><br><p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">If you have any questions and/or suggestions, please contact us</span></p><br><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">[1]</span><a href="https://review.openstack.org/#/c/509280/" style="text-decoration-line:none" target="_blank"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"> </span><span style="font-family:Arial;background-color:transparent;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap">https://review.openstack.org/#<wbr>/c/509280/</span></a></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">[2] <a href="https://review.openstack.org/#/c/525743/" target="_blank">https://review.openstack.org/#<wbr>/c/525743/</a></span></p><p dir="ltr" style="line-height:1.656;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">[3] <a href="https://etherpad.openstack.org/p/ruck-rover-reproduce-jobs" target="_blank">https://etherpad.openstack.<wbr>org/p/ruck-rover-reproduce-<wbr>jobs</a></span></p><div><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><br></span></div></span></div>