<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 8, 2019 at 8:00 PM Wesley Hayutin <<a href="mailto:whayutin@redhat.com">whayutin@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Seem the jobs go red at midnight May 9th UTC.<div><br></div><div>What I'm mainly seeing is the overcloud hang after the ssh keys are created, it seems the overcloud nodes do not have network connectivity. </div><div><a href="http://logs.openstack.org/47/657547/5/check/tripleo-ci-centos-7-containers-multinode/af82c9f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz">http://logs.openstack.org/47/657547/5/check/tripleo-ci-centos-7-containers-multinode/af82c9f/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz</a></div><div><br></div><div><a href="http://logs.openstack.org/47/657547/5/check/tripleo-ci-centos-7-containers-multinode/af82c9f/logs/subnode-2/var/log/extra/failed_services.txt.gz">http://logs.openstack.org/47/657547/5/check/tripleo-ci-centos-7-containers-multinode/af82c9f/logs/subnode-2/var/log/extra/failed_services.txt.gz</a><br></div><div><br></div><div>This looks normal eth0, eth1 come up</div><div><a href="http://logs.openstack.org/47/657547/5/check/tripleo-ci-centos-7-containers-multinode/af82c9f/logs/subnode-2/var/log/journal.txt.gz#_May_08_21_42_19">http://logs.openstack.org/47/657547/5/check/tripleo-ci-centos-7-containers-multinode/af82c9f/logs/subnode-2/var/log/journal.txt.gz#_May_08_21_42_19</a><br></div><div><br></div><div>I'm not 100% if this related to some of the latest patches, or if this impacts all jobs atm.</div><div>Looking into it.</div></div></blockquote><div><br></div><div>AFAICT, the issue is either with a few of the patches submitted or a blip in infra.</div><div>A clean test patch is working w/ multinode-containers and ovb fs001</div><div>CI should be green'ish quoting #tripleo </div><div><br></div><div>Sorry for the spam</div><div> </div></div></div>