<div dir="ltr">Gate jobs seem to be intermittently broken across all of OpenStack.<br><div class="gmail_quote"><div dir="ltr"><div><br>Bug filed:<br><a href="https://bugs.launchpad.net/openstack-gate/+bug/1652186" target="_blank">https://bugs.launchpad.net/<wbr>openstack-gate/+bug/1652186</a><br><br></div>They appear during the keystone phase of devstack and look like this:<br><br><a href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_787" target="_blank">http://logs.openstack.org/39/<wbr>414339/2/check/gate-tempest-<wbr>dsvm-ironic-ipa-partition-pxe_<wbr>snmp-tinyipa-ubuntu-xenial-nv/<wbr>1d5e63d/logs/devstacklog.txt.<wbr>gz#_2016-12-22_22_25_10_787</a><br><br><pre><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_607"><a name="m_7862380079793542757__2016-12-22_22_25_08_607" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_607" target="_blank">2016-12-22 22:25:08.607</a> | 2016-12-22 22:25:08.606 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created domain default
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_637"><a name="m_7862380079793542757__2016-12-22_22_25_08_637" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_637" target="_blank">2016-12-22 22:25:08.637</a> | 2016-12-22 22:25:08.637 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created project admin
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_657"><a name="m_7862380079793542757__2016-12-22_22_25_08_657" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_657" target="_blank">2016-12-22 22:25:08.657</a> | 2016-12-22 22:25:08.656 31714 DEBUG passlib.registry [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] registered 'sha512_crypt' handler: <class 'passlib.handlers.sha2_crypt.<wbr>sha512_crypt'> register_crypt_handler /usr/local/lib/python2.7/dist-<wbr>packages/passlib/registry.py:<wbr>294
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_684"><a name="m_7862380079793542757__2016-12-22_22_25_08_684" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_684" target="_blank">2016-12-22 22:25:08.684</a> | 2016-12-22 22:25:08.684 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created user admin
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_700"><a name="m_7862380079793542757__2016-12-22_22_25_08_700" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_700" target="_blank">2016-12-22 22:25:08.700</a> | 2016-12-22 22:25:08.699 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created role admin
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_714"><a name="m_7862380079793542757__2016-12-22_22_25_08_714" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_714" target="_blank">2016-12-22 22:25:08.714</a> | 2016-12-22 22:25:08.714 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Granted admin on admin to user admin.
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_731"><a name="m_7862380079793542757__2016-12-22_22_25_08_731" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_731" target="_blank">2016-12-22 22:25:08.731</a> | 2016-12-22 22:25:08.730 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created region RegionOne
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_763"><a name="m_7862380079793542757__2016-12-22_22_25_08_763" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_763" target="_blank">2016-12-22 22:25:08.763</a> | 2016-12-22 22:25:08.763 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created admin endpoint <a href="http://15.184.67.84/identity_admin" target="_blank">http://15.184.67.84/identity_<wbr>admin</a>
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_775"><a name="m_7862380079793542757__2016-12-22_22_25_08_775" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_775" target="_blank">2016-12-22 22:25:08.775</a> | 2016-12-22 22:25:08.774 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created internal endpoint <a href="http://15.184.67.84/identity" target="_blank">http://15.184.67.84/identity</a>
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_787"><a name="m_7862380079793542757__2016-12-22_22_25_08_787" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_787" target="_blank">2016-12-22 22:25:08.787</a> | 2016-12-22 22:25:08.787 31714 INFO keystone.cmd.cli [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Created public endpoint <a href="http://15.184.67.84/identity" target="_blank">http://15.184.67.84/identity</a>
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_793"><a name="m_7862380079793542757__2016-12-22_22_25_08_793" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_793" target="_blank">2016-12-22 22:25:08.793</a> | 2016-12-22 22:25:08.793 31714 INFO keystone.assignment.core [req-f336127d-0628-4b9d-bcc7-<wbr>a87863c3127e - - - - -] Creating the default role 9fe2ff9ee4384b1894a90878d3e92b<wbr>ab because it does not exist.
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_923"><a name="m_7862380079793542757__2016-12-22_22_25_08_923" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_923" target="_blank">2016-12-22 22:25:08.923</a> | + ./stack.sh:main:1065 : create_keystone_accounts
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_927"><a name="m_7862380079793542757__2016-12-22_22_25_08_927" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_927" target="_blank">2016-12-22 22:25:08.927</a> | + lib/keystone:create_keystone_<wbr>accounts:371 : local admin_project
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_08_931"><a name="m_7862380079793542757__2016-12-22_22_25_08_931" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_931" target="_blank">2016-12-22 22:25:08.931</a> | + lib/keystone:create_keystone_<wbr>accounts:372 : openstack project show admin -f value -c id
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_10_787 m_7862380079793542757gmail-highlight"><a name="m_7862380079793542757__2016-12-22_22_25_10_787" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_787" target="_blank">2016-12-22 22:25:10.787</a> | /opt/stack/new/devstack/lib/<wbr>keystone: line 366: 31727 Illegal instruction openstack project show "admin" -f value -c id
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_10_791"><a name="m_7862380079793542757__2016-12-22_22_25_10_791" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_791" target="_blank">2016-12-22 22:25:10.791</a> | + lib/keystone:create_keystone_<wbr>accounts:372 : true
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_10_796"><a name="m_7862380079793542757__2016-12-22_22_25_10_796" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_796" target="_blank">2016-12-22 22:25:10.796</a> | ++ lib/keystone:create_keystone_<wbr>accounts:373 : openstack project show admin -f value -c id
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_589"><a name="m_7862380079793542757__2016-12-22_22_25_12_589" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_589" target="_blank">2016-12-22 22:25:12.589</a> | + lib/keystone:create_keystone_<wbr>accounts:373 : admin_project=
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_593"><a name="m_7862380079793542757__2016-12-22_22_25_12_593" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_593" target="_blank">2016-12-22 22:25:12.593</a> | + lib/keystone:create_keystone_<wbr>accounts:1 : exit_trap
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_596"><a name="m_7862380079793542757__2016-12-22_22_25_12_596" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_596" target="_blank">2016-12-22 22:25:12.596</a> | + ./stack.sh:exit_trap:487 : local r=132
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_602"><a name="m_7862380079793542757__2016-12-22_22_25_12_602" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_602" target="_blank">2016-12-22 22:25:12.602</a> | ++ ./stack.sh:exit_trap:488 : jobs -p
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_606"><a name="m_7862380079793542757__2016-12-22_22_25_12_606" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_606" target="_blank">2016-12-22 22:25:12.606</a> | + ./stack.sh:exit_trap:488 : jobs=31048
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_610"><a name="m_7862380079793542757__2016-12-22_22_25_12_610" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_610" target="_blank">2016-12-22 22:25:12.610</a> | + ./stack.sh:exit_trap:491 : [[ -n 31048 ]]
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_614"><a name="m_7862380079793542757__2016-12-22_22_25_12_614" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_614" target="_blank">2016-12-22 22:25:12.614</a> | + ./stack.sh:exit_trap:491 : [[ -n /opt/stack/new/devstacklog.<wbr>txt.2016-12-22-221622 ]]
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_617"><a name="m_7862380079793542757__2016-12-22_22_25_12_617" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_617" target="_blank">2016-12-22 22:25:12.617</a> | + ./stack.sh:exit_trap:491 : [[ True == \T\r\u\e ]]
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_621"><a name="m_7862380079793542757__2016-12-22_22_25_12_621" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_621" target="_blank">2016-12-22 22:25:12.621</a> | + ./stack.sh:exit_trap:492 : echo 'exit_trap: cleaning up child processes'
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_621"><a name="m_7862380079793542757__2016-12-22_22_25_12_621" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_621" target="_blank">2016-12-22 22:25:12.621</a> | exit_trap: cleaning up child processes
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_625"><a name="m_7862380079793542757__2016-12-22_22_25_12_625" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_625" target="_blank">2016-12-22 22:25:12.625</a> | + ./stack.sh:exit_trap:493 : kill 31048
</span><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_625"><a name="m_7862380079793542757__2016-12-22_22_25_12_625" class="m_7862380079793542757gmail-date" href="http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_625" target="_blank">2016-12-22 22:25:12.625</a> | ./stack.sh: line 493: kill: (31048) - No such process<br><br></span></pre><pre><span class="m_7862380079793542757gmail-NONE m_7862380079793542757gmail-_2016-12-22_22_25_12_625">The "Illegal instruction" message is seen in this log because of this test patch:<br><a href="https://review.openstack.org/414341" target="_blank">https://review.openstack.org/<wbr>414341</a> <br></span></pre><div><br><br></div></div>
<br></div><br></div>