[openstack-dev] [all] Intermittent gate failures across multiple projects

John Villalovos openstack.org at sodarock.com
Thu Dec 22 23:40:38 UTC 2016


Gate jobs seem to be intermittently broken across all of OpenStack.

Bug filed:
https://bugs.launchpad.net/openstack-gate/+bug/1652186

They appear during the keystone phase of devstack and look like this:

http://logs.openstack.org/39/414339/2/check/gate-tempest-
dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/
1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_787

2016-12-22 22:25:08.607
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_607>
| 2016-12-22 22:25:08.606 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created domain
default2016-12-22 22:25:08.637
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_637>
| 2016-12-22 22:25:08.637 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created project
admin2016-12-22 22:25:08.657
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_657>
| 2016-12-22 22:25:08.656 31714 DEBUG passlib.registry
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] registered
'sha512_crypt' handler: <class
'passlib.handlers.sha2_crypt.sha512_crypt'> register_crypt_handler
/usr/local/lib/python2.7/dist-packages/passlib/registry.py:2942016-12-22
22:25:08.684 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_684>
| 2016-12-22 22:25:08.684 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created user
admin2016-12-22 22:25:08.700
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_700>
| 2016-12-22 22:25:08.699 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created role
admin2016-12-22 22:25:08.714
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_714>
| 2016-12-22 22:25:08.714 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Granted admin on
admin to user admin.2016-12-22 22:25:08.731
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_731>
| 2016-12-22 22:25:08.730 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created region
RegionOne2016-12-22 22:25:08.763
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_763>
| 2016-12-22 22:25:08.763 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created admin
endpoint http://15.184.67.84/identity_admin2016-12-22 22:25:08.775
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_775>
| 2016-12-22 22:25:08.774 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created internal
endpoint http://15.184.67.84/identity2016-12-22 22:25:08.787
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_787>
| 2016-12-22 22:25:08.787 31714 INFO keystone.cmd.cli
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Created public
endpoint http://15.184.67.84/identity2016-12-22 22:25:08.793
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_793>
| 2016-12-22 22:25:08.793 31714 INFO keystone.assignment.core
[req-f336127d-0628-4b9d-bcc7-a87863c3127e - - - - -] Creating the
default role 9fe2ff9ee4384b1894a90878d3e92bab because it does not
exist.2016-12-22 22:25:08.923
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_923>
| + ./stack.sh:main:1065                     :
create_keystone_accounts2016-12-22 22:25:08.927
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_927>
| + lib/keystone:create_keystone_accounts:371 :   local
admin_project2016-12-22 22:25:08.931
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_08_931>
| + lib/keystone:create_keystone_accounts:372 :   openstack project
show admin -f value -c id2016-12-22 22:25:10.787
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_787>
| /opt/stack/new/devstack/lib/keystone: line 366: 31727 Illegal
instruction     openstack project show "admin" -f value -c
id2016-12-22 22:25:10.791
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_791>
| + lib/keystone:create_keystone_accounts:372 :   true2016-12-22
22:25:10.796 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_10_796>
| ++ lib/keystone:create_keystone_accounts:373 :   openstack project
show admin -f value -c id2016-12-22 22:25:12.589
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_589>
| + lib/keystone:create_keystone_accounts:373 :
admin_project=2016-12-22 22:25:12.593
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_593>
| + lib/keystone:create_keystone_accounts:1  :   exit_trap2016-12-22
22:25:12.596 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_596>
| + ./stack.sh:exit_trap:487                 :   local r=1322016-12-22
22:25:12.602 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_602>
| ++ ./stack.sh:exit_trap:488                 :   jobs -p2016-12-22
22:25:12.606 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_606>
| + ./stack.sh:exit_trap:488                 :   jobs=310482016-12-22
22:25:12.610 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_610>
| + ./stack.sh:exit_trap:491                 :   [[ -n 31048
]]2016-12-22 22:25:12.614
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_614>
| + ./stack.sh:exit_trap:491                 :   [[ -n
/opt/stack/new/devstacklog.txt.2016-12-22-221622 ]]2016-12-22
22:25:12.617 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_617>
| + ./stack.sh:exit_trap:491                 :   [[ True == \T\r\u\e
]]2016-12-22 22:25:12.621
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_621>
| + ./stack.sh:exit_trap:492                 :   echo 'exit_trap:
cleaning up child processes'2016-12-22 22:25:12.621
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_621>
| exit_trap: cleaning up child processes2016-12-22 22:25:12.625
<http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_625>
| + ./stack.sh:exit_trap:493                 :   kill 310482016-12-22
22:25:12.625 <http://logs.openstack.org/39/414339/2/check/gate-tempest-dsvm-ironic-ipa-partition-pxe_snmp-tinyipa-ubuntu-xenial-nv/1d5e63d/logs/devstacklog.txt.gz#_2016-12-22_22_25_12_625>
| ./stack.sh: line 493: kill: (31048) - No such process

The "Illegal instruction" message is seen in this log because of this
test patch:
https://review.openstack.org/414341
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161222/5a8c7c6a/attachment.html>


More information about the OpenStack-dev mailing list