On 06/23/2016 02:56 PM, Dan Prince wrote: > After discovering some regressions today we found what we think is a > package build issue in our CI environment which might be the cause of > our issues: > > https://bugs.launchpad.net/tripleo/+bug/1595660 > > Specifically, there is a case where DLRN might not be giving an error > code if build failures occur, and thus our jobs don't get the updated > package symlink and thus give a false positive. > > Until we get this solved be careful when merging. You might look for > 'packages not built correctly: not updating the consistent symlink' in > the job output. I see over 200 of these in the last 24 hours: > I updated the bug, but will reply here for completeness. The "not updating the consistent symlink" message appears 100% of the time when not building all packages in rdoinfo. Instead what happened is we built HEAD of master instead of the refspec from zuul. http://logs.openstack.org/17/324117/22/check-tripleo/gate-tripleo-ci-centos-7-nonha/3758449/console.html#_2016-06-23_03_40_49_234238 c48410a05ec0ffd11c717bcf350badc9e5f0e910 is the commit it should have built. 4ef338574b1a7cef8b1b884d439556b24fb09718 was built instead. So the logstash query we could use is instead: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22cleaning%20directory%20and%20cloning%20again%5C%22%20AND%20filename%3A%5C%22console.html%5C%22 I think https://review.rdoproject.org/r/1500 is the fix.