Stable check of openstack/tempest failed
Build failed. - periodic-tempest-dsvm-full-havana http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-full-havana/... : FAILURE in 11m 49s - periodic-tempest-dsvm-neutron-havana http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-neutron-hava... : FAILURE in 11m 32s - periodic-tempest-dsvm-postgres-full-havana http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-postgres-ful... : FAILURE in 11m 26s - periodic-tempest-dsvm-full-icehouse http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-full-icehous... : SUCCESS in 46m 02s - periodic-tempest-dsvm-neutron-icehouse http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-neutron-iceh... : SUCCESS in 56m 07s - periodic-tempest-dsvm-postgres-full-icehouse http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-postgres-ful... : SUCCESS in 44m 56s
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Havana jobs failed, probably due to new oslo.config release. Since tags for the last release are already created, we won't fix those failures. On 22/09/14 09:14, jenkins@openstack.org wrote:
Build failed.
- periodic-tempest-dsvm-full-havana http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-full-havana/... : FAILURE in 11m 49s - periodic-tempest-dsvm-neutron-havana http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-neutron-hava... : FAILURE in 11m 32s - periodic-tempest-dsvm-postgres-full-havana http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-postgres-ful... : FAILURE in 11m 26s - periodic-tempest-dsvm-full-icehouse http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-full-icehous... : SUCCESS in 46m 02s - periodic-tempest-dsvm-neutron-icehouse http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-neutron-iceh... : SUCCESS in 56m 07s - periodic-tempest-dsvm-postgres-full-icehouse http://logs.openstack.org/periodic-stable/periodic-tempest-dsvm-postgres-ful... : SUCCESS in 44m 56s
_______________________________________________ Openstack-stable-maint mailing list Openstack-stable-maint@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-stable-maint
-----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUH+NnAAoJEC5aWaUY1u57wjoIALLj4SLwF9KUKGg9PvPcd0Gf zQNzY8JPdv9/lSKgqx8dwkbONb/l4tvOwr76pk3t4FE+xvduUaEmctuixjgey9em 0teq67BiDKow+KQgzk3CAZ0/BdT0ySH2WYZEWfGCVVHHmvIPbC844dWA1IDeWmLF lhmxtMiFLUrOL9e+5+oiV0VmQuhHmtTKKFTf4rFQI2BC2+7e5R2cgO5VR9cb4xBj CLkEBKyLyMfFohm9RnIaZT+u9BawjwmJOg5vmc4PEkpflsIFLZcI9jUKjfyVq+9o UwjqZfUEbomUvZoY1q9IVaZ7WGTRtJWlTmbroXqM3XCALmLPEXd+nwf8t8JjzdM= =FLGO -----END PGP SIGNATURE-----
2014-09-22 10:52 GMT+02:00 Ihar Hrachyshka <ihrachys@redhat.com>:
Havana jobs failed, probably due to new oslo.config release. Since tags for the last release are already created, we won't fix those failures.
That's correct, 2013.2.4 tags have been pushed and we're not fixing this in Havana but it should be investigated. Oslo releases are supposed to be backward compatible so we need to understand if that's an oslo.config regression or not. Cheers, Alan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 22/09/14 11:23, Alan Pevec wrote:
2014-09-22 10:52 GMT+02:00 Ihar Hrachyshka <ihrachys@redhat.com>:
Havana jobs failed, probably due to new oslo.config release. Since tags for the last release are already created, we won't fix those failures.
That's correct, 2013.2.4 tags have been pushed and we're not fixing this in Havana but it should be investigated. Oslo releases are supposed to be backward compatible so we need to understand if that's an oslo.config regression or not.
I suspect this may be due to oslo-config-options-validation work done in 1.4.0 [1]. That said, the failure is observed in glance/havana only, and logs are not very informative. The only thing that is present is the following critical error in glance-api log (no traceback): 2014-09-22 06:27:04.281 13352 CRITICAL glance [-] 'StrOpt' object has no attribute 'type' [1]: https://review.openstack.org/gitweb?p=openstack%2Foslo.config.git;a=commitdi... /Ihar -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUH/xmAAoJEC5aWaUY1u571roIAN8x4KgPkJl41dqeKlnG5kD8 6wanTpixRu6Sl+pPbUU9M5zvH9dhY0RuGaMYA4OM1gjicR2vjfeWeVZqooZKvgor uNNkcg1NHqgKNTJNI+wf9DbljRwrwwdkHBfSgcqjO+NAYq3flAKEXP5taOJkJePk zUC8QeKEXbYTqmCgY4hxrTnkBMCeEoscIpIqbXFUNiuR3xafdmp1FzG31Oao+rKi kLmQNXYifduowFcVBfc6+KyIKQUhs3Uws7HIOINdCa2NKoCALAYuZTY+rCe9Okl3 lj2RJStGEtxLj+IJ/48uKAiMl4kn88Y43ufTPXqmHISFthUxYOr6nm6rcfCRC+8= =V/jK -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 (Fixing the wrong Doug's email address.) On 22/09/14 11:23, Alan Pevec wrote:
2014-09-22 10:52 GMT+02:00 Ihar Hrachyshka <ihrachys@redhat.com>:
Havana jobs failed, probably due to new oslo.config release. Since tags for the last release are already created, we won't fix those failures.
That's correct, 2013.2.4 tags have been pushed and we're not fixing this in Havana but it should be investigated. Oslo releases are supposed to be backward compatible so we need to understand if that's an oslo.config regression or not.
I suspect this may be due to oslo-config-options-validation work done in 1.4.0 [1]. That said, the failure is observed in glance/havana only, and logs are not very informative. The only thing that is present is the following critical error in glance-api log (no traceback): 2014-09-22 06:27:04.281 13352 CRITICAL glance [-] 'StrOpt' object has no attribute 'type' [1]: https://review.openstack.org/gitweb?p=openstack%2Foslo.config.git;a=commitdi... /Ihar -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iQEcBAEBCgAGBQJUH/yhAAoJEC5aWaUY1u57WDoIAMMYBpio1k0iYVoQyFAi5BdI 306BXUsywij1DBdTdbEwqiRET2yAmFPuD4IaYXX0aeVu2quHvRxvzGCLRZQa6jXS I5ICFlOzBdCBqdauVXzqiipNr7ADSTIJ/QLexPLG3JbEsJ+F1LaFx+SsO+oI0J86 64MbbjORpr0nF5PNp5MaDbrCLvlnYyJEag76Am9UQsqTnS6YYsFyCLD45uBd223T c3x5+gPuw/20qh2dakcOoe242JcINJbNuWbDUC81mf99NAHuS0xvwatlll+rW2xS o9aKFB3sCK8J92GyHY/vDdiZGrRbsLgzLGjYo5teULSgV8b5qkcbF36egFd/5LM= =dOJu -----END PGP SIGNATURE-----
Looks like somewhere new oslo.config module is used with old configuration options. I'm not sure how this is possible. Could be some serialization/deserialization code that has constructed config options from old data and thus bypassing normal construction and lacking some new object properties. Unfortunately, I do not have cycles currently to look into this in more detail. On Mon, Sep 22, 2014 at 2:40 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
(Fixing the wrong Doug's email address.)
On 22/09/14 11:23, Alan Pevec wrote:
2014-09-22 10:52 GMT+02:00 Ihar Hrachyshka <ihrachys@redhat.com>:
Havana jobs failed, probably due to new oslo.config release. Since tags for the last release are already created, we won't fix those failures.
That's correct, 2013.2.4 tags have been pushed and we're not fixing this in Havana but it should be investigated. Oslo releases are supposed to be backward compatible so we need to understand if that's an oslo.config regression or not.
I suspect this may be due to oslo-config-options-validation work done in 1.4.0 [1]. That said, the failure is observed in glance/havana only, and logs are not very informative. The only thing that is present is the following critical error in glance-api log (no traceback):
2014-09-22 06:27:04.281 13352 CRITICAL glance [-] 'StrOpt' object has no attribute 'type'
[1]:
https://review.openstack.org/gitweb?p=openstack%2Foslo.config.git;a=commitdi...
/Ihar -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
iQEcBAEBCgAGBQJUH/yhAAoJEC5aWaUY1u57WDoIAMMYBpio1k0iYVoQyFAi5BdI 306BXUsywij1DBdTdbEwqiRET2yAmFPuD4IaYXX0aeVu2quHvRxvzGCLRZQa6jXS I5ICFlOzBdCBqdauVXzqiipNr7ADSTIJ/QLexPLG3JbEsJ+F1LaFx+SsO+oI0J86 64MbbjORpr0nF5PNp5MaDbrCLvlnYyJEag76Am9UQsqTnS6YYsFyCLD45uBd223T c3x5+gPuw/20qh2dakcOoe242JcINJbNuWbDUC81mf99NAHuS0xvwatlll+rW2xS o9aKFB3sCK8J92GyHY/vDdiZGrRbsLgzLGjYo5teULSgV8b5qkcbF36egFd/5LM= =dOJu -----END PGP SIGNATURE-----
On Sep 22, 2014, at 11:47 AM, Maxim Kulkin <mkulkin@mirantis.com> wrote:
Looks like somewhere new oslo.config module is used with old configuration options. I'm not sure how this is possible. Could be some serialization/deserialization code that has constructed config options from old data and thus bypassing normal construction and lacking some new object properties.
The keystone client library is using a feature that is not available in the old version of oslo.config. It’s not clear why the old version of oslo.config is present for some jobs and the new one is present for others. Doug
Unfortunately, I do not have cycles currently to look into this in more detail.
On Mon, Sep 22, 2014 at 2:40 PM, Ihar Hrachyshka <ihrachys@redhat.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
(Fixing the wrong Doug's email address.)
On 22/09/14 11:23, Alan Pevec wrote:
2014-09-22 10:52 GMT+02:00 Ihar Hrachyshka <ihrachys@redhat.com>:
Havana jobs failed, probably due to new oslo.config release. Since tags for the last release are already created, we won't fix those failures.
That's correct, 2013.2.4 tags have been pushed and we're not fixing this in Havana but it should be investigated. Oslo releases are supposed to be backward compatible so we need to understand if that's an oslo.config regression or not.
I suspect this may be due to oslo-config-options-validation work done in 1.4.0 [1]. That said, the failure is observed in glance/havana only, and logs are not very informative. The only thing that is present is the following critical error in glance-api log (no traceback):
2014-09-22 06:27:04.281 13352 CRITICAL glance [-] 'StrOpt' object has no attribute 'type'
[1]:
https://review.openstack.org/gitweb?p=openstack%2Foslo.config.git;a=commitdi...
/Ihar -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
iQEcBAEBCgAGBQJUH/yhAAoJEC5aWaUY1u57WDoIAMMYBpio1k0iYVoQyFAi5BdI 306BXUsywij1DBdTdbEwqiRET2yAmFPuD4IaYXX0aeVu2quHvRxvzGCLRZQa6jXS I5ICFlOzBdCBqdauVXzqiipNr7ADSTIJ/QLexPLG3JbEsJ+F1LaFx+SsO+oI0J86 64MbbjORpr0nF5PNp5MaDbrCLvlnYyJEag76Am9UQsqTnS6YYsFyCLD45uBd223T c3x5+gPuw/20qh2dakcOoe242JcINJbNuWbDUC81mf99NAHuS0xvwatlll+rW2xS o9aKFB3sCK8J92GyHY/vDdiZGrRbsLgzLGjYo5teULSgV8b5qkcbF36egFd/5LM= =dOJu -----END PGP SIGNATURE-----
On 2014-09-22 12:17:20 -0400 (-0400), Doug Hellmann wrote:
The keystone client library is using a feature that is not available in the old version of oslo.config. It’s not clear why the old version of oslo.config is present for some jobs and the new one is present for others.
You would probably need to grep through the requirements lists for all projects' stable/havana branches to find out... remember that openstack/requirements:global-requirements.txt was mostly advisory back then, and as a result not especially consistent with what various projects used. One or more may have capped a maximum version of oslo.config? -- Jeremy Stanley
On Sep 22, 2014, at 2:28 PM, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2014-09-22 12:17:20 -0400 (-0400), Doug Hellmann wrote:
The keystone client library is using a feature that is not available in the old version of oslo.config. It’s not clear why the old version of oslo.config is present for some jobs and the new one is present for others.
You would probably need to grep through the requirements lists for all projects' stable/havana branches to find out... remember that openstack/requirements:global-requirements.txt was mostly advisory back then, and as a result not especially consistent with what various projects used. One or more may have capped a maximum version of oslo.config?
bknudson sent these links: pip freeze output for job that failed: http://logs.openstack.org/98/122998/2/check/check-grenade-dsvm-partial-ncpu-... pip freeze output for job that passed: http://logs.openstack.org/72/122772/4/check/check-grenade-dsvm/1c316e8/logs/... I see those are 2 different variations of the grenade job, but I don’t know precisely what those differences are, yet. In any case, the point is that this isn’t a regression in oslo.config. It’s an issue either with the job not installing the required version of oslo.config or with the client not being flexible enough in its use of the available libraries. Doug
-- Jeremy Stanley
-----Original Message----- From: Doug Hellmann <doug@doughellmann.com> Reply: Doug Hellmann <doug@doughellmann.com>> Date: September 22, 2014 at 13:03:34 To: Jeremy Stanley <fungi@yuggoth.org>> Cc: openstack-stable-maint <openstack-stable-maint@lists.openstack.org>>, Maxim Kulkin <mkulkin@mirantis.com>> Subject: Re: [Openstack-stable-maint] Stable check of openstack/tempest failed
On Sep 22, 2014, at 2:28 PM, Jeremy Stanley wrote:
On 2014-09-22 12:17:20 -0400 (-0400), Doug Hellmann wrote:
The keystone client library is using a feature that is not available in the old version of oslo.config. It’s not clear why the old version of oslo.config is present for some jobs and the new one is present for others.
You would probably need to grep through the requirements lists for all projects' stable/havana branches to find out... remember that openstack/requirements:global-requirements.txt was mostly advisory back then, and as a result not especially consistent with what various projects used. One or more may have capped a maximum version of oslo.config?
bknudson sent these links:
pip freeze output for job that failed: http://logs.openstack.org/98/122998/2/check/check-grenade-dsvm-partial-ncpu-...
pip freeze output for job that passed: http://logs.openstack.org/72/122772/4/check/check-grenade-dsvm/1c316e8/logs/...
I see those are 2 different variations of the grenade job, but I don’t know precisely what those differences are, yet.
In any case, the point is that this isn’t a regression in oslo.config. It’s an issue either with the job not installing the required version of oslo.config or with the client not being flexible enough in its use of the available libraries.
Doug
There is a fix being worked on for keystoneclient here: https://review.openstack.org/#/c/123198 and it has been targeted for a 0.11.1 release. Cheers, Morgan
participants (7)
-
Alan Pevec
-
Doug Hellmann
-
Ihar Hrachyshka
-
jenkins＠openstack.org
-
Jeremy Stanley
-
Maxim Kulkin
-
Morgan Fainberg