Open Stack

Hi Radek

Thanks for your information!
Umm.. When I checked the log, the following workarounds was suggested.
-----------
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
-----------

The trigger seems to be that requirements upgraded the version of protobuf from 3.20.1 to 4.21.X.
https://opendev.org/openstack/requirements/commit/0f0b7024ece8a47316e4d9775f09f0e8d53b4edb

Anyway, we will try workaround "2" as a temporary fix and wait for the telemetry team to solve this problem.
Thank you for your help!

Best Regards,
Ueha

-----Original Message-----
From: Radosław Piliszek <radoslaw.piliszek at gmail.com> 
Sent: Tuesday, August 2, 2022 7:03 PM
To: Ueha, Ayumu/植波 歩 <ueha.ayumu at fujitsu.com>
Cc: openstack-discuss at lists.openstack.org
Subject: Re: [ceilometer][gnocchi][tacker] Internal Server Error in devstack for Zuul gate job

Hi Ueha,

It seems gnocchi is failing and requires a regeneration of the protobuf client:

https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_712/851478/2/check/tacker-functional-devstack-multinode-sol/71262e6/controller-tacker/logs/screen-gnocchi-api.txt

---

Jul 29 05:20:41.105688 ubuntu-focal-ovh-bhs1-0030564625
devstack at gnocchi-api.service[61040]:   File
"/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py",
line 755, in __new__
Jul 29 05:20:41.105688 ubuntu-focal-ovh-bhs1-0030564625
devstack at gnocchi-api.service[61040]:
_message.Message._CheckCalledFromGeneratedFile()
Jul 29 05:20:41.105688 ubuntu-focal-ovh-bhs1-0030564625
devstack at gnocchi-api.service[61040]: TypeError: Descriptors cannot not be created directly.
Jul 29 05:20:41.105688 ubuntu-focal-ovh-bhs1-0030564625
devstack at gnocchi-api.service[61040]: If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.

---

Kind regards,
Radek
-yoctozepto

On Tue, 2 Aug 2022 at 11:53, ueha.ayumu at fujitsu.com <ueha.ayumu at fujitsu.com> wrote:
>
> Hi telemetry team,
>
>
>
> I’m Ueha from Tacker team,
>
> The Zuul gate job of Tacker failed with the following error. Do you know the solution?
>
> The gate job has failed, so we would appreciate it if you could deal it with high priority.
>
> Thanks!
>
>
>
> for reference, the same error occurring in the ceilometer patch.
>
> (https://review.opendev.org/c/openstack/ceilometer/+/851338 ‘s 
> telemetry-dsvm-integration-centos-9s job)
>
>
>
> ------------------
>
>   ++ /opt/stack/ceilometer/devstack/plugin.sh:start_ceilometer:322 :   /usr/local/bin/ceilometer-upgrade
>
>   2022-07-29 05:20:42.125 61523 DEBUG ceilometer.cmd.storage [-] 
> Upgrading Gnocchi resource types upgrade 
> /opt/stack/ceilometer/ceilometer/cmd/storage.py:42
>
>   2022-07-29 05:20:42.228 61523 CRITICAL ceilometer [-] Unhandled 
> error: gnocchiclient.exceptions.ClientException: Internal Server Error 
> (HTTP 500)
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer Traceback (most recent call last):
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer   File "/usr/local/bin/ceilometer-upgrade", line 10, in <module>
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer     sys.exit(upgrade())
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer   File "/opt/stack/ceilometer/ceilometer/cmd/storage.py", line 49, in upgrade
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer     tenacity.Retrying(
>
>   ......... omit .........
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer   File "/usr/local/lib/python3.8/dist-packages/gnocchiclient/client.py", line 52, in request
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer     raise exceptions.from_response(resp, method)
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer 
> gnocchiclient.exceptions.ClientException: Internal Server Error (HTTP 
> 500)
>
>   2022-07-29 05:20:42.228 61523 ERROR ceilometer
>
> ------------------
>
> Full log: 
> https://zuul.opendev.org/t/openstack/build/71262e66ecf34827a8a3435657a
> a9b3f
>
>
>
> Best Regards,
>
> Ueha
>
>