[kolla-ansible][watcher]Time zone issue in container
Hi all, Recently I created fresh deployment based on kolla-ansible yoga and debian 11. Everything works correctly except Watcher services. Containers are in constant restart loop. On Debian host OS time-related files are as follows: lrwxrwxrwx 1 root root 33 Mar 21 07:53 localtime -> /usr/share/zoneinfo/Europe/Warsaw -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone x@control01:/etc$ cat timezone Europe/Warsaw However every container have: lrwxrwxrwx 1 root root 27 Apr 11 02:00 localtime -> /usr/share/zoneinfo/Etc/UTC -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone (nova-api)[root@control01 etc]# cat timezone Europe/Warsaw In watcher logs I found error: 2023-04-27 10:08:12.607 7 ERROR python-watcher tzlocal.utils.ZoneInfoNotFoundError: 'Multiple conflicting time zone configurations found:\n/etc/timezone: Europe/Warsaw\n/etc/localtime is a symlink to: UTC\nFix the configuration, or set the time zone in a TZ environment variable.\n' 2023-04-27 10:08:12.607 7 ERROR python-watcher Which I believe is caused by wrong symlink for /etc/localtime in containers. Did anyone experience similar issue and know how to resolve it. Best regards, Stefan
Hi Stefan, Kolla Ansible is configured to bind mount both files when running Debian: watcher_api_default_volumes: - "{{ node_config_directory }}/watcher-api/:{{ container_config_directory }}/:ro" - "/etc/localtime:/etc/localtime:ro" - "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}" - "{{ kolla_dev_repos_directory ~ '/watcher/watcher:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/watcher' if watcher_dev_mode | bool else '' }}" - "kolla_logs:/var/log/kolla/" Can you share the HostConfig/Binds part of `docker inspect watcher_api`. For example this is the configuration for placement_api on a non-Debian host: "HostConfig": { "Binds": [ "/etc/kolla/placement-api/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "kolla_logs:/var/log/kolla/:rw" ], On Thu, 27 Apr 2023 at 15:16, Stefan Bryzga <stefanbryzga@gmail.com> wrote:
Hi all,
Recently I created fresh deployment based on kolla-ansible yoga and debian 11. Everything works correctly except Watcher services. Containers are in constant restart loop. On Debian host OS time-related files are as follows:
lrwxrwxrwx 1 root root 33 Mar 21 07:53 localtime -> /usr/share/zoneinfo/Europe/Warsaw -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone x@control01:/etc$ cat timezone Europe/Warsaw
However every container have:
lrwxrwxrwx 1 root root 27 Apr 11 02:00 localtime -> /usr/share/zoneinfo/Etc/UTC -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone (nova-api)[root@control01 etc]# cat timezone Europe/Warsaw
In watcher logs I found error: 2023-04-27 10:08:12.607 7 ERROR python-watcher tzlocal.utils.ZoneInfoNotFoundError: 'Multiple conflicting time zone configurations found:\n/etc/timezone: Europe/Warsaw\n/etc/localtime is a symlink to: UTC\nFix the configuration, or set the time zone in a TZ environment variable.\n' 2023-04-27 10:08:12.607 7 ERROR python-watcher Which I believe is caused by wrong symlink for /etc/localtime in containers.
Did anyone experience similar issue and know how to resolve it.
Best regards, Stefan
Thank you for looking into this, Here is mentioned info: docker inspect for watcher_api container (binds for localtime and timezone are identical for all containers): "HostConfig": { "Binds": [ "/etc/kolla/watcher-api/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "/etc/timezone:/etc/timezone:ro", "kolla_logs:/var/log/kolla/:rw" ], If this could help: Time zone on debian host was setup during OS installation Base OS is synced to NTP server with systemd-timesyncd Best regards, Stefan czw., 27 kwi 2023 o 18:31 Pierre Riteau <pierre@stackhpc.com> napisał(a):
Hi Stefan,
Kolla Ansible is configured to bind mount both files when running Debian:
watcher_api_default_volumes: - "{{ node_config_directory }}/watcher-api/:{{ container_config_directory }}/:ro" - "/etc/localtime:/etc/localtime:ro" - "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}" - "{{ kolla_dev_repos_directory ~ '/watcher/watcher:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/watcher' if watcher_dev_mode | bool else '' }}" - "kolla_logs:/var/log/kolla/"
Can you share the HostConfig/Binds part of `docker inspect watcher_api`. For example this is the configuration for placement_api on a non-Debian host:
"HostConfig": { "Binds": [
"/etc/kolla/placement-api/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "kolla_logs:/var/log/kolla/:rw" ],
On Thu, 27 Apr 2023 at 15:16, Stefan Bryzga <stefanbryzga@gmail.com> wrote:
Hi all,
Recently I created fresh deployment based on kolla-ansible yoga and debian 11. Everything works correctly except Watcher services. Containers are in constant restart loop. On Debian host OS time-related files are as follows:
lrwxrwxrwx 1 root root 33 Mar 21 07:53 localtime -> /usr/share/zoneinfo/Europe/Warsaw -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone x@control01:/etc$ cat timezone Europe/Warsaw
However every container have:
lrwxrwxrwx 1 root root 27 Apr 11 02:00 localtime -> /usr/share/zoneinfo/Etc/UTC -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone (nova-api)[root@control01 etc]# cat timezone Europe/Warsaw
In watcher logs I found error: 2023-04-27 10:08:12.607 7 ERROR python-watcher tzlocal.utils.ZoneInfoNotFoundError: 'Multiple conflicting time zone configurations found:\n/etc/timezone: Europe/Warsaw\n/etc/localtime is a symlink to: UTC\nFix the configuration, or set the time zone in a TZ environment variable.\n' 2023-04-27 10:08:12.607 7 ERROR python-watcher Which I believe is caused by wrong symlink for /etc/localtime in containers.
Did anyone experience similar issue and know how to resolve it.
Best regards, Stefan
participants (2)
-
Pierre Riteau
-
Stefan Bryzga