I found the issue it was kolla-ansible role had some typo in jinja2 template https://bugs.launchpad.net/kolla-ansible/+bug/2107983 On Tue, Apr 22, 2025 at 7:23 AM Sean Mooney <smooney@redhat.com> wrote:
On 22/04/2025 05:30, Michał Nasiadka wrote:
Thank you Satish, can you please raise those bugs in Launchpad with more extensive logs and your configuration?
It’s still not rc1, so you can expect some major bugs.
Best regards
Michał Nasiadka mnasiadka@gmail.com
W dniu wt., 22 kwi 2025 o 05:40 Satish Patel <satish.txt@gmail.com> napisał(a):
Here is the output of debug mode related TASK [nova-cell : Waiting for nova-compute services to register themselves]
TASK [nova-cell : Waiting for nova-compute services to register themselves]
******************************************************************************************************************************************
task path:
/opt/xenv/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml:8
The full traceback is: Traceback (most recent call last): File
"/opt/xenv/lib/python3.10/site-packages/ansible/executor/task_executor.py",
line 165, in run res = self._execute() File
"/opt/xenv/lib/python3.10/site-packages/ansible/executor/task_executor.py",
line 766, in _execute if cond.evaluate_conditional(templar, vars_copy): File
"/opt/xenv/lib/python3.10/site-packages/ansible/playbook/conditional.py",
line 59, in evaluate_conditional return self.evaluate_conditional_with_result(templar,
all_vars)[0]
File
"/opt/xenv/lib/python3.10/site-packages/ansible/playbook/conditional.py",
line 73, in evaluate_conditional_with_result res = self._check_conditional(conditional, templar, all_vars) File
"/opt/xenv/lib/python3.10/site-packages/ansible/playbook/conditional.py",
line 109, in _check_conditional return templar.template( File
"/opt/xenv/lib/python3.10/site-packages/ansible/template/__init__.py",
line 718, in template result = self.do_template( File
"/opt/xenv/lib/python3.10/site-packages/ansible/template/__init__.py",
line 964, in do_template res = myenv.concat(rf) File
"/opt/xenv/lib/python3.10/site-packages/ansible/template/native_helpers.py",
line 81, in ansible_eval_concat head = list(islice(nodes, 2)) File "<template>", line 37, in root File
"/opt/xenv/lib/python3.10/site-packages/ansible/template/__init__.py",
line 294, in wrapper ret = func(*args, **kwargs) File "/usr/lib/python3.10/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) fatal: [vsan2]: FAILED! => { "msg": "Unexpected failure during module execution: Expecting value: line 1 column 1 (char 0)", "stdout": "" }
are you trying to run this on top of xen hypervisor?
there are issue with newer kernels and old version of xen related to how xen virtualization the kernel.
i ask because /opt/xenv/share/kolla-ansible looks odd.
On Mon, Apr 21, 2025 at 11:24 PM Satish Patel <satish.txt@gmail.com> wrote:
Folks,
For the experiment, I am deploying epoxy in the lab and getting some nova related issues. not sure if they are bugs or just config tuning.
#BUG 1
placement_api container failed to start because of uWSIG provider issue. I have changed to apache that fixed the issue.
placement_api_backend: "apache" placement_wsgi_provider: "apache"
please include logs when reporting this but this is likely a kolla config issue, it might be a placement bug but we deploy under uwsgi in devstack jobs so we know placement work with uwsgi but kolla and devstack might deploy things differently.
# BUG 2 or Failed it this point.
TASK [nova-cell : Waiting for nova-compute services to register themselves]
******************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) fatal: [vsan2]: FAILED! => {"msg": "Unexpected failure during module execution: Expecting value: line 1 column 1 (char 0)", "stdout": ""}
# BUG 3
inside the nova-computer container I didn't see any service or process related privsep-helper --config-file /etc/nova/nova.conf is this something new or missed by bug or config?
(xenv) root@vsan2:/etc/kolla# docker exec -it -u0 nova_compute bash (nova-compute)[root@vsan2 /]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND nova 1 0.0 0.0 2644 956 ? Ss 08:28 0:00 dumb-init --single-child -- kolla_start nova 7 1.0 0.0 1883120 188576 ? Sl 08:28 0:15 /var/lib/kolla/venv/bin/python3 /var/lib/kolla/venv/bin/nova-compute root 611 1.0 0.0 7636 4128 pts/2 Ss 08:52 0:00 bash root 627 0.0 0.0 10072 1580 pts/2 R+ 08:52 0:00 ps aux
This is normal. you will only see privsep after a call that need higher privalages is executed, so unless you have created a new vms or something like that
it wont have a privsep process intially on startup.