Hey,

It looks like my reply email was too big and has been blocked. The problem is the square brackets in the IPv6 address being returned from the hiera command. Can you try this to workaround the problem:
Change this:
[root@tripleo-director ~]# egrep -r 'tftp_bind_host' /etc/puppet/hieradata/
/etc/puppet/hieradata/service_configs.json:    "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}",

To this:
 "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1"

Then restart the service:
sudo systemctl restart tripleo_ironic_pxe_http.service tripleo_ironic_pxe_tftp.service

I raised a bug for this for you:
https://bugs.launchpad.net/tripleo/+bug/1978892

Brendan Shephard

Software Engineer

Red Hat APAC

193 N Quay

Brisbane City QLD 4000


On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour <lokendrarathour@gmail.com> wrote:
Hi Shephard,
I am getting the local_ip (ipv6) of the undercloud :

[root@undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml
[aaaa:aaaa:aaaa::1]

is this because of some ipv6 reasons?


On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard <bshephar@redhat.com> wrote:
Hey,

Ok, that command looks fine. What about that variable there? Do you get anything back when you run:
sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml

Mine returns:
sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml
192.168.24.115

Brendan Shephard

Software Engineer

Red Hat APAC

193 N Quay

Brisbane City QLD 4000


On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour <lokendrarathour@gmail.com> wrote:
Hi Shephard,

this is the command from my wallaby:
[root@undercloud ~]# sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json
{
  "cap_add": [
    "NET_ADMIN",
    "NET_RAW",
    "SETUID"
  ],
  "command": [
    "/bin/bash",
    "-c",
    "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot"
  ],
  "environment": {
    "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS",
    "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543"
  },
  "healthcheck": {
    "test": "/openstack/healthcheck"
  },
  "image": "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo",
  "net": "host",
  "privileged": false,
  "restart": "always",
  "start_order": 90,
  "volumes": [
    "/etc/hosts:/etc/hosts:ro",
    "/etc/localtime:/etc/localtime:ro",
    "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro",
    "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro",
    "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro",
    "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro",
    "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro",
    "/dev/log:/dev/log",
    "/etc/puppet:/etc/puppet:ro",
    "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro",
    "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro",
    "/var/lib/ironic:/var/lib/ironic:shared,z",
    "/var/log/containers/ironic:/var/log/ironic:z",
    "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z"
  ]
}[root@undercloud ~]#

Comparing both, they look alike.
please check once.

On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard <bshephar@redhat.com> wrote:
Hi,

Looks like the command was in a different file in Wallaby, can you check:
sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json

That one should have the dnsmasq command it's trying to run. For example, here it is from my Wallaby environment:
[stack@undercloud-0 ~]$ sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json | jq .command
[
  "/bin/bash",
  "-c",
  "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot"
]



Brendan Shephard

Software Engineer

Red Hat APAC

193 N Quay

Brisbane City QLD 4000


On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour <lokendrarathour@gmail.com> wrote:
Hi Shephard,
Here is the o/p of the file:

[root@undercloud ~]# sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json
{
  "config_files": [
    {
      "dest": "/",
      "merge": true,
      "preserve_properties": true,
      "source": "/var/lib/kolla/config_files/src/*"
    }
  ],
  "permissions": [
    {
      "owner": "ironic:ironic",
      "path": "/var/log/ironic",
      "recurse": true
    },
    {
      "owner": "ironic:ironic",
      "path": "/var/lib/ironic",
      "recurse": true
    }
  ]
}[root@undercloud ~]#


Thanks once agan. 

-Lokendra


On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard <bshephar@redhat.com> wrote:
Looks like something wrong with the dnsmasq command the container is being launched with. What command is it trying to run?

sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json

Brendan Shephard

Software Engineer

Red Hat APAC

193 N Quay

Brisbane City QLD 4000


On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta <anyrude10@gmail.com> wrote:
Hi Brendan,

Thanks for your response.

Please find the log below.

[stack@undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp

dnsmasq: bad command line options: try --help
dnsmasq: bad command line options: try --help
dnsmasq: bad command line options: try --help
dnsmasq: bad command line options: try --help
dnsmasq: bad command line options: try --help
dnsmasq: bad command line options: try --help

[stack@undercloud t2u2v2w]$  sudo podman ps --filter name=ironic_pxe -a
CONTAINER ID  IMAGE                                                                                     COMMAND               CREATED      STATUS                              PORTS       NAMES
02dacbc74cec  undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo  /bin/bash -c BIND...  3 hours ago  Exited (1) 3 hours ago (unhealthy)              ironic_pxe_tftp
1f8ca39fba32  undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo  kolla_start           3 hours ago  Up 3 hours ago (healthy)                        ironic_pxe_http


Regards

Anirudh Gupta


On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard <bshephar@redhat.com> wrote:
Hey Anirudh,

You would need to look at the logs for the ironic_pxe_tftp container to see why it's failing.

I assume the tftp container is not Up when you run this command?
[stack@tripleo-director overcloud_playbooks]$ sudo podman ps --filter name=ironic_pxe -a
CONTAINER ID  IMAGE                                                                                       COMMAND      CREATED      STATUS                     PORTS       NAMES
0170be36e291  registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo  kolla_start  12 days ago  Up 30 hours ago (healthy)              ironic_pxe_tftp
e507f722bdf0  registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo  kolla_start  12 days ago  Up 30 hours ago (healthy)              ironic_pxe_http

Then check the logs to see what the error is:
[stack@tripleo-director overcloud_playbooks]$ sudo podman logs ironic_pxe_tftp



Brendan Shephard

Software Engineer

Red Hat APAC

193 N Quay

Brisbane City QLD 4000


On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta <anyrude10@gmail.com> wrote:
Hi Team,

I am trying to deploy Openstack Wallaby Undercloud on IPv6, but facing the below error:

2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f | TASK | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4
2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f | FATAL | Manage container systemd services and cleanup old systemd healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has not started yet"}
2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f | TIMING | tripleo_container_manage : Manage container systemd

Sample Undercloud.conf is as follows:

[DEFAULT]
clean_nodes = true
cleanup = false
container_cli = podman
container_healthcheck_disabled = true
container_images_file = /home/stack/containers-prepare-parameter.yaml
deployment_user = stack
enable_ironic = true
enable_ironic_inspector = true
enable_neutron = true
enable_routed_networks = false
generate_service_certificate = false
ipv6_address_mode = dhcpv6-stateful
ipxe_enabled = true
local_interface = enp8s0
local_ip = aaaa:aaaa:aaaa::1/64
subnets = ctlplane-subnet
undercloud_admin_host = aaaa:aaaa:aaaa::1
undercloud_hostname = undercloud.com
undercloud_ntp_servers = 30.30.30.3
undercloud_public_host = aaaa:aaaa:aaaa::1
undercloud_timezone = UTC

[ctlplane-subnet]
cidr = aaaa:aaaa:aaaa::/64
dhcp_end = aaaa:aaaa:aaaa::f
dhcp_start = aaaa:aaaa:aaaa::a
gateway = aaaa:aaaa:aaaa::1
inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9

Can someone please help in this regard.

Anirudh Gupta



--
~ Lokendra
skype: lokendrarathour




--
~ Lokendra
skype: lokendrarathour




--
~ Lokendra
skype: lokendrarathour