Hey, Looks like that is the problem. The [ ] around the IP address are causing the issue. If I try to run dnsmasq using exactly the output you get, it gives me the same error: [root@tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot dnsmasq: bad command line options: try --help VS without the [ ] I can see it starts up normally. The settings in your undercloud.conf file look to be correct I believe. So I think there might be a bug here. I don't think we should be saving that value with the square brackets, or we would need to filter them out when we gather the value in that variable. I raised a bug for it here so that we can dig into this and find what needs fixing: https://bugs.launchpad.net/tripleo/+bug/1978892 In the meantime, if you edit that hieradata value, are you able to get that container started? Change this: [root@tripleo-director ~]# egrep -r 'tftp_bind_host' /etc/puppet/hieradata/ /etc/puppet/hieradata/service_configs.json: "ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}", To this: "ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1" Then restart the service: sudo systemctl restart tripleo_ironic_pxe_http.service tripleo_ironic_pxe_tftp.service Does that get the container running without the error? I did the same in my environment and can see that dnsmasq is running properly like that: [root@tripleo-director ~]# ps -ef | grep aaaa root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot Brendan Shephard Software Engineer Red Hat APAC <https://www.redhat.com> 193 N Quay Brisbane City QLD 4000 @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://red.ht/sig> <https://redhat.com/summit> On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour <lokendrarathour@gmail.com> wrote:
Hi Shephard, I am getting the local_ip (ipv6) of the undercloud :
[root@undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml [aaaa:aaaa:aaaa::1]
is this because of some ipv6 reasons?
On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard <bshephar@redhat.com> wrote:
Hey,
Ok, that command looks fine. What about that variable there? Do you get anything back when you run: sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml
Mine returns: sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml 192.168.24.115
Brendan Shephard
Software Engineer
Red Hat APAC <https://www.redhat.com>
193 N Quay
Brisbane City QLD 4000 @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://red.ht/sig> <https://redhat.com/summit>
On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour < lokendrarathour@gmail.com> wrote:
Hi Shephard,
this is the command from my wallaby: [root@undercloud ~]# sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json { "cap_add": [ "NET_ADMIN", "NET_RAW", "SETUID" ], "command": [ "/bin/bash", "-c", "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" ], "environment": { "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543" }, "healthcheck": { "test": "/openstack/healthcheck" }, "image": "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo", "net": "host", "privileged": false, "restart": "always", "start_order": 90, "volumes": [ "/etc/hosts:/etc/hosts:ro", "/etc/localtime:/etc/localtime:ro", "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro",
"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro",
"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro",
"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro", "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro", "/dev/log:/dev/log", "/etc/puppet:/etc/puppet:ro",
"/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro",
"/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro", "/var/lib/ironic:/var/lib/ironic:shared,z", "/var/log/containers/ironic:/var/log/ironic:z", "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z" ] }[root@undercloud ~]#
Comparing both, they look alike. please check once.
On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard <bshephar@redhat.com> wrote:
Hi,
Looks like the command was in a different file in Wallaby, can you check: sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json
That one should have the dnsmasq command it's trying to run. For example, here it is from my Wallaby environment: [stack@undercloud-0 ~]$ sudo cat /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json | jq .command [ "/bin/bash", "-c", "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp --tftp-root=/var/lib/ironic/tftpboot" ]
Brendan Shephard
Software Engineer
Red Hat APAC <https://www.redhat.com>
193 N Quay
Brisbane City QLD 4000 @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://red.ht/sig> <https://redhat.com/summit>
On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour < lokendrarathour@gmail.com> wrote:
Hi Shephard, Here is the o/p of the file:
[root@undercloud ~]# sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json { "config_files": [ { "dest": "/", "merge": true, "preserve_properties": true, "source": "/var/lib/kolla/config_files/src/*" } ], "permissions": [ { "owner": "ironic:ironic", "path": "/var/log/ironic", "recurse": true }, { "owner": "ironic:ironic", "path": "/var/lib/ironic", "recurse": true } ] }[root@undercloud ~]#
Thanks once agan.
-Lokendra
On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard <bshephar@redhat.com> wrote:
Looks like something wrong with the dnsmasq command the container is being launched with. What command is it trying to run?
sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json
Brendan Shephard
Software Engineer
Red Hat APAC <https://www.redhat.com>
193 N Quay
Brisbane City QLD 4000 @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://red.ht/sig> <https://redhat.com/summit>
On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta <anyrude10@gmail.com> wrote:
> Hi Brendan, > > Thanks for your response. > > Please find the log below. > > [stack@undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp > > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > dnsmasq: bad command line options: try --help > > [stack@undercloud t2u2v2w]$ sudo podman ps --filter > name=ironic_pxe -a > CONTAINER ID IMAGE > COMMAND CREATED > STATUS PORTS NAMES > 02dacbc74cec > undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo > /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy) > ironic_pxe_tftp > 1f8ca39fba32 > undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo > kolla_start 3 hours ago Up 3 hours ago (healthy) > ironic_pxe_http > > > Regards > > Anirudh Gupta > > On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard < > bshephar@redhat.com> wrote: > >> Hey Anirudh, >> >> You would need to look at the logs for the ironic_pxe_tftp container >> to see why it's failing. >> >> I assume the tftp container is not Up when you run this command? >> [stack@tripleo-director overcloud_playbooks]$ sudo podman ps >> --filter name=ironic_pxe -a >> CONTAINER ID IMAGE >> COMMAND CREATED STATUS >> PORTS NAMES >> 0170be36e291 >> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >> kolla_start 12 days ago Up 30 hours ago (healthy) >> ironic_pxe_tftp >> e507f722bdf0 >> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo >> kolla_start 12 days ago Up 30 hours ago (healthy) >> ironic_pxe_http >> >> Then check the logs to see what the error is: >> [stack@tripleo-director overcloud_playbooks]$ sudo podman logs >> ironic_pxe_tftp >> >> >> >> Brendan Shephard >> >> Software Engineer >> >> Red Hat APAC <https://www.redhat.com> >> >> 193 N Quay >> >> Brisbane City QLD 4000 >> @RedHat <https://twitter.com/redhat> Red Hat >> <https://www.linkedin.com/company/red-hat> Red Hat >> <https://www.facebook.com/RedHatInc> >> <https://red.ht/sig> >> <https://redhat.com/summit> >> >> >> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta <anyrude10@gmail.com> >> wrote: >> >>> Hi Team, >>> >>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but >>> facing the below error: >>> >>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f >>> | TASK | Manage container systemd services and cleanup old systemd >>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 >>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f >>> | FATAL | Manage container systemd services and cleanup old systemd >>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 | >>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has >>> not started yet"} >>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f >>> | TIMING | tripleo_container_manage : Manage container systemd >>> >>> Sample Undercloud.conf is as follows: >>> >>> [DEFAULT] >>> clean_nodes = true >>> cleanup = false >>> container_cli = podman >>> container_healthcheck_disabled = true >>> container_images_file = >>> /home/stack/containers-prepare-parameter.yaml >>> deployment_user = stack >>> enable_ironic = true >>> enable_ironic_inspector = true >>> enable_neutron = true >>> enable_routed_networks = false >>> generate_service_certificate = false >>> ipv6_address_mode = dhcpv6-stateful >>> ipxe_enabled = true >>> local_interface = enp8s0 >>> local_ip = aaaa:aaaa:aaaa::1/64 >>> subnets = ctlplane-subnet >>> undercloud_admin_host = aaaa:aaaa:aaaa::1 >>> undercloud_hostname = undercloud.com >>> undercloud_ntp_servers = 30.30.30.3 >>> undercloud_public_host = aaaa:aaaa:aaaa::1 >>> undercloud_timezone = UTC >>> >>> [ctlplane-subnet] >>> cidr = aaaa:aaaa:aaaa::/64 >>> dhcp_end = aaaa:aaaa:aaaa::f >>> dhcp_start = aaaa:aaaa:aaaa::a >>> gateway = aaaa:aaaa:aaaa::1 >>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9 >>> >>> Can someone please help in this regard. >>> >>> Anirudh Gupta >>> >>>
-- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour
-- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour
-- ~ Lokendra www.inertiaspeaks.com www.inertiagroups.com skype: lokendrarathour