[Tripleo] - IPv6 Wallaby Undercloud Installation failure
Brendan Shephard
bshephar at redhat.com
Wed Jun 15 23:45:22 UTC 2022
Hey,
Looks like that is the problem. The [ ] around the IP address are causing
the issue. If I try to run dnsmasq using exactly the output you get, it
gives me the same error:
[root at tripleo-director ~]# /usr/sbin/dnsmasq --keep-in-foreground
--log-facility=/var/log/ironic/dnsmasq.log --user=root
--conf-file=/dev/null --listen-address=[aaaa:aaaa:aaaa::1] --port=0
--enable-tftp --tftp-root=/var/lib/ironic/tftpboot
dnsmasq: bad command line options: try --help
VS without the [ ] I can see it starts up normally.
The settings in your undercloud.conf file look to be correct I believe. So
I think there might be a bug here. I don't think we should be saving that
value with the square brackets, or we would need to filter them out when we
gather the value in that variable.
I raised a bug for it here so that we can dig into this and find what needs
fixing:
https://bugs.launchpad.net/tripleo/+bug/1978892
In the meantime, if you edit that hieradata value, are you able to get that
container started?
Change this:
[root at tripleo-director ~]# egrep -r 'tftp_bind_host' /etc/puppet/hieradata/
/etc/puppet/hieradata/service_configs.json:
"ironic::pxe::tftp_bind_host": "%{lookup('ctlplane_uri')}",
To this:
"ironic::pxe::tftp_bind_host": "aaaa:aaaa:aaaa::1"
Then restart the service:
sudo systemctl restart tripleo_ironic_pxe_http.service
tripleo_ironic_pxe_tftp.service
Does that get the container running without the error? I did the same in my
environment and can see that dnsmasq is running properly like that:
[root at tripleo-director ~]# ps -ef | grep aaaa
root 71180 52675 0 19:24 pts/4 00:00:00 /usr/sbin/dnsmasq
--keep-in-foreground --log-facility=/var/log/ironic/dnsmasq.log --user=root
--conf-file=/dev/null --listen-address=aaaa:aaaa:aaaa::1 --port=0
--enable-tftp --tftp-root=/var/lib/ironic/tftpboot
Brendan Shephard
Software Engineer
Red Hat APAC <https://www.redhat.com>
193 N Quay
Brisbane City QLD 4000
@RedHat <https://twitter.com/redhat> Red Hat
<https://www.linkedin.com/company/red-hat> Red Hat
<https://www.facebook.com/RedHatInc>
<https://red.ht/sig>
<https://redhat.com/summit>
On Thu, Jun 16, 2022 at 12:12 AM Lokendra Rathour <lokendrarathour at gmail.com>
wrote:
> Hi Shephard,
> I am getting the local_ip (ipv6) of the undercloud :
>
> [root at undercloud stack]# sudo hiera ironic::pxe::tftp_bind_host -c
> /etc/puppet/hiera.yaml
> [aaaa:aaaa:aaaa::1]
>
> is this because of some ipv6 reasons?
>
>
> On Wed, Jun 15, 2022 at 6:08 PM Brendan Shephard <bshephar at redhat.com>
> wrote:
>
>> Hey,
>>
>> Ok, that command looks fine. What about that variable there? Do you get
>> anything back when you run:
>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml
>>
>> Mine returns:
>> sudo hiera ironic::pxe::tftp_bind_host -c /etc/puppet/hiera.yaml
>> 192.168.24.115
>>
>> Brendan Shephard
>>
>> Software Engineer
>>
>> Red Hat APAC <https://www.redhat.com>
>>
>> 193 N Quay
>>
>> Brisbane City QLD 4000
>> @RedHat <https://twitter.com/redhat> Red Hat
>> <https://www.linkedin.com/company/red-hat> Red Hat
>> <https://www.facebook.com/RedHatInc>
>> <https://red.ht/sig>
>> <https://redhat.com/summit>
>>
>>
>> On Wed, Jun 15, 2022 at 8:20 PM Lokendra Rathour <
>> lokendrarathour at gmail.com> wrote:
>>
>>> Hi Shephard,
>>>
>>> this is the command from my wallaby:
>>> [root at undercloud ~]# sudo cat
>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json
>>> {
>>> "cap_add": [
>>> "NET_ADMIN",
>>> "NET_RAW",
>>> "SETUID"
>>> ],
>>> "command": [
>>> "/bin/bash",
>>> "-c",
>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c
>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground
>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root
>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp
>>> --tftp-root=/var/lib/ironic/tftpboot"
>>> ],
>>> "environment": {
>>> "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS",
>>> "TRIPLEO_CONFIG_HASH": "9fb3e4e0e35ee35fdf74cfccb16a7543"
>>> },
>>> "healthcheck": {
>>> "test": "/openstack/healthcheck"
>>> },
>>> "image":
>>> "undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo",
>>> "net": "host",
>>> "privileged": false,
>>> "restart": "always",
>>> "start_order": 90,
>>> "volumes": [
>>> "/etc/hosts:/etc/hosts:ro",
>>> "/etc/localtime:/etc/localtime:ro",
>>> "/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro",
>>>
>>> "/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro",
>>>
>>> "/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro",
>>>
>>> "/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro",
>>> "/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro",
>>> "/dev/log:/dev/log",
>>> "/etc/puppet:/etc/puppet:ro",
>>>
>>> "/var/lib/kolla/config_files/ironic_pxe_tftp.json:/var/lib/kolla/config_files/config.json:ro",
>>>
>>> "/var/lib/config-data/puppet-generated/ironic:/var/lib/kolla/config_files/src:ro",
>>> "/var/lib/ironic:/var/lib/ironic:shared,z",
>>> "/var/log/containers/ironic:/var/log/ironic:z",
>>> "/var/log/containers/httpd/ironic-pxe:/var/log/httpd:z"
>>> ]
>>> }[root at undercloud ~]#
>>>
>>> Comparing both, they look alike.
>>> please check once.
>>>
>>> On Wed, Jun 15, 2022 at 3:30 PM Brendan Shephard <bshephar at redhat.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Looks like the command was in a different file in Wallaby, can you
>>>> check:
>>>> sudo cat
>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json
>>>>
>>>> That one should have the dnsmasq command it's trying to run. For
>>>> example, here it is from my Wallaby environment:
>>>> [stack at undercloud-0 ~]$ sudo cat
>>>> /var/lib/tripleo-config/container-startup-config/step_4/ironic_pxe_tftp.json
>>>> | jq .command
>>>> [
>>>> "/bin/bash",
>>>> "-c",
>>>> "BIND_HOST=$(hiera ironic::pxe::tftp_bind_host -c
>>>> /etc/puppet/hiera.yaml); /usr/sbin/dnsmasq --keep-in-foreground
>>>> --log-facility=/var/log/ironic/dnsmasq.log --user=root
>>>> --conf-file=/dev/null --listen-address=$BIND_HOST --port=0 --enable-tftp
>>>> --tftp-root=/var/lib/ironic/tftpboot"
>>>> ]
>>>>
>>>>
>>>>
>>>> Brendan Shephard
>>>>
>>>> Software Engineer
>>>>
>>>> Red Hat APAC <https://www.redhat.com>
>>>>
>>>> 193 N Quay
>>>>
>>>> Brisbane City QLD 4000
>>>> @RedHat <https://twitter.com/redhat> Red Hat
>>>> <https://www.linkedin.com/company/red-hat> Red Hat
>>>> <https://www.facebook.com/RedHatInc>
>>>> <https://red.ht/sig>
>>>> <https://redhat.com/summit>
>>>>
>>>>
>>>> On Wed, Jun 15, 2022 at 7:19 PM Lokendra Rathour <
>>>> lokendrarathour at gmail.com> wrote:
>>>>
>>>>> Hi Shephard,
>>>>> Here is the o/p of the file:
>>>>>
>>>>> [root at undercloud ~]# sudo cat
>>>>> /var/lib/kolla/config_files/ironic_pxe_tftp.json
>>>>> {
>>>>> "config_files": [
>>>>> {
>>>>> "dest": "/",
>>>>> "merge": true,
>>>>> "preserve_properties": true,
>>>>> "source": "/var/lib/kolla/config_files/src/*"
>>>>> }
>>>>> ],
>>>>> "permissions": [
>>>>> {
>>>>> "owner": "ironic:ironic",
>>>>> "path": "/var/log/ironic",
>>>>> "recurse": true
>>>>> },
>>>>> {
>>>>> "owner": "ironic:ironic",
>>>>> "path": "/var/lib/ironic",
>>>>> "recurse": true
>>>>> }
>>>>> ]
>>>>> }[root at undercloud ~]#
>>>>>
>>>>>
>>>>> Thanks once agan.
>>>>>
>>>>> -Lokendra
>>>>>
>>>>>
>>>>> On Wed, Jun 15, 2022 at 2:38 PM Brendan Shephard <bshephar at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Looks like something wrong with the dnsmasq command the container is
>>>>>> being launched with. What command is it trying to run?
>>>>>>
>>>>>> sudo cat /var/lib/kolla/config_files/ironic_pxe_tftp.json
>>>>>>
>>>>>> Brendan Shephard
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> Red Hat APAC <https://www.redhat.com>
>>>>>>
>>>>>> 193 N Quay
>>>>>>
>>>>>> Brisbane City QLD 4000
>>>>>> @RedHat <https://twitter.com/redhat> Red Hat
>>>>>> <https://www.linkedin.com/company/red-hat> Red Hat
>>>>>> <https://www.facebook.com/RedHatInc>
>>>>>> <https://red.ht/sig>
>>>>>> <https://redhat.com/summit>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 15, 2022 at 6:22 PM Anirudh Gupta <anyrude10 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Brendan,
>>>>>>>
>>>>>>> Thanks for your response.
>>>>>>>
>>>>>>> Please find the log below.
>>>>>>>
>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman logs ironic_pxe_tftp
>>>>>>>
>>>>>>> dnsmasq: bad command line options: try --help
>>>>>>> dnsmasq: bad command line options: try --help
>>>>>>> dnsmasq: bad command line options: try --help
>>>>>>> dnsmasq: bad command line options: try --help
>>>>>>> dnsmasq: bad command line options: try --help
>>>>>>> dnsmasq: bad command line options: try --help
>>>>>>>
>>>>>>> [stack at undercloud t2u2v2w]$ sudo podman ps --filter
>>>>>>> name=ironic_pxe -a
>>>>>>> CONTAINER ID IMAGE
>>>>>>> COMMAND CREATED
>>>>>>> STATUS PORTS NAMES
>>>>>>> 02dacbc74cec
>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo
>>>>>>> /bin/bash -c BIND... 3 hours ago Exited (1) 3 hours ago (unhealthy)
>>>>>>> ironic_pxe_tftp
>>>>>>> 1f8ca39fba32
>>>>>>> undercloud.ctlplane.localdomain:8787/tripleowallaby/openstack-ironic-pxe:current-tripleo
>>>>>>> kolla_start 3 hours ago Up 3 hours ago (healthy)
>>>>>>> ironic_pxe_http
>>>>>>>
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Anirudh Gupta
>>>>>>>
>>>>>>> On Wed, Jun 15, 2022 at 11:30 AM Brendan Shephard <
>>>>>>> bshephar at redhat.com> wrote:
>>>>>>>
>>>>>>>> Hey Anirudh,
>>>>>>>>
>>>>>>>> You would need to look at the logs for the ironic_pxe_tftp container
>>>>>>>> to see why it's failing.
>>>>>>>>
>>>>>>>> I assume the tftp container is not Up when you run this command?
>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman ps
>>>>>>>> --filter name=ironic_pxe -a
>>>>>>>> CONTAINER ID IMAGE
>>>>>>>> COMMAND CREATED STATUS
>>>>>>>> PORTS NAMES
>>>>>>>> 0170be36e291
>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo
>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy)
>>>>>>>> ironic_pxe_tftp
>>>>>>>> e507f722bdf0
>>>>>>>> registry.okd4.bne-shift.net:8443/tripleomastercentos9/openstack-ironic-pxe:current-tripleo
>>>>>>>> kolla_start 12 days ago Up 30 hours ago (healthy)
>>>>>>>> ironic_pxe_http
>>>>>>>>
>>>>>>>> Then check the logs to see what the error is:
>>>>>>>> [stack at tripleo-director overcloud_playbooks]$ sudo podman logs
>>>>>>>> ironic_pxe_tftp
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Brendan Shephard
>>>>>>>>
>>>>>>>> Software Engineer
>>>>>>>>
>>>>>>>> Red Hat APAC <https://www.redhat.com>
>>>>>>>>
>>>>>>>> 193 N Quay
>>>>>>>>
>>>>>>>> Brisbane City QLD 4000
>>>>>>>> @RedHat <https://twitter.com/redhat> Red Hat
>>>>>>>> <https://www.linkedin.com/company/red-hat> Red Hat
>>>>>>>> <https://www.facebook.com/RedHatInc>
>>>>>>>> <https://red.ht/sig>
>>>>>>>> <https://redhat.com/summit>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 15, 2022 at 7:53 AM Anirudh Gupta <anyrude10 at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Team,
>>>>>>>>>
>>>>>>>>> I am trying to deploy Openstack Wallaby Undercloud on IPv6, but
>>>>>>>>> facing the below error:
>>>>>>>>>
>>>>>>>>> 2022-06-14 05:01:23.213708 | 52540083-cfa2-3f20-e9dc-00000000286f
>>>>>>>>> | TASK | Manage container systemd services and cleanup old systemd
>>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4
>>>>>>>>> 2022-06-14 05:03:22.912816 | 52540083-cfa2-3f20-e9dc-00000000286f
>>>>>>>>> | FATAL | Manage container systemd services and cleanup old systemd
>>>>>>>>> healthchecks for /var/lib/tripleo-config/container-startup-config/step_4 |
>>>>>>>>> undercloud | error={"changed": false, "msg": "Service ironic_pxe_tftp has
>>>>>>>>> not started yet"}
>>>>>>>>> 2022-06-14 05:03:22.914400 | 52540083-cfa2-3f20-e9dc-00000000286f
>>>>>>>>> | TIMING | tripleo_container_manage : Manage container systemd
>>>>>>>>>
>>>>>>>>> Sample Undercloud.conf is as follows:
>>>>>>>>>
>>>>>>>>> [DEFAULT]
>>>>>>>>> clean_nodes = true
>>>>>>>>> cleanup = false
>>>>>>>>> container_cli = podman
>>>>>>>>> container_healthcheck_disabled = true
>>>>>>>>> container_images_file =
>>>>>>>>> /home/stack/containers-prepare-parameter.yaml
>>>>>>>>> deployment_user = stack
>>>>>>>>> enable_ironic = true
>>>>>>>>> enable_ironic_inspector = true
>>>>>>>>> enable_neutron = true
>>>>>>>>> enable_routed_networks = false
>>>>>>>>> generate_service_certificate = false
>>>>>>>>> ipv6_address_mode = dhcpv6-stateful
>>>>>>>>> ipxe_enabled = true
>>>>>>>>> local_interface = enp8s0
>>>>>>>>> local_ip = aaaa:aaaa:aaaa::1/64
>>>>>>>>> subnets = ctlplane-subnet
>>>>>>>>> undercloud_admin_host = aaaa:aaaa:aaaa::1
>>>>>>>>> undercloud_hostname = undercloud.com
>>>>>>>>> undercloud_ntp_servers = 30.30.30.3
>>>>>>>>> undercloud_public_host = aaaa:aaaa:aaaa::1
>>>>>>>>> undercloud_timezone = UTC
>>>>>>>>>
>>>>>>>>> [ctlplane-subnet]
>>>>>>>>> cidr = aaaa:aaaa:aaaa::/64
>>>>>>>>> dhcp_end = aaaa:aaaa:aaaa::f
>>>>>>>>> dhcp_start = aaaa:aaaa:aaaa::a
>>>>>>>>> gateway = aaaa:aaaa:aaaa::1
>>>>>>>>> inspection_iprange = aaaa:aaaa:aaaa::3,aaaa:aaaa:aaaa::9
>>>>>>>>>
>>>>>>>>> Can someone please help in this regard.
>>>>>>>>>
>>>>>>>>> Anirudh Gupta
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>> --
>>>>> ~ Lokendra
>>>>> www.inertiaspeaks.com
>>>>> www.inertiagroups.com
>>>>> skype: lokendrarathour
>>>>>
>>>>>
>>>>>
>>>
>>> --
>>> ~ Lokendra
>>> www.inertiaspeaks.com
>>> www.inertiagroups.com
>>> skype: lokendrarathour
>>>
>>>
>>>
>
> --
> ~ Lokendra
> www.inertiaspeaks.com
> www.inertiagroups.com
> skype: lokendrarathour
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20220616/a7c572ff/attachment-0001.htm>
More information about the openstack-discuss
mailing list