Should ports created by ironic have PXE parameters after deployment?
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways. 1. If its volume based instance, delete the instance but not the volume. Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
Our openstack env automatically injects SSH keys) and already does that with all other images I have downloaded to deployed e.g fedora cloud images and ceros cloud image.
However core os is different and I have tried to edit grub added coreos.autologin=tty1 but nothing.
Also tried to do this via cloud-config
#cloud-config
coreos: units: - name: etcd.service command: start
users: - name: core passwd: coreos ssh_authorized_keys: - "ssh-rsa xxxxx"
And not luck - when vm boots it hangs.
On 16 Sep 2020 at 13:31:10, Florian Rommel florian@datalounges.com wrote:
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways.
- If its volume based instance, delete the instance but not the volume.
Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
On Wed, 2020-09-16 at 06:39 -0700, Michael STFC wrote:
Our openstack env automatically injects SSH keys) and already does that with all other images I have downloaded to deployed e.g fedora cloud images and ceros cloud image.
However core os is different and I have tried to edit grub added coreos.autologin=tty1 but nothing.
Also tried to do this via cloud-config
#cloud-config
coreos: units: - name: etcd.service command: start
users:
- name: core passwd: coreos ssh_authorized_keys:
- "ssh-rsa xxxxx"
And not luck - when vm boots it hangs.
Coreos does not use cloud config by default it uses ignition. i belive you can still configure it with cloud init but you have to do it slightly differnet then normal. https://coreos.com/os/docs/latest/booting-on-openstack.html#container-linux-... has the detail you need. basically you have to either pass an ignition script as the user data or Container Linux Config format.
cloud init wont work.
e.g. nova boot \ --user-data ./config.ign \ --image cdf3874c-c27f-4816-bc8c-046b240e0edd \ --key-name coreos \ --flavor m1.medium \ --min-count 3 \ --security-groups default,coreos
were ./config.ign is an ignition file.
On 16 Sep 2020 at 13:31:10, Florian Rommel florian@datalounges.com wrote:
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways.
- If its volume based instance, delete the instance but not the volume.
Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
Just as a side question, is there a benefit of ignition over cloud-init? (Not trying to start a flame war.. genuinely interested, and not trying to hijack the thread either)
//florian
On 16. Sep 2020, at 16.45, Sean Mooney smooney@redhat.com wrote:
On Wed, 2020-09-16 at 06:39 -0700, Michael STFC wrote:
Our openstack env automatically injects SSH keys) and already does that with all other images I have downloaded to deployed e.g fedora cloud images and ceros cloud image.
However core os is different and I have tried to edit grub added coreos.autologin=tty1 but nothing.
Also tried to do this via cloud-config
#cloud-config
coreos: units: - name: etcd.service command: start
users:
- name: core passwd: coreos ssh_authorized_keys:
- "ssh-rsa xxxxx"
And not luck - when vm boots it hangs.
Coreos does not use cloud config by default it uses ignition. i belive you can still configure it with cloud init but you have to do it slightly differnet then normal. https://coreos.com/os/docs/latest/booting-on-openstack.html#container-linux-... has the detail you need. basically you have to either pass an ignition script as the user data or Container Linux Config format.
cloud init wont work.
e.g. nova boot \ --user-data ./config.ign \ --image cdf3874c-c27f-4816-bc8c-046b240e0edd \ --key-name coreos \ --flavor m1.medium \ --min-count 3 \ --security-groups default,coreos
were ./config.ign is an ignition file.
On 16 Sep 2020 at 13:31:10, Florian Rommel florian@datalounges.com wrote:
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways.
- If its volume based instance, delete the instance but not the volume.
Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
On Wed, 2020-09-16 at 17:25 +0300, Florian Rommel wrote:
Just as a side question, is there a benefit of ignition over cloud-init? (Not trying to start a flame war.. genuinely interested, and not trying to hijack the thread either)
im not really sure. i think core os created ignition because of some limitation with cloud init
they use it for a lot more then jsut basic first boot setup. its used for all system configtion in container linux so i imagin they hit edgecases and developed ignition to adress those. most cloud image like ubuntu or fedora i think dont ship with ignition support out of the box. i did not have anything in my history on this topic but i did fine this https://coreos.com/ignition/docs/latest/what-is-ignition.html coreos are generally pretty good at documenting there deision are thigns like this also https://coreos.com/ignition/docs/latest/rationale.html
i really have not had much interaction with it however so in practis i dont know which is "better" in general.
//florian
On 16. Sep 2020, at 16.45, Sean Mooney smooney@redhat.com wrote:
On Wed, 2020-09-16 at 06:39 -0700, Michael STFC wrote:
Our openstack env automatically injects SSH keys) and already does that with all other images I have downloaded to deployed e.g fedora cloud images and ceros cloud image.
However core os is different and I have tried to edit grub added coreos.autologin=tty1 but nothing.
Also tried to do this via cloud-config
#cloud-config
coreos: units: - name: etcd.service command: start
users:
- name: core passwd: coreos ssh_authorized_keys:
- "ssh-rsa xxxxx"
And not luck - when vm boots it hangs.
Coreos does not use cloud config by default it uses ignition. i belive you can still configure it with cloud init but you have to do it slightly differnet then normal. https://coreos.com/os/docs/latest/booting-on-openstack.html#container-linux-... has the detail you need. basically you have to either pass an ignition script as the user data or Container Linux Config format.
cloud init wont work.
e.g. nova boot \ --user-data ./config.ign \ --image cdf3874c-c27f-4816-bc8c-046b240e0edd \ --key-name coreos \ --flavor m1.medium \ --min-count 3 \ --security-groups default,coreos
were ./config.ign is an ignition file.
On 16 Sep 2020 at 13:31:10, Florian Rommel florian@datalounges.com wrote:
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways.
- If its volume based instance, delete the instance but not the volume.
Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
Thank you Sean, appreciate it. Learned something new !:)
//Florian
On 16. Sep 2020, at 17.36, Sean Mooney smooney@redhat.com wrote:
On Wed, 2020-09-16 at 17:25 +0300, Florian Rommel wrote:
Just as a side question, is there a benefit of ignition over cloud-init? (Not trying to start a flame war.. genuinely interested, and not trying to hijack the thread either)
im not really sure. i think core os created ignition because of some limitation with cloud init
they use it for a lot more then jsut basic first boot setup. its used for all system configtion in container linux so i imagin they hit edgecases and developed ignition to adress those. most cloud image like ubuntu or fedora i think dont ship with ignition support out of the box. i did not have anything in my history on this topic but i did fine this https://coreos.com/ignition/docs/latest/what-is-ignition.html coreos are generally pretty good at documenting there deision are thigns like this also https://coreos.com/ignition/docs/latest/rationale.html
i really have not had much interaction with it however so in practis i dont know which is "better" in general.
//florian
On 16. Sep 2020, at 16.45, Sean Mooney smooney@redhat.com wrote:
On Wed, 2020-09-16 at 06:39 -0700, Michael STFC wrote:
Our openstack env automatically injects SSH keys) and already does that with all other images I have downloaded to deployed e.g fedora cloud images and ceros cloud image.
However core os is different and I have tried to edit grub added coreos.autologin=tty1 but nothing.
Also tried to do this via cloud-config
#cloud-config
coreos: units: - name: etcd.service command: start
users:
- name: core passwd: coreos ssh_authorized_keys:
- "ssh-rsa xxxxx"
And not luck - when vm boots it hangs.
Coreos does not use cloud config by default it uses ignition. i belive you can still configure it with cloud init but you have to do it slightly differnet then normal. https://coreos.com/os/docs/latest/booting-on-openstack.html#container-linux-... has the detail you need. basically you have to either pass an ignition script as the user data or Container Linux Config format.
cloud init wont work.
e.g. nova boot \ --user-data ./config.ign \ --image cdf3874c-c27f-4816-bc8c-046b240e0edd \ --key-name coreos \ --flavor m1.medium \ --min-count 3 \ --security-groups default,coreos
were ./config.ign is an ignition file.
On 16 Sep 2020 at 13:31:10, Florian Rommel florian@datalounges.com wrote:
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways.
- If its volume based instance, delete the instance but not the volume.
Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
Thanks - its sorted and many thanks for the advise.
On 16 Sep 2020, at 15:49, Florian Rommel florian@datalounges.com wrote:
Thank you Sean, appreciate it. Learned something new !:)
//Florian
On 16. Sep 2020, at 17.36, Sean Mooney smooney@redhat.com wrote:
On Wed, 2020-09-16 at 17:25 +0300, Florian Rommel wrote:
Just as a side question, is there a benefit of ignition over cloud-init? (Not trying to start a flame war.. genuinely interested, and not trying to hijack the thread either)
im not really sure. i think core os created ignition because of some limitation with cloud init
they use it for a lot more then jsut basic first boot setup. its used for all system configtion in container linux so i imagin they hit edgecases and developed ignition to adress those. most cloud image like ubuntu or fedora i think dont ship with ignition support out of the box. i did not have anything in my history on this topic but i did fine this https://coreos.com/ignition/docs/latest/what-is-ignition.html coreos are generally pretty good at documenting there deision are thigns like this also https://coreos.com/ignition/docs/latest/rationale.html
i really have not had much interaction with it however so in practis i dont know which is "better" in general.
//florian
On 16. Sep 2020, at 16.45, Sean Mooney smooney@redhat.com wrote:
On Wed, 2020-09-16 at 06:39 -0700, Michael STFC wrote:
Our openstack env automatically injects SSH keys) and already does that with all other images I have downloaded to deployed e.g fedora cloud images and ceros cloud image.
However core os is different and I have tried to edit grub added coreos.autologin=tty1 but nothing.
Also tried to do this via cloud-config
#cloud-config
coreos: units:
- name: etcd.service command: start
users:
- name: core
passwd: coreos ssh_authorized_keys:
- "ssh-rsa xxxxx"
And not luck - when vm boots it hangs.
Coreos does not use cloud config by default it uses ignition. i belive you can still configure it with cloud init but you have to do it slightly differnet then normal. https://coreos.com/os/docs/latest/booting-on-openstack.html#container-linux-... has the detail you need. basically you have to either pass an ignition script as the user data or Container Linux Config format.
cloud init wont work.
e.g. nova boot \ --user-data ./config.ign \ --image cdf3874c-c27f-4816-bc8c-046b240e0edd \ --key-name coreos \ --flavor m1.medium \ --min-count 3 \ --security-groups default,coreos
were ./config.ign is an ignition file.
On 16 Sep 2020 at 13:31:10, Florian Rommel florian@datalounges.com wrote:
Hi Michael. So, if I remember coreOS correctly, its the same as all of the cloud based images. It uses SSH keys to authenticate. If you have a an SSH public key in there where you do no longer have the private key for, you can “easily” reset it by 2 ways.
- If its volume based instance, delete the instance but not the volume.
Create the instance again by adding your own ssh key into the boot process. This will ADD the ssh key, but not overwrite the existing one in the authorized_key file 2. If it is normal ephermal disk based instance, make a snapshot and create a new instance from the snapshot, adding your own ssh key into it.
Either or, if they are ssh key authenticated (which they should be), there isn’t really an EASY way unless you want to have the volume directly.
Best regards, //Florian
On 16. Sep 2020, at 13.53, Michael STFC mtint.stfc@gmail.com wrote:
Hi
New to openstack and wanting to know how to get boot core os and reset user core password.
Please advise.
Michael
On 2020-09-16 17:25:11 +0300 (+0300), Florian Rommel wrote:
Just as a side question, is there a benefit of ignition over cloud-init? (Not trying to start a flame war.. genuinely interested, and not trying to hijack the thread either)
[...]
Throwing fuel on the fire, the OpenDev Collaboratory uses this alternative on their CI nodes, mainly as a reaction to cloud-init's massive dependency sprawl: https://docs.openstack.org/infra/glean/
Normally yes but I am having the PXE added to NON provision ports as well.
I tore down the dnsmasq and inspector containers, rediscovered the hosts and it hasn’t came back.. but that still doesn’t answer how that could happen. On Sep 16, 2020, 3:53 AM -0400, Mark Goddard mark@stackhpc.com, wrote:
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
I guess we need to understand if your machines are set to network boot by default in ironic's configuration? If it is set to the flat network_interface and the instances are configured for network booting? If so, I'd expect this to happen for a deployed instance.
Out of curiosity, is this master branch code? Ussuri? Are the other environments the same?
-Julia
On Wed, Sep 16, 2020 at 11:33 AM Tyler Bishop tbishop@liquidweb.com wrote:
Normally yes but I am having the PXE added to NON provision ports as well.
I tore down the dnsmasq and inspector containers, rediscovered the hosts and it hasn’t came back.. but that still doesn’t answer how that could happen. On Sep 16, 2020, 3:53 AM -0400, Mark Goddard mark@stackhpc.com, wrote:
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
Hi Julia,
All of these are latest stable train using Kolla-ansible.
I am using local disk booting for all deployed instances and we utilize neutron networking with plans.
Attached screenshot of driver config.
On Sep 16, 2020, 2:53 PM -0400, Julia Kreger juliaashleykreger@gmail.com, wrote:
I guess we need to understand if your machines are set to network boot by default in ironic's configuration? If it is set to the flat network_interface and the instances are configured for network booting? If so, I'd expect this to happen for a deployed instance.
Out of curiosity, is this master branch code? Ussuri? Are the other environments the same?
-Julia
On Wed, Sep 16, 2020 at 11:33 AM Tyler Bishop tbishop@liquidweb.com wrote:
Normally yes but I am having the PXE added to NON provision ports as well.
I tore down the dnsmasq and inspector containers, rediscovered the hosts and it hasn’t came back.. but that still doesn’t answer how that could happen. On Sep 16, 2020, 3:53 AM -0400, Mark Goddard mark@stackhpc.com, wrote:
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
(Resending after stripping the image out for the mailing list, hopefully!)
Well, I <3 that your using the ironic horizon plugin!
Can you confirm the contents of the flavors you are using for scheduling, specifically the capabilities field.
Are you using whole disk images, or are you using partition images?
On Wed, Sep 16, 2020 at 2:13 PM Tyler Bishop tbishop@liquidweb.com wrote:
Hi Julia,
All of these are latest stable train using Kolla-ansible.
I am using local disk booting for all deployed instances and we utilize neutron networking with plans.
Attached screenshot of driver config.
On Sep 16, 2020, 2:53 PM -0400, Julia Kreger juliaashleykreger@gmail.com, wrote:
I guess we need to understand if your machines are set to network boot by default in ironic's configuration? If it is set to the flat network_interface and the instances are configured for network booting? If so, I'd expect this to happen for a deployed instance.
Out of curiosity, is this master branch code? Ussuri? Are the other environments the same?
-Julia
On Wed, Sep 16, 2020 at 11:33 AM Tyler Bishop tbishop@liquidweb.com wrote:
Normally yes but I am having the PXE added to NON provision ports as well.
I tore down the dnsmasq and inspector containers, rediscovered the hosts and it hasn’t came back.. but that still doesn’t answer how that could happen. On Sep 16, 2020, 3:53 AM -0400, Mark Goddard mark@stackhpc.com, wrote:
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
Thanks Julia
While investigating this further I found that if I delete the host then re-discovery it the issue goes away. So something I’ve done in my deployment has broken this on the other hosts. I need to open up the database and start digging around the tables to figure out if there is any differences in the two enrolled host.
Yes I use kolla-ansible which natively deploys the ironic dashboard. It seems to work pretty good but its very noisy with errors during host state change constantly throwing errors on the page even though things are progressing as normal.
On Sep 16, 2020, 6:38 PM -0400, Julia Kreger juliaashleykreger@gmail.com, wrote:
(Resending after stripping the image out for the mailing list, hopefully!)
Well, I <3 that your using the ironic horizon plugin!
Can you confirm the contents of the flavors you are using for scheduling, specifically the capabilities field.
Are you using whole disk images, or are you using partition images?
On Wed, Sep 16, 2020 at 2:13 PM Tyler Bishop tbishop@liquidweb.com wrote:
Hi Julia,
All of these are latest stable train using Kolla-ansible.
I am using local disk booting for all deployed instances and we utilize neutron networking with plans.
Attached screenshot of driver config.
On Sep 16, 2020, 2:53 PM -0400, Julia Kreger juliaashleykreger@gmail.com, wrote:
I guess we need to understand if your machines are set to network boot by default in ironic's configuration? If it is set to the flat network_interface and the instances are configured for network booting? If so, I'd expect this to happen for a deployed instance.
Out of curiosity, is this master branch code? Ussuri? Are the other environments the same?
-Julia
On Wed, Sep 16, 2020 at 11:33 AM Tyler Bishop tbishop@liquidweb.com wrote:
Normally yes but I am having the PXE added to NON provision ports as well.
I tore down the dnsmasq and inspector containers, rediscovered the hosts and it hasn’t came back.. but that still doesn’t answer how that could happen. On Sep 16, 2020, 3:53 AM -0400, Mark Goddard mark@stackhpc.com, wrote:
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
Well... That is new! (The different port behavior)
I wonder if the port data in question was already on the port from a prior introspection, or maybe a service got unexpected in a really unexpected way. I guess without figuring out the exact state things are in nor being able to reproduce it, it is going to be difficult to pin this down.
Re: The dashboard. I remember it locks some fields/options on unstable states. Is that the error you're seeing?
-Julia
On Wed, Sep 16, 2020 at 5:16 PM Tyler Bishop tbishop@liquidweb.com wrote:
Thanks Julia
While investigating this further I found that if I delete the host then re-discovery it the issue goes away. So something I’ve done in my deployment has broken this on the other hosts. I need to open up the database and start digging around the tables to figure out if there is any differences in the two enrolled host.
Yes I use kolla-ansible which natively deploys the ironic dashboard. It seems to work pretty good but its very noisy with errors during host state change constantly throwing errors on the page even though things are progressing as normal.
On Sep 16, 2020, 6:38 PM -0400, Julia Kreger juliaashleykreger@gmail.com, wrote:
(Resending after stripping the image out for the mailing list, hopefully!)
Well, I <3 that your using the ironic horizon plugin!
Can you confirm the contents of the flavors you are using for scheduling, specifically the capabilities field.
Are you using whole disk images, or are you using partition images?
On Wed, Sep 16, 2020 at 2:13 PM Tyler Bishop tbishop@liquidweb.com wrote:
Hi Julia,
All of these are latest stable train using Kolla-ansible.
I am using local disk booting for all deployed instances and we utilize neutron networking with plans.
Attached screenshot of driver config.
On Sep 16, 2020, 2:53 PM -0400, Julia Kreger juliaashleykreger@gmail.com, wrote:
I guess we need to understand if your machines are set to network boot by default in ironic's configuration? If it is set to the flat network_interface and the instances are configured for network booting? If so, I'd expect this to happen for a deployed instance.
Out of curiosity, is this master branch code? Ussuri? Are the other environments the same?
-Julia
On Wed, Sep 16, 2020 at 11:33 AM Tyler Bishop tbishop@liquidweb.com wrote:
Normally yes but I am having the PXE added to NON provision ports as well.
I tore down the dnsmasq and inspector containers, rediscovered the hosts and it hasn’t came back.. but that still doesn’t answer how that could happen. On Sep 16, 2020, 3:53 AM -0400, Mark Goddard mark@stackhpc.com, wrote:
On Tue, 15 Sep 2020 at 20:13, Tyler Bishop tbishop@liquidweb.com wrote:
Hi,
My issue is i have a neutron network (not discovery or cleaning) that is adding the PXE entries for the ironic pxe server and my baremetal host are rebooting into discovery upon successful deployment.
I am curious how the driver implementation works for adding the PXE options to neutron-dhcp-agent configuration and if that is being done to help non flat networks where no SDN is being used? I have several environments using Kolla-Ansible and this one seems to be the only behaving like this. My neutron-dhcp-agent dnsmasq opt file looks like this after a host is deployed.
dhcp/7d0b7e78-6506-4f4a-b524-d5c03e4ca4a8/opts cat /var/lib/neutron/dhcp/ffdf5f9b-b4ad-4a53-b154-69eb3b4a81c5/opts tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:dns-server,10.60.3.240,10.60.10.240,10.60.1.240 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:classless-static-route,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,249,169.254.169.254/32,10.60.66.2,0.0.0.0/0,10.60.66.1 tag:subnet-57b772c1-7878-4458-8c60-cf21eac99ac2,option:router,10.60.66.1 tag:port-08908db1-360b-4973-87c7-15049a484ac6,150,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,210,/tftpboot/ tag:port-08908db1-360b-4973-87c7-15049a484ac6,66,10.60.66.11 tag:port-08908db1-360b-4973-87c7-15049a484ac6,67,pxelinux.0 tag:port-08908db1-360b-4973-87c7-15049a484ac6,option:server-ip-address,10.60.66.11
Hi Tyler, Ironic adds DHCP options to the neutron port on the provisioning network. Specifically, the boot interface in ironic is responsible for adding DHCP options. See the PXEBaseMixin class.
participants (7)
-
Florian Rommel
-
Jeremy Stanley
-
Julia Kreger
-
Mark Goddard
-
Michael STFC
-
Sean Mooney
-
Tyler Bishop