Wallaby install via kayobe onto ubuntu 20 all in one host

Mark Goddard mark at stackhpc.com
Wed Jun 16 08:09:11 UTC 2021


On Wed, 16 Jun 2021 at 07:09, Tony Pearce <tonyppe at gmail.com> wrote:
>
> Hi Mark, thanks again for your reply and direction on this.
>
> Regarding the need to do a pip install within the venv, I re-installed the host and re-installed kayobe and this issue is no longer present as of yesterday afternoon.
>
> Regarding the "Waiting for nova-compute services to register themselves" - I tailed *.log in the kolla/nova log dir and to be honest there wasnt much being logged at that time and no errors, either.
>
> Regarding Wallaby hasnt yet been released, I went back to Victoria on Centos 8.4 this morning, using the "pip install ." (installation from source). I will use this install from source going forward to make sure I am getting the latest kayobe patches. I had two blockers during "host configure":
>
> First was that python was not found on the host:
>
> TASK [Verify that a command can be executed] *****************************************************************************************************************************
> fatal: [juc-ucsb-5-p]: FAILED! => {"changed": false, "module_stderr": "/bin/sh: /usr/bin/python3: No such file or directory\n", "module_stdout": "", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}

In stable/victoria there is a task called 'Ensure python is installed'
which should ensure that Python 3 is installed. Did that task run?

>
> So I manually installed python to see what would happen. I was able to get past that point but then it fails again:
>
> TASK [Ensure the Python virtualenv package is installed] *****************************************************************************************************************
> [WARNING]: Updating cache and auto-installing missing dependency: python3-apt
> fatal: [juc-ucsb-5-p]: FAILED! => {"changed": false, "cmd": "apt-get update", "msg": "[Errno 2] No such file or directory: b'apt-get': b'apt-get'", "rc": 2}
>
> PLAY RECAP ***************************************************************************************************************************************************************
> juc-ucsb-5-p               : ok=7    changed=0    unreachable=0    failed=1    skipped=10   rescued=0    ignored=0
>
> The above log suggests that the CentOS 8.4 host is being matched as ubuntu. As an FYI, what I mean when I say "went back to Victoria" is that I followed the initial kayobe install steps again with new top directory structure and then manually transferred over the kayobe config to the new empty config which is downloaded during the git clone. "manually transferred" means I opened each yml config file individually and copied over only the variables that have been configured within the file, making sure to leave defaults as a prefernce.

Something odd is clearly going on here. We use the package module,
which uses ansible's ansible_pkg_mgr fact to dispatch the correct
packaging module. Perhaps you have fact caching enabled, and have
stale facts from when the host was running Ubuntu?

>
> After the above issue I decided to go back to Train and have been able to successfully deploy Openstack using kayobe Train today - no blocking issues, first time.
>
> Summary
> I really appreciate your time to help me with this, so thanks again. I Still think kayobe is the best way to deploy and manage openstack and I have been "telling my friends" also :) Now I am back to this point I just have some minor things to work through before I can start using it.
>
> Kind regards,
>
> Tony Pearce
>
>
> On Tue, 15 Jun 2021 at 16:02, Mark Goddard <mark at stackhpc.com> wrote:
>>
>> On Tue, 15 Jun 2021 at 06:51, Tony Pearce <tonyppe at gmail.com> wrote:
>> >
>> > Hi Mark,
>> >
>> > I had never used the "pip install ." method. Maybe a miscomprehension on my side, from the documentation [1] there are three ways to install kayobe. I had opted for the first way which is "pip install kayobe" since January 2020. The understanding was as conveyed in the doc "Installing from PyPI ensures the use of well used and tested software".
>>
>> That is true, but since Wallaby has not been released for Kayobe yet,
>> it is not on PyPI. If you do install from PyPI, I would advise using a
>> version constraint to ensure you get the release series you need.
>>
>> >
>> > I have since followed your steps in your mail which is the installation from source. I had new problems:
>> >
>> > During ansible bootstrap:
>> > During ansible host bootstrap it errors out and says the kolla_ansible is not found and needs to be installed in the same virtual environment. In all previous times, I had understood that kolla ansible is installed by kayobe at this point. I eventually done "pip install kolla-ansible" and it seemed to take care of that and allowed me to move on to "host configure"
>>
>> Kolla Ansible should be installed automatically during 'kayobe control
>> host bootstrap', in a separate virtualenv from Kayobe. You should not
>> need to install it manually, and I would again advise against doing so
>> without version constraints.
>>
>> >
>> > During host configure:
>> > I was able to get past the previous python issue but then it failed on the network due to a "duplicate bond name", though this config was deployed successfully in Train. I dont think I really need a bond at this point so I deleted the bond and the host configure is now successful. (fyi this is an all-in-one host.)
>> >
>> > During kayobe service deploy:
>> > This then fails with "no module named docker" on the host. To troubleshoot this I logged into the host and activated the kayobe virtual env (/opt/kayobe/venvs/kayobe/bin/activate) and then "pip install docker". It was already installed. Eventually, I issued "pip install --ignore-installed docker" within these three (environment) locations which resolved this and allowed the kayobe command to complete successfully and progress further:
>> > - /opt/kayobe/venvs/kayobe/
>> > - /opt/kayobe/venvs/kolla-ansible/
>> > - native on the host after deactivating the venv.
>> >
>> > Now the blocker is the following failure;
>> >
>> > TASK [nova-cell : Waiting for nova-compute services to register themselves] **********************************************************************************************
>> > FAILED - RETRYING: Waiting for nova-compute services to register themselves (20 retries left).
>> > FAILED - RETRYING: Waiting for nova-compute services to register themselves (19 retries left).
>> >
>> > I haven't seen this one before but previously I had seen something similar with mariadb because the API dns was not available. What I have been using here is a /etc/hosts entry for this. I checked that this entry is available on the host and in the nova containers. I decided to reboot the host anyway (previously resolved similar mariadb issue) to restart the containers just in case the dns was not available in one of them and I missed it.
>>
>> I'd check the nova compute logs here, to find why they are not
>> registering themselves.
>> >
>> > Unfortunately I now have two additional issues which are hard blockers:
>> > 1. The network is no longer working on the host after reboot, so I am unable to ssh
>> > 2. The user password has been changed by kayobe, so I am unable to login using the console
>> >
>> > Due to the above, I am unable to login to the host to investigate or remediate. Previously when this happened with centos I could use the root user to log in. This time around as it's ubuntu I do not have a root user.
>> > The user I am using for both "kolla_ansible_user" and "kayobe_ansible_user" is the same - is this causing a problem with Victoria and Wallaby? I had this user password change issue beginning with Victoria.
>> >
>> > So at this point I need to re-install the host and go back to the host configure before service deploy.
>> >
>> > Summary
>> > Any guidance is well appreciated as I'm at a loss at this point. Last week I had a working Openstack Train deployment in a single host. "Kayobe" stopped working (maybe because I had previously always used pip install kayobe).
>> >
>> > I would like to deploy Wallaby, should I be able to successfully do this today or should I be using Victoria at the moment (or even, Train)?
>>
>> We are very close to release of Wallaby, and I expect that it should
>> generally work, but Ubuntu is a new distro for Kayobe, and Wallaby is
>> a new release. There may be teething problems, so if you're looking
>> for something more stable then I'd suggest CentOS & Victoria.
>>
>> >
>> > [1] OpenStack Docs: Installation
>> >
>> > Regards,
>> >
>> > Tony Pearce
>> >
>> >
>> > On Mon, 14 Jun 2021 at 18:36, Mark Goddard <mark at stackhpc.com> wrote:
>> >>
>> >> On Mon, 14 Jun 2021 at 09:40, Tony Pearce <tonyppe at gmail.com> wrote:
>> >> >
>> >> > Hi Mark,
>> >> >
>> >> > I followed this guide to do a "git clone" specifying the branch "-b" to "stable/wallaby" [1]. What additional steps do I need to do to get the latest commits?
>> >>
>> >> That should be sufficient. When you install it via pip, note that 'pip
>> >> install kayobe' will still pull from PyPI, even if there is a local
>> >> kayobe directory. Use ./kayobe, or 'pip install .'  if in the same
>> >> directory.
>> >>
>> >> Mark
>> >> >
>> >> > [1] OpenStack Docs: Overcloud
>> >> >
>> >> > Kind regards,
>> >> >
>> >> > Tony Pearce
>> >> >
>> >> >
>> >> > On Mon, 14 Jun 2021 at 16:10, Mark Goddard <mark at stackhpc.com> wrote:
>> >> >>
>> >> >> On Mon, 14 Jun 2021 at 07:21, Tony Pearce <tonyppe at gmail.com> wrote:
>> >> >> >
>> >> >> > Hi Pierre, thanks for replying to my message.
>> >> >> >
>> >> >> > To install kayobe I followed the documentation which summarise: installing a few system packages and setting up the kayobe virtual environment and then pulling the correct kayobe git version for the openstack to be installed. After configuring the yaml files I have run these commands:
>> >> >> >
>> >> >> > - kayobe control host bootstrap
>> >> >> > - kayobe overcloud host configure -> this one is failing with /usr/libexec/platform-python: not found
>> >> >> >
>> >> >> > After reading your message on the weekend I concluded that maybe I had done something wrong. Today, I re-pulled the kayobe wallaby git and manually transferred the configuration over to the new directory structure on the ansible host and set up again as per the guide but the same issue is seen.
>> >> >> >
>> >> >> > What I ended up doing to try and resolve was finding where this "platform-python" is coming from. It is coming from the virtual environment which is being set up during the kayobe ansible host bootstrap. Initially, I found the base.yml and it looks like it tries to match what the host is. I noticed that there is no ubuntu 20 listed there so I created it however it did not resolve the issue.
>> >> >> >
>> >> >> > So then I tried systematically replacing this reference in the other files found in the same location "venvs\kayobe\share\kayobe\ansible". The file I changed which allowed it to progress is "kayobe-target-venv.yml"
>> >> >> >
>> >> >> > But unfortunately it fails a bit further on, failing to find an selinux package [1]
>> >> >> >
>> >> >> > Seeing as the error is mentioning selinux (a RedHat security feature not installed on ubuntu) could the root cause issue be that kayobe is not matching the host as ubuntu? I did already set in kayobe that I am using ubuntu OS distribution within globals.yml [2].
>> >> >> >
>> >> >> > Are there any extra steps that I need to complete that maybe are not listed in the documentation / guide?
>> >> >> >
>> >> >> > [1] TASK [MichaelRigart.interfaces : Debian | install current/latest network package - Pastebin.com
>> >> >> > [2] ---# Kayobe global configuration.######################################### - Pastebin.com
>> >> >>
>> >> >> Hi Tony,
>> >> >>
>> >> >> That's definitely not a recent Wallaby checkout you're using. Ubuntu
>> >> >> no longer uses that MichaelRigart.interfaces role. Check that you have
>> >> >> recent commits. Here is the most recent on stable/wallaby:
>> >> >> 13169077aaec0f7a28ae1f15b419dafc2456faf7.
>> >> >>
>> >> >> Mark
>> >> >>
>> >> >> >
>> >> >> > Regards,
>> >> >> >
>> >> >> > Tony Pearce
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > On Fri, 11 Jun 2021 at 21:05, Pierre Riteau <pierre at stackhpc.com> wrote:
>> >> >> >>
>> >> >> >> Hi Tony,
>> >> >> >>
>> >> >> >> Kayobe doesn't use platform-python anymore, on both stable/wallaby and
>> >> >> >> stable/victoria:
>> >> >> >> https://review.opendev.org/q/I0d477325e0edd13d1aba211c13dc2e8b7a9b4c98
>> >> >> >>
>> >> >> >> Can you double-check what version you are using, and share how you
>> >> >> >> installed it? Note that only stable/wallaby supports Ubuntu 20 hosts.
>> >> >> >>
>> >> >> >> Best wishes,
>> >> >> >> Pierre
>> >> >> >>
>> >> >> >> On Fri, 11 Jun 2021 at 13:20, Tony Pearce <tonyppe at gmail.com> wrote:
>> >> >> >> >
>> >> >> >> > I'm trying to run "kayobe overcloud host configure" against an ubuntu 20 machine to deploy Wallaby. I'm getting an error that python is not found during the host configure part.
>> >> >> >> >
>> >> >> >> > PLAY [Verify that the Kayobe Ansible user account is accessible]
>> >> >> >> > TASK [Verify that a command can be executed]
>> >> >> >> >
>> >> >> >> > fatal: [juc-ucsb-5-p]: FAILED! => {"changed": false, "module_stderr": "/bin/sh: 1: /usr/libexec/platform-python: not found\n", "module_stdout": "", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}
>> >> >> >> >
>> >> >> >> > Python3 is installed on the host. When searching where this platform-python is coming from it returns the kolla-ansible virtual envs:
>> >> >> >> >
>> >> >> >> > $ grep -rni -e "platform-python"
>> >> >> >> > venvs/kolla-ansible/lib/python3.8/site-packages/ansible/config/base.yml:1450:      '8': /usr/libexec/platform-python
>> >> >> >> > venvs/kolla-ansible/lib/python3.8/site-packages/ansible/config/base.yml:1470:  - /usr/libexec/platform-python
>> >> >> >> >
>> >> >> >> > I had a look through the deployment guide for Kayobe Wallaby and didnt see a note about changing this.
>> >> >> >> >
>> >> >> >> > Do I need to do further steps to support the ubuntu overcloud host? I have already set (as per the doc):
>> >> >> >> >
>> >> >> >> > os_distribution: ubuntu
>> >> >> >> > os_release: focal
>> >> >> >> >
>> >> >> >> > Regards,
>> >> >> >> >
>> >> >> >> > Tony Pearce
>> >> >> >> >



More information about the openstack-discuss mailing list