Open Stack

On Tue, 15 Jun 2021 at 06:51, Tony Pearce <tonyppe at gmail.com> wrote:
>
> Hi Mark,
>
> I had never used the "pip install ." method. Maybe a miscomprehension on my side, from the documentation [1] there are three ways to install kayobe. I had opted for the first way which is "pip install kayobe" since January 2020. The understanding was as conveyed in the doc "Installing from PyPI ensures the use of well used and tested software".

That is true, but since Wallaby has not been released for Kayobe yet,
it is not on PyPI. If you do install from PyPI, I would advise using a
version constraint to ensure you get the release series you need.

>
> I have since followed your steps in your mail which is the installation from source. I had new problems:
>
> During ansible bootstrap:
> During ansible host bootstrap it errors out and says the kolla_ansible is not found and needs to be installed in the same virtual environment. In all previous times, I had understood that kolla ansible is installed by kayobe at this point. I eventually done "pip install kolla-ansible" and it seemed to take care of that and allowed me to move on to "host configure"

Kolla Ansible should be installed automatically during 'kayobe control
host bootstrap', in a separate virtualenv from Kayobe. You should not
need to install it manually, and I would again advise against doing so
without version constraints.

>
> During host configure:
> I was able to get past the previous python issue but then it failed on the network due to a "duplicate bond name", though this config was deployed successfully in Train. I dont think I really need a bond at this point so I deleted the bond and the host configure is now successful. (fyi this is an all-in-one host.)
>
> During kayobe service deploy:
> This then fails with "no module named docker" on the host. To troubleshoot this I logged into the host and activated the kayobe virtual env (/opt/kayobe/venvs/kayobe/bin/activate) and then "pip install docker". It was already installed. Eventually, I issued "pip install --ignore-installed docker" within these three (environment) locations which resolved this and allowed the kayobe command to complete successfully and progress further:
> - /opt/kayobe/venvs/kayobe/
> - /opt/kayobe/venvs/kolla-ansible/
> - native on the host after deactivating the venv.
>
> Now the blocker is the following failure;
>
> TASK [nova-cell : Waiting for nova-compute services to register themselves] **********************************************************************************************
> FAILED - RETRYING: Waiting for nova-compute services to register themselves (20 retries left).
> FAILED - RETRYING: Waiting for nova-compute services to register themselves (19 retries left).
>
> I haven't seen this one before but previously I had seen something similar with mariadb because the API dns was not available. What I have been using here is a /etc/hosts entry for this. I checked that this entry is available on the host and in the nova containers. I decided to reboot the host anyway (previously resolved similar mariadb issue) to restart the containers just in case the dns was not available in one of them and I missed it.

I'd check the nova compute logs here, to find why they are not
registering themselves.
>
> Unfortunately I now have two additional issues which are hard blockers:
> 1. The network is no longer working on the host after reboot, so I am unable to ssh
> 2. The user password has been changed by kayobe, so I am unable to login using the console
>
> Due to the above, I am unable to login to the host to investigate or remediate. Previously when this happened with centos I could use the root user to log in. This time around as it's ubuntu I do not have a root user.
> The user I am using for both "kolla_ansible_user" and "kayobe_ansible_user" is the same - is this causing a problem with Victoria and Wallaby? I had this user password change issue beginning with Victoria.
>
> So at this point I need to re-install the host and go back to the host configure before service deploy.
>
> Summary
> Any guidance is well appreciated as I'm at a loss at this point. Last week I had a working Openstack Train deployment in a single host. "Kayobe" stopped working (maybe because I had previously always used pip install kayobe).
>
> I would like to deploy Wallaby, should I be able to successfully do this today or should I be using Victoria at the moment (or even, Train)?

We are very close to release of Wallaby, and I expect that it should
generally work, but Ubuntu is a new distro for Kayobe, and Wallaby is
a new release. There may be teething problems, so if you're looking
for something more stable then I'd suggest CentOS & Victoria.

>
> [1] OpenStack Docs: Installation
>
> Regards,
>
> Tony Pearce
>
>
> On Mon, 14 Jun 2021 at 18:36, Mark Goddard <mark at stackhpc.com> wrote:
>>
>> On Mon, 14 Jun 2021 at 09:40, Tony Pearce <tonyppe at gmail.com> wrote:
>> >
>> > Hi Mark,
>> >
>> > I followed this guide to do a "git clone" specifying the branch "-b" to "stable/wallaby" [1]. What additional steps do I need to do to get the latest commits?
>>
>> That should be sufficient. When you install it via pip, note that 'pip
>> install kayobe' will still pull from PyPI, even if there is a local
>> kayobe directory. Use ./kayobe, or 'pip install .'  if in the same
>> directory.
>>
>> Mark
>> >
>> > [1] OpenStack Docs: Overcloud
>> >
>> > Kind regards,
>> >
>> > Tony Pearce
>> >
>> >
>> > On Mon, 14 Jun 2021 at 16:10, Mark Goddard <mark at stackhpc.com> wrote:
>> >>
>> >> On Mon, 14 Jun 2021 at 07:21, Tony Pearce <tonyppe at gmail.com> wrote:
>> >> >
>> >> > Hi Pierre, thanks for replying to my message.
>> >> >
>> >> > To install kayobe I followed the documentation which summarise: installing a few system packages and setting up the kayobe virtual environment and then pulling the correct kayobe git version for the openstack to be installed. After configuring the yaml files I have run these commands:
>> >> >
>> >> > - kayobe control host bootstrap
>> >> > - kayobe overcloud host configure -> this one is failing with /usr/libexec/platform-python: not found
>> >> >
>> >> > After reading your message on the weekend I concluded that maybe I had done something wrong. Today, I re-pulled the kayobe wallaby git and manually transferred the configuration over to the new directory structure on the ansible host and set up again as per the guide but the same issue is seen.
>> >> >
>> >> > What I ended up doing to try and resolve was finding where this "platform-python" is coming from. It is coming from the virtual environment which is being set up during the kayobe ansible host bootstrap. Initially, I found the base.yml and it looks like it tries to match what the host is. I noticed that there is no ubuntu 20 listed there so I created it however it did not resolve the issue.
>> >> >
>> >> > So then I tried systematically replacing this reference in the other files found in the same location "venvs\kayobe\share\kayobe\ansible". The file I changed which allowed it to progress is "kayobe-target-venv.yml"
>> >> >
>> >> > But unfortunately it fails a bit further on, failing to find an selinux package [1]
>> >> >
>> >> > Seeing as the error is mentioning selinux (a RedHat security feature not installed on ubuntu) could the root cause issue be that kayobe is not matching the host as ubuntu? I did already set in kayobe that I am using ubuntu OS distribution within globals.yml [2].
>> >> >
>> >> > Are there any extra steps that I need to complete that maybe are not listed in the documentation / guide?
>> >> >
>> >> > [1] TASK [MichaelRigart.interfaces : Debian | install current/latest network package - Pastebin.com
>> >> > [2] ---# Kayobe global configuration.######################################### - Pastebin.com
>> >>
>> >> Hi Tony,
>> >>
>> >> That's definitely not a recent Wallaby checkout you're using. Ubuntu
>> >> no longer uses that MichaelRigart.interfaces role. Check that you have
>> >> recent commits. Here is the most recent on stable/wallaby:
>> >> 13169077aaec0f7a28ae1f15b419dafc2456faf7.
>> >>
>> >> Mark
>> >>
>> >> >
>> >> > Regards,
>> >> >
>> >> > Tony Pearce
>> >> >
>> >> >
>> >> >
>> >> > On Fri, 11 Jun 2021 at 21:05, Pierre Riteau <pierre at stackhpc.com> wrote:
>> >> >>
>> >> >> Hi Tony,
>> >> >>
>> >> >> Kayobe doesn't use platform-python anymore, on both stable/wallaby and
>> >> >> stable/victoria:
>> >> >> https://review.opendev.org/q/I0d477325e0edd13d1aba211c13dc2e8b7a9b4c98
>> >> >>
>> >> >> Can you double-check what version you are using, and share how you
>> >> >> installed it? Note that only stable/wallaby supports Ubuntu 20 hosts.
>> >> >>
>> >> >> Best wishes,
>> >> >> Pierre
>> >> >>
>> >> >> On Fri, 11 Jun 2021 at 13:20, Tony Pearce <tonyppe at gmail.com> wrote:
>> >> >> >
>> >> >> > I'm trying to run "kayobe overcloud host configure" against an ubuntu 20 machine to deploy Wallaby. I'm getting an error that python is not found during the host configure part.
>> >> >> >
>> >> >> > PLAY [Verify that the Kayobe Ansible user account is accessible]
>> >> >> > TASK [Verify that a command can be executed]
>> >> >> >
>> >> >> > fatal: [juc-ucsb-5-p]: FAILED! => {"changed": false, "module_stderr": "/bin/sh: 1: /usr/libexec/platform-python: not found\n", "module_stdout": "", "msg": "The module failed to execute correctly, you probably need to set the interpreter.\nSee stdout/stderr for the exact error", "rc": 127}
>> >> >> >
>> >> >> > Python3 is installed on the host. When searching where this platform-python is coming from it returns the kolla-ansible virtual envs:
>> >> >> >
>> >> >> > $ grep -rni -e "platform-python"
>> >> >> > venvs/kolla-ansible/lib/python3.8/site-packages/ansible/config/base.yml:1450:      '8': /usr/libexec/platform-python
>> >> >> > venvs/kolla-ansible/lib/python3.8/site-packages/ansible/config/base.yml:1470:  - /usr/libexec/platform-python
>> >> >> >
>> >> >> > I had a look through the deployment guide for Kayobe Wallaby and didnt see a note about changing this.
>> >> >> >
>> >> >> > Do I need to do further steps to support the ubuntu overcloud host? I have already set (as per the doc):
>> >> >> >
>> >> >> > os_distribution: ubuntu
>> >> >> > os_release: focal
>> >> >> >
>> >> >> > Regards,
>> >> >> >
>> >> >> > Tony Pearce
>> >> >> >