<div dir="ltr"><div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Hey Feilong,</div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Thanks a lot.</div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">The services are fine and indeed the log files are there in the directory [<span style="color:rgb(34,34,34)">/var/log/heat-config/heat-</span><span style="color:rgb(34,34,34)">config-script</span>]</div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">After checking, the master log is fine but the cluster log has this error below as I had mentioned earlier</div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><div class="gmail_default">Starting to run kube-apiserver-to-kubelet-role<br>+ echo 'Waiting for Kubernetes API...'<br>Waiting for Kubernetes API...<br>++ kubectl get --raw=/healthz<br>The connection to the server localhost:8080 was refused - did you specify the right host or port?<br>+ '[' ok = '' ']'<br></div><br class="gmail-Apple-interchange-newline"></div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br></div><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div><font face="arial, sans-serif">Regards</font></div><div><font face="arial, sans-serif"><br></font></div><div dir="ltr"><font face="arial, sans-serif">Tony Karera</font></div><div dir="ltr"><br></div><div><br></div></div></div></div></div></div></div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 31, 2021 at 9:52 PM feilong <<a href="mailto:feilong@catalyst.net.nz">feilong@catalyst.net.nz</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div>
    <p>Hi Karea,</p>
    <p>Given you can see heat-container-agent container from podman
      which means you should be able to see logs from below path:</p>
    <p>[root@k8s-100-eh6s5l6d73ie-master-0 heat-config-script]# ls<br>
c64e6ac2-db7e-4786-a387-1d45359812b8-k8s-100-eh6s5l6d73ie-kube_cluster_config-uxpsylgnayjy.log<br>
fa1f6247-51a8-4e70-befa-cbc61ee99e59-k8s-100-eh6s5l6d73ie-kube_masters-kmi423lgbjw3-0-oii7uzemq7aj-master_config-dhfam54i456j.log<br>
      [root@k8s-100-eh6s5l6d73ie-master-0 heat-config-script]# pwd<br>
      /var/log/heat-config/heat-config-script<br>
    </p>
    <p>If you can not see the path and the log, then it means the
      heat-container-agent didn't work well. You need to check the
      service status by systemctl command and check the log by
      journalctl. From there, you should be able to see why the cluster
      failed.</p>
    <p><br>
    </p>
    <p><br>
    </p>
    <div>On 1/09/21 1:41 am, Karera Tony wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Dear
          Ammad,</div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
        </div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Sorry to
          bother you again but I have failed to get the right command to
          use to check.</div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
        </div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Every
          Kubectl command I run on either the master or worker.</div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">The
          connection to the server localhost:8080 was refused - did you
          specify the right host or port?<br>
        </div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I
          get the error below.</div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
        </div>
        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
        </div>
        <div>
          <div dir="ltr">
            <div dir="ltr">
              <div>
                <div dir="ltr">
                  <div>
                    <div><font face="arial, sans-serif">Regards</font></div>
                    <div><font face="arial, sans-serif"><br>
                      </font></div>
                    <div dir="ltr"><font face="arial, sans-serif">Tony
                        Karera</font></div>
                    <div dir="ltr"><br>
                    </div>
                    <div><br>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
        <br>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Fri, Aug 27, 2021 at 9:15
          AM Ammad Syed <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>> wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div dir="auto">Your hyperkube services are not started.</div>
          <div dir="auto"><br>
          </div>
          <div dir="auto">You need to check hyperkube services.</div>
          <div dir="auto"><br>
          </div>
          <div dir="auto">Ammad</div>
          <div><br>
            <div class="gmail_quote">
              <div dir="ltr" class="gmail_attr">On Fri, Aug 27, 2021 at
                10:35 AM Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                wrote:<br>
              </div>
              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                <div dir="ltr">
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Dear
                    Ammad,</div>
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                  </div>
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Below is
                    the output of podman ps</div>
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                  </div>
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">CONTAINER
                    ID  IMAGE                                          
                                     COMMAND               CREATED      
                    STATUS           PORTS       NAMES<br>
                    319fbebc2f50  <a href="http://docker.io/openstackmagnum/heat-container-agent:wallaby-stable-1" target="_blank">docker.io/openstackmagnum/heat-container-agent:wallaby-stable-1</a>
                     /usr/bin/start-he...  23 hours ago  Up 23 hours ago
                                 heat-container-agent<br>
                    [root@k8s-cluster-2-4faiphvzsmzu-master-0 core]#<br>
                  </div>
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                  </div>
                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                  </div>
                  <div>
                    <div dir="ltr">
                      <div dir="ltr">
                        <div>
                          <div dir="ltr">
                            <div>
                              <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial, sans-serif">Regards</font></div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
                <div dir="ltr">
                  <div>
                    <div dir="ltr">
                      <div dir="ltr">
                        <div>
                          <div dir="ltr">
                            <div>
                              <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial, sans-serif"><br>
                                </font></div>
                              <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial, sans-serif">Tony Karera</font></div>
                              <div dir="ltr"><br>
                              </div>
                              <div><br>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                  <br>
                </div>
                <br>
                <div class="gmail_quote">
                  <div dir="ltr" class="gmail_attr">On Thu, Aug 26, 2021
                    at 9:54 AM Ammad Syed <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                    wrote:<br>
                  </div>
                  <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                    <div dir="ltr">The output in
logfile 29a37aff-f1f6-46b3-8541-887491c6cfe8-k8s-cluster3-dcu52bgzpbuu-kube_masters-ocfrn2ikpcgd-0-32tmkqgdq7wl-master_config-gihyfv3wlyzd
                      is incomplete.
                      <div><br>
                      </div>
                      <div>There should be the installation and
                        configuration of many other things that are
                        missing. Also it looks that hyperkube is not
                        installed.</div>
                      <div><br>
                      </div>
                      <div>Can you check the response of "podman ps"
                        command on master nodes. </div>
                      <div><br>
                      </div>
                      <div>Ammad </div>
                    </div>
                    <br>
                    <div class="gmail_quote">
                      <div dir="ltr" class="gmail_attr">On Thu, Aug 26,
                        2021 at 11:30 AM Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                        wrote:<br>
                      </div>
                      <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                        <div dir="ltr">
                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Here
                            is the beginning of the Log</div>
                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                          </div>
                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Starting
                            to run kube-apiserver-to-kubelet-role<br>
                            + echo 'Waiting for Kubernetes API...'<br>
                            Waiting for Kubernetes API...<br>
                            ++ kubectl get --raw=/healthz<br>
                            The connection to the server localhost:8080
                            was refused - did you specify the right host
                            or port?<br>
                            + '[' ok = '' ']'<br>
                            <br>
                            <br>
                          </div>
                          <div>
                            <div dir="ltr">
                              <div dir="ltr">
                                <div>
                                  <div dir="ltr">
                                    <div>
                                      <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial, sans-serif">Regards</font></div>
                                      <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial, sans-serif"><br>
                                        </font></div>
                                      <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial, sans-serif">Tony
                                          Karera</font></div>
                                      <div dir="ltr"><br>
                                      </div>
                                      <div><br>
                                      </div>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                          <br>
                        </div>
                        <br>
                        <div class="gmail_quote">
                          <div dir="ltr" class="gmail_attr">On Thu, Aug
                            26, 2021 at 7:53 AM Bharat Kunwar <<a href="mailto:bharat@stackhpc.com" target="_blank">bharat@stackhpc.com</a>>
                            wrote:<br>
                          </div>
                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                            <div dir="auto">I assume these are from the
                              master nodes? Can you share the logs
                              shortly after creation rather than when it
                              times out? I think there is some missing
                              logs from the top.<br>
                              <br>
                              <div dir="ltr">Sent from my iPhone</div>
                              <div dir="ltr"><br>
                                <blockquote type="cite">On 26 Aug 2021,
                                  at 06:14, Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                  wrote:<br>
                                  <br>
                                </blockquote>
                              </div>
                              <blockquote type="cite">
                                <div dir="ltr">
                                  <div dir="ltr">
                                    <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Hello
                                      Guys,</div>
                                    <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                    </div>
                                    <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Attached
                                      are the two logs from the
                                      /var/log/heat-config/heat-config-script
                                      directory<br>
                                    </div>
                                    <div>
                                      <div dir="ltr">
                                        <div dir="ltr">
                                          <div>
                                            <div dir="ltr">
                                              <div>
                                                <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                    sans-serif">Regards</font></div>
                                                <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                    sans-serif"><br>
                                                  </font></div>
                                                <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                    sans-serif">Tony
                                                    Karera</font></div>
                                                <div dir="ltr"><br>
                                                </div>
                                                <div><br>
                                                </div>
                                              </div>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                    <br>
                                  </div>
                                  <br>
                                  <div class="gmail_quote">
                                    <div dir="ltr" class="gmail_attr">On
                                      Thu, Aug 26, 2021 at 5:59 AM
                                      Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                      wrote:<br>
                                    </div>
                                    <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                      <div dir="ltr">
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Dear
                                          Sir,</div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                        </div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">You
                                          are right.</div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                        </div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I
                                          am getting this error <br>
                                        </div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                        </div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">kubectl
                                          get --raw=/healthz<br>
                                          The connection to the server
                                          localhost:8080 was refused -
                                          did you specify the right host
                                          or port?</div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                        </div>
                                        <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                        </div>
                                        <div>
                                          <div dir="ltr">
                                            <div dir="ltr">
                                              <div>
                                                <div dir="ltr">
                                                  <div>
                                                    <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                        sans-serif">Regards</font></div>
                                                    <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                        sans-serif"><br>
                                                      </font></div>
                                                    <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                        sans-serif">Tony
                                                        Karera</font></div>
                                                    <div dir="ltr"><br>
                                                    </div>
                                                    <div><br>
                                                    </div>
                                                  </div>
                                                </div>
                                              </div>
                                            </div>
                                          </div>
                                        </div>
                                        <br>
                                      </div>
                                      <br>
                                      <div class="gmail_quote">
                                        <div dir="ltr" class="gmail_attr">On Wed, Aug
                                          25, 2021 at 10:55 PM Bharat
                                          Kunwar <<a href="mailto:bharat@stackhpc.com" target="_blank">bharat@stackhpc.com</a>>
                                          wrote:<br>
                                        </div>
                                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                          <div dir="auto">I’d check the
                                            logs under
                                            /var/log/heat-config.<br>
                                            <br>
                                            <div dir="ltr">Sent from my
                                              iPhone</div>
                                            <div dir="ltr"><br>
                                              <blockquote type="cite">On
                                                25 Aug 2021, at 19:39,
                                                Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                wrote:<br>
                                                <br>
                                              </blockquote>
                                            </div>
                                            <blockquote type="cite">
                                              <div dir="ltr">
                                                <div dir="ltr">
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">DeaR Ammad,</div>
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                  </div>
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I was able to make the
                                                    communication work
                                                    and the Worker nodes
                                                    were created as well
                                                    but the cluster
                                                    failed.</div>
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                  </div>
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I logged in to the master
                                                    node and there was
                                                    no error but below
                                                    are the error when I
                                                    run systemctl status
                                                    heat-container-agent
                                                    on the worker noed.</div>
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                  </div>
                                                  <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Aug 25 17:52:24
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:52:55
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:53:26
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:53:57
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:54:28
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:54:59
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:55:29
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:56:00
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:56:31
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                    Aug 25 17:57:02
                                                    cluster1-fmkpva3nozf7-node-0
                                                    podman[2268]:
                                                    /var/lib/os-collect-config/local-data
                                                    not found. Skipping<br>
                                                  </div>
                                                  <div>
                                                    <div dir="ltr">
                                                      <div dir="ltr">
                                                        <div>
                                                          <div dir="ltr">
                                                          <div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Regards</font></div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif"><br>
                                                          </font></div>
                                                          <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Tony
                                                          Karera</font></div>
                                                          <div dir="ltr"><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </div>
                                                    </div>
                                                  </div>
                                                  <br>
                                                </div>
                                                <br>
                                                <div class="gmail_quote">
                                                  <div dir="ltr" class="gmail_attr">On
                                                    Wed, Aug 25, 2021 at
                                                    10:38 AM Ammad Syed
                                                    <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                                                    wrote:<br>
                                                  </div>
                                                  <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                    <div dir="ltr">Yes,
                                                      keystone, Heat,
                                                      Barbicane and
                                                      magnum public
                                                      endpoints must be
                                                      reachable from
                                                      master and worker
                                                      nodes.
                                                      <div><br>
                                                      </div>
                                                      <div>You can use
                                                        below guide for
                                                        the reference as
                                                        well.</div>
                                                      <div><br>
                                                      </div>
                                                      <div><a href="https://www.server-world.info/en/note?os=Ubuntu_20.04&p=openstack_victoria4&f=11" target="_blank">https://www.server-world.info/en/note?os=Ubuntu_20.04&p=openstack_victoria4&f=11</a><br>
                                                        <div><br>
                                                        </div>
                                                        <div>Ammad</div>
                                                      </div>
                                                    </div>
                                                    <br>
                                                    <div class="gmail_quote">
                                                      <div dir="ltr" class="gmail_attr">On
                                                        Wed, Aug 25,
                                                        2021 at 12:08 PM
                                                        Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                        wrote:<br>
                                                      </div>
                                                      <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                        <div dir="ltr">
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Hello Ammad,</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I have deployed using the
                                                          given image
                                                          but I think
                                                          there is an
                                                          issue with
                                                          keystone as
                                                          per the screen
                                                          shot below
                                                          when I checked
                                                          the master
                                                          node's
                                                          heat-container-agent
                                                          status</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">
                                                          <div><image.png></div>
                                                          <br>
                                                          </div>
                                                          <div>
                                                          <div dir="ltr">
                                                          <div dir="ltr">
                                                          <div>
                                                          <div dir="ltr">
                                                          <div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Regards</font></div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif"><br>
                                                          </font></div>
                                                          <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Tony
                                                          Karera</font></div>
                                                          <div dir="ltr"><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <br>
                                                        </div>
                                                        <br>
                                                        <div class="gmail_quote">
                                                          <div dir="ltr" class="gmail_attr">On Wed, Aug 25, 2021 at 8:28 AM Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Hello Ammad,</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I actually first used that
                                                          one and it was
                                                          also getting
                                                          stuck.</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">I will try this one again
                                                          and update you
                                                          with the Logs
                                                          though.</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div>
                                                          <div dir="ltr">
                                                          <div dir="ltr">
                                                          <div>
                                                          <div dir="ltr">
                                                          <div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Regards</font></div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif"><br>
                                                          </font></div>
                                                          <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Tony
                                                          Karera</font></div>
                                                          <div dir="ltr"><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          <br>
                                                          <div class="gmail_quote">
                                                          <div dir="ltr" class="gmail_attr">On Wed, Aug 25, 2021 at 8:25 AM Ammad Syed <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">It
                                                          seems from the
                                                          logs that you
                                                          are using
                                                          fedora atomic.
                                                          Can you try
                                                          with FCOS 32
                                                          image.
                                                          <div><br>
                                                          </div>
                                                          <div><a href="https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/32.20201004.3.0/x86_64/fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2.xz" target="_blank">https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/32.20201004.3.0/x86_64/fedora-coreos-32.20201004.3.0-openstack.x86_64.qcow2.xz</a><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <div>Ammad</div>
                                                          </div>
                                                          <br>
                                                          <div class="gmail_quote">
                                                          <div dir="ltr" class="gmail_attr">On Wed, Aug 25, 2021 at 11:20 AM Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Hello Sir,</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Attached is the Log file</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div>
                                                          <div dir="ltr">
                                                          <div dir="ltr">
                                                          <div>
                                                          <div dir="ltr">
                                                          <div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Regards</font></div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif"><br>
                                                          </font></div>
                                                          <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Tony
                                                          Karera</font></div>
                                                          <div dir="ltr"><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          <br>
                                                          <div class="gmail_quote">
                                                          <div dir="ltr" class="gmail_attr">On Wed, Aug 25, 2021 at 7:31 AM Ammad Syed <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">Hi Karera,
                                                          <div><br>
                                                          </div>
                                                          <div>Can you
                                                          share us the
                                                          full log file.</div>
                                                          <div><br>
                                                          </div>
                                                          <div>Ammad</div>
                                                          </div>
                                                          <br>
                                                          <div class="gmail_quote">
                                                          <div dir="ltr" class="gmail_attr">On Wed, Aug 25, 2021 at 9:42 AM Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                          <div dir="ltr">
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Hello Guys,</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">Thanks a lot for the help
                                                          but
                                                          unfortunately
                                                          I dont see
                                                          much
                                                          information in
                                                          the log file
                                                          indicating a
                                                          failure apart
                                                          from the log
                                                          that keeps
                                                          appearing.</div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)"><br>
                                                          </div>
                                                          <div class="gmail_default" style="font-size:small;color:rgb(68,68,68)">
                                                          <div><image.png></div>
                                                          <br>
                                                          <br>
                                                          </div>
                                                          <div>
                                                          <div dir="ltr">
                                                          <div dir="ltr">
                                                          <div>
                                                          <div dir="ltr">
                                                          <div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Regards</font></div>
                                                          <div><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif"><br>
                                                          </font></div>
                                                          <div dir="ltr"><font style="font-family:arial,sans-serif;color:rgb(0,0,0)" face="arial,
                                                          sans-serif">Tony
                                                          Karera</font></div>
                                                          <div dir="ltr"><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <br>
                                                          </div>
                                                          <br>
                                                          <div class="gmail_quote">
                                                          <div dir="ltr" class="gmail_attr">On Tue, Aug 24, 2021 at 8:12 PM Mohammed Naser <<a href="mailto:mnaser@vexxhost.com" target="_blank">mnaser@vexxhost.com</a>>
                                                          wrote:<br>
                                                          </div>
                                                          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Also
                                                          check out
                                                          /var/log/cloud-init.log
                                                          :)<br>
                                                          <br>
                                                          On Tue, Aug
                                                          24, 2021 at
                                                          1:39 PM Ammad
                                                          Syed <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                                                          wrote:<br>
                                                          ><br>
                                                          > Then
                                                          check
                                                          journalctl -xe
                                                          or status of
                                                          heat agent
                                                          service
                                                          status.<br>
                                                          ><br>
                                                          ><br>
                                                          > Ammad<br>
                                                          > On Tue,
                                                          Aug 24, 2021
                                                          at 10:36 PM
                                                          Karera Tony
                                                          <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
                                                          >><br>
                                                          >> Hello
                                                          Ammad,<br>
                                                          >><br>
                                                          >> There
                                                          is no
                                                          directory or
                                                          log relevant
                                                          to heat in the
                                                          /var/log
                                                          directory<br>
                                                          >><br>
                                                          >>
                                                          Regards<br>
                                                          >><br>
                                                          >> Tony
                                                          Karera<br>
                                                          >><br>
                                                          >><br>
                                                          >><br>
                                                          >><br>
                                                          >> On
                                                          Tue, Aug 24,
                                                          2021 at 12:43
                                                          PM Ammad Syed
                                                          <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                                                          wrote:<br>
                                                          >>><br>
                                                          >>>
                                                          Hi Karera,<br>
                                                          >>><br>
                                                          >>>
                                                          Login to
                                                          master node
                                                          and check the
                                                          logs of heat
                                                          agent in var
                                                          log. There
                                                          must be
                                                          something the
                                                          cluster is
                                                          stucking
                                                          somewhere in
                                                          creating.<br>
                                                          >>><br>
                                                          >>>
                                                          Ammad<br>
                                                          >>><br>
                                                          >>>
                                                          On Tue, Aug
                                                          24, 2021 at
                                                          3:41 PM Karera
                                                          Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
>>>><br>
>>>> Hello Ammad,<br>
>>>><br>
>>>> I had done as explained and it worked upto a certain
                                                          point. The
                                                          master node
                                                          was created
                                                          but the
                                                          cluster
                                                          remained in
                                                          Creation in
                                                          progress for
                                                          over an hour
                                                          and failed
                                                          with error
                                                          below<br>
>>>><br>
>>>> Stack Faults<br>
>>>> as follows:<br>
>>>> default-master<br>
>>>> Timed out<br>
>>>> default-worker<br>
>>>> Timed out<br>
>>>><br>
>>>><br>
>>>> Regards<br>
>>>><br>
>>>> Tony Karera<br>
>>>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> On Tue, Aug 24, 2021 at 9:25 AM Ammad Syed <<a href="mailto:syedammad83@gmail.com" target="_blank">syedammad83@gmail.com</a>>
                                                          wrote:<br>
>>>>><br>
>>>>> Hi Tony,<br>
>>>>><br>
>>>>> You can try by creating your private vxlan network
                                                          prior to
                                                          deployment of
                                                          cluster and
                                                          explicitly
                                                          create your
                                                          cluster in
                                                          vxlan network.<br>
>>>>><br>
>>>>> --fixed-network private --fixed-subnet
                                                          private-subnet<br>
>>>>><br>
>>>>> You can specify above while creating a cluster.<br>
>>>>><br>
>>>>> Ammad<br>
>>>>><br>
>>>>> On Tue, Aug 24, 2021 at 11:59 AM Karera Tony <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
>>>>>><br>
>>>>>> Hello MOhamed,<br>
>>>>>><br>
>>>>>> I think the Kubernetes cluster is ok but it
                                                          when I deploy
                                                          it, It creates
                                                          a fixed
                                                          network using
                                                          vlan which I
                                                          am not using
                                                          for internal
                                                          networks.<br>
>>>>>><br>
>>>>>> When I create a a vxlan Network and use it in
                                                          the cluster
                                                          creation, It
                                                          fails. Is
                                                          there a trick
                                                          around this ?<br>
>>>>>> Regards<br>
>>>>>><br>
>>>>>> Tony Karera<br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> On Fri, Aug 20, 2021 at 9:00 AM feilong <<a href="mailto:feilong@catalyst.net.nz" target="_blank">feilong@catalyst.net.nz</a>>
                                                          wrote:<br>
>>>>>>><br>
>>>>>>> Oooh, are you using Swarm? I don't think
                                                          that driver is
                                                          well
                                                          maintained. I
                                                          didn't see any
                                                          interest in
                                                          the last 4
                                                          years since I
                                                          involved in
                                                          the Magnum
                                                          project. If
                                                          there is no
                                                          specific
                                                          reason, I
                                                          would suggest
                                                          go for k8s.<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> On 20/08/21 5:08 pm, Mohammed Naser wrote:<br>
>>>>>>><br>
>>>>>>> Please keep replies on list so others can
                                                          help too.<br>
>>>>>>><br>
>>>>>>> I don’t know how well tested the Swarm
                                                          driver is at
                                                          this point. I
                                                          believe most
                                                          Magnum users
                                                          are using it
                                                          for Kubernetes
                                                          only.<br>
>>>>>>><br>
>>>>>>> On Fri, Aug 20, 2021 at 1:05 AM Karera Tony
                                                          <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
>>>>>>>><br>
>>>>>>>> Hello Naser,<br>
>>>>>>>><br>
>>>>>>>> Please check below.<br>
>>>>>>>><br>
>>>>>>>> openstack coe cluster template create
                                                          swarm-cluster-template1
                                                          \<br>
>>>>>>>>                      --image
                                                          fedora-atomic-latest
                                                          \<br>
>>>>>>>>                      --external-network
                                                          External_1700\<br>
>>>>>>>>                      --dns-nameserver
                                                          8.8.8.8 \<br>
>>>>>>>>                      --master-flavor
                                                          m1.small \<br>
>>>>>>>>                      --flavor m1.small
                                                          \<br>
>>>>>>>>                      --coe swarm<br>
>>>>>>>> openstack coe cluster create
                                                          swarm-cluster
                                                          \<br>
>>>>>>>>                       
                                                           --cluster-template
swarm-cluster-template \<br>
>>>>>>>>                         --master-count
                                                          1 \<br>
>>>>>>>>                         --node-count 2
                                                          \<br>
>>>>>>>>                         --keypair
                                                          Newkey<br>
>>>>>>>><br>
>>>>>>>> Regards<br>
>>>>>>>><br>
>>>>>>>> Tony Karera<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> On Fri, Aug 20, 2021 at 7:03 AM
                                                          Mohammed Naser
                                                          <<a href="mailto:mnaser@vexxhost.com" target="_blank">mnaser@vexxhost.com</a>>
                                                          wrote:<br>
>>>>>>>>><br>
>>>>>>>>> What does your cluster template and
                                                          cluster create
                                                          command look
                                                          like?<br>
>>>>>>>>><br>
>>>>>>>>> On Fri, Aug 20, 2021 at 12:59 AM
                                                          Karera Tony
                                                          <<a href="mailto:tonykarera@gmail.com" target="_blank">tonykarera@gmail.com</a>>
                                                          wrote:<br>
>>>>>>>>>><br>
>>>>>>>>>> Hello Wang,<br>
>>>>>>>>>><br>
>>>>>>>>>> Thanks for the feedback.<br>
>>>>>>>>>><br>
>>>>>>>>>> Unfortunately Octavia is not
                                                          deployed in my
                                                          environment
                                                          (at least not
                                                          yet) and LB is
                                                          not enabled on
                                                          either the
                                                          cluster
                                                          template or
                                                          the cluster
                                                          itself.<br>
>>>>>>>>>><br>
>>>>>>>>>> Regards<br>
>>>>>>>>>><br>
>>>>>>>>>> Tony Karera<br>
>>>>>>>>>><br>
>>>>>>>>>><br>
>>>>>>>>>><br>
>>>>>>>>>><br>
>>>>>>>>>> On Fri, Aug 20, 2021 at 6:52 AM
                                                          feilong <<a href="mailto:feilong@catalyst.net.nz" target="_blank">feilong@catalyst.net.nz</a>>
                                                          wrote:<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Hi Karera,<br>
>>>>>>>>>>><br>
>>>>>>>>>>> It's probably a bug. If you
                                                          do have
                                                          Octavia
                                                          deployed, can
                                                          you try to not
                                                          disable the LB
                                                          and see how it
                                                          goes?<br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>> On 20/08/21 4:18 pm, Karera
                                                          Tony wrote:<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Hello Team,<br>
>>>>>>>>>>><br>
>>>>>>>>>>> I deployed Openstack Wallby
                                                          on Ubuntu20
                                                          and enabled
                                                          Magum, however
                                                          when I create
                                                          a cluster I
                                                          get the error
                                                          below.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Status Reason<br>
>>>>>>>>>>> ERROR: Property error: :
                                                          resources.api_lb.properties:
                                                          : Property
                                                          allowed_cidrs
                                                          not assigned<br>
>>>>>>>>>>> Can someone advise on where
                                                          I could be
                                                          wrong. Btw, I
                                                          disabled load
                                                          balancer while
                                                          creating the
                                                          cluster.<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Regards<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Tony Karera<br>
>>>>>>>>>>><br>
>>>>>>>>>>><br>
>>>>>>>>>>> --<br>
>>>>>>>>>>> Cheers & Best regards,<br>
>>>>>>>>>>>
------------------------------------------------------------------------------<br>
>>>>>>>>>>> Feilong Wang (王飞龙) (he/him)<br>
>>>>>>>>>>> Head of Research &
                                                          Development<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Catalyst Cloud<br>
>>>>>>>>>>> Aotearoa's own<br>
>>>>>>>>>>><br>
>>>>>>>>>>> Mob: +64 21 0832 6348 | <a href="http://www.catalystcloud.nz" rel="noreferrer" target="_blank">www.catalystcloud.nz</a><br>
>>>>>>>>>>> Level 6, <a href="https://www.google.com/maps/search/150+Willis+Street,+Wellington+6011,+New+Zealand?entry=gmail&source=g" target="_blank">150 Willis Street, Wellington
                                                          6011, New
                                                          Zealand</a><br>
>>>>>>>>>>><br>
>>>>>>>>>>> CONFIDENTIALITY NOTICE:
                                                          This email is
                                                          intended for
                                                          the named
                                                          recipients
                                                          only.<br>
>>>>>>>>>>> It may contain privileged,
                                                          confidential
                                                          or copyright
                                                          information.
                                                          If you are<br>
>>>>>>>>>>> not the named recipient,
                                                          any use,
                                                          reliance upon,
                                                          disclosure or
                                                          copying of
                                                          this<br>
>>>>>>>>>>> email or its attachments is
                                                          unauthorised.
                                                          If you have
                                                          received this
                                                          email in<br>
>>>>>>>>>>> error, please reply via
                                                          email or call
                                                          +64 21 0832
                                                          6348.<br>
>>>>>>>>>>>
------------------------------------------------------------------------------<br>
>>>>>>>>><br>
>>>>>>>>> --<br>
>>>>>>>>> Mohammed Naser<br>
>>>>>>>>> VEXXHOST, Inc.<br>
>>>>>>><br>
>>>>>>> --<br>
>>>>>>> Mohammed Naser<br>
>>>>>>> VEXXHOST, Inc.<br>
>>>>>>><br>
>>>>>>> --<br>
>>>>>>> Cheers & Best regards,<br>
>>>>>>>
------------------------------------------------------------------------------<br>
>>>>>>> Feilong Wang (王飞龙) (he/him)<br>
>>>>>>> Head of Research & Development<br>
>>>>>>><br>
>>>>>>> Catalyst Cloud<br>
>>>>>>> Aotearoa's own<br>
>>>>>>><br>
>>>>>>> Mob: +64 21 0832 6348 | <a href="http://www.catalystcloud.nz" rel="noreferrer" target="_blank">www.catalystcloud.nz</a><br>
>>>>>>> Level 6, <a href="https://www.google.com/maps/search/150+Willis+Street,+Wellington+6011,+New+Zealand?entry=gmail&source=g" target="_blank">150 Willis Street, Wellington
                                                          6011, New
                                                          Zealand</a><br>
>>>>>>><br>
>>>>>>> CONFIDENTIALITY NOTICE: This email is
                                                          intended for
                                                          the named
                                                          recipients
                                                          only.<br>
>>>>>>> It may contain privileged, confidential or
                                                          copyright
                                                          information.
                                                          If you are<br>
>>>>>>> not the named recipient, any use, reliance
                                                          upon,
                                                          disclosure or
                                                          copying of
                                                          this<br>
>>>>>>> email or its attachments is unauthorised.
                                                          If you have
                                                          received this
                                                          email in<br>
>>>>>>> error, please reply via email or call +64
                                                          21 0832 6348.<br>
>>>>>>>
------------------------------------------------------------------------------<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> --<br>
>>>>> Regards,<br>
>>>>><br>
>>>>><br>
>>>>> Syed Ammad Ali<br>
                                                          >>><br>
                                                          >>><br>
                                                          >>><br>
                                                          >>>
                                                          --<br>
                                                          >>>
                                                          Regards,<br>
                                                          >>><br>
                                                          >>><br>
                                                          >>>
                                                          Syed Ammad Ali<br>
                                                          ><br>
                                                          > --<br>
                                                          > Regards,<br>
                                                          ><br>
                                                          ><br>
                                                          > Syed
                                                          Ammad Ali<br>
                                                          <br>
                                                          <br>
                                                          <br>
                                                          -- <br>
                                                          Mohammed Naser<br>
                                                          VEXXHOST, Inc.<br>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br clear="all">
                                                          <div><br>
                                                          </div>
                                                          -- <br>
                                                          <div dir="ltr">Regards,
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <div>Syed
                                                          Ammad Ali</div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <br clear="all">
                                                          <div><br>
                                                          </div>
                                                          -- <br>
                                                          <div dir="ltr">Regards,
                                                          <div><br>
                                                          </div>
                                                          <div><br>
                                                          </div>
                                                          <div>Syed
                                                          Ammad Ali</div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          </blockquote>
                                                        </div>
                                                      </blockquote>
                                                    </div>
                                                    <br clear="all">
                                                    <div><br>
                                                    </div>
                                                    -- <br>
                                                    <div dir="ltr">Regards,
                                                      <div><br>
                                                      </div>
                                                      <div><br>
                                                      </div>
                                                      <div>Syed Ammad
                                                        Ali</div>
                                                    </div>
                                                  </blockquote>
                                                </div>
                                              </div>
                                            </blockquote>
                                          </div>
                                        </blockquote>
                                      </div>
                                    </blockquote>
                                  </div>
                                  <div><29a37aff-f1f6-46b3-8541-887491c6cfe8-k8s-cluster3-dcu52bgzpbuu-kube_masters-ocfrn2ikpcgd-0-32tmkqgdq7wl-master_config-gihyfv3wlyzd.log></div>
                                  <div><6fca39b1-8eda-4786-8424-e5b04434cce7-k8s-cluster3-dcu52bgzpbuu-kube_cluster_config-aht4it6p7wfk.log></div>
                                </div>
                              </blockquote>
                            </div>
                          </blockquote>
                        </div>
                      </blockquote>
                    </div>
                    <br clear="all">
                    <div><br>
                    </div>
                    -- <br>
                    <div dir="ltr">Regards,
                      <div><br>
                      </div>
                      <div><br>
                      </div>
                      <div>Syed Ammad Ali</div>
                    </div>
                  </blockquote>
                </div>
              </blockquote>
            </div>
          </div>
          -- <br>
          <div dir="ltr">Regards,
            <div><br>
            </div>
            <div><br>
            </div>
            <div>Syed Ammad Ali</div>
          </div>
        </blockquote>
      </div>
    </blockquote>
    <pre cols="72">-- 
Cheers & Best regards,
------------------------------------------------------------------------------
Feilong Wang (王飞龙) (he/him)
Head of Research & Development

Catalyst Cloud
Aotearoa's own

Mob: +64 21 0832 6348 | <a href="http://www.catalystcloud.nz" target="_blank">www.catalystcloud.nz</a>
Level 6, 150 Willis Street, Wellington 6011, New Zealand

CONFIDENTIALITY NOTICE: This email is intended for the named recipients only.
It may contain privileged, confidential or copyright information. If you are 
not the named recipient, any use, reliance upon, disclosure or copying of this 
email or its attachments is unauthorised. If you have received this email in 
error, please reply via email or call +64 21 0832 6348.
------------------------------------------------------------------------------</pre>
  </div>

</blockquote></div>