
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hullo again.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 27 May 2022, at 20:11, Michael Johnson <<a href="mailto:johnsomor@gmail.com" class="">johnsomor@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi Tim,<br class=""><br class="">This should work fine. You will want a localrc/local.conf file to<br class="">configure devstack. I didn't see that mentioned in your steps.<br class="">See this section in the docs:<br class=""><a href="https://docs.openstack.org/devstack/latest/#create-a-local-conf" class="">https://docs.openstack.org/devstack/latest/#create-a-local-conf</a><br class=""></div></div></blockquote>Sorry, I missed that step in the description. local.conf is being created.<br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">The only caveat I would mention is the VM instances in Nova will run<br class="">super slow on virtualbox as it lacks the required "nested<br class="">virtualization" support and will run them all in software emulation.<br class=""></div></div></blockquote>I’m not worried about this atm. Once I’ve got something working, I can sort that out.<br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">To find the root cause of the issue in nova, I would look through the<br class="">devstack@n-cpu log file (journal -u devstack@n-cpui) and the<br class="">devstack@n-sch logs.<br class=""></div></div></blockquote>Neither of these logs contain errors. However, if I look for ‘ERROR’ in `sudo journalctl`, I do get ~400 keystone errors, where users, domains, roles and services cannot be found. I’m not sure whether this is expected or not. They occur at the beginning of the log and may be significant, or just an artefact of the `stack.sh` script. Here’s a few (filtered by `sudo journalctl |<span style="color: rgb(16, 16, 16); font-family: Monaco; background-color: rgb(255, 255, 255);" class="">grep keyston |grep exception|grep "Could not find"|grep -v "None admin"</span>) examples:</div><div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 15:57:39 node1.example.dd <a href="mailto:devstack@keystone.service" class="">devstack@keystone.service</a>[94392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service.</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 15:57:41 node1.example.dd <a href="mailto:devstack@keystone.service" class="">devstack@keystone.service</a>[94392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: glance.</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 15:57:41 node1.example.dd <a href="mailto:devstack@keystone.service" class="">devstack@keystone.service</a>[94392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: glance.</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(16, 16, 16); background-color: rgb(255, 255, 255);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 15:57:42 node1.example.dd <a href="mailto:devstack@keystone.service" class="">devstack@keystone.service</a>[94393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder.</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Subsequently, this is what I suspect is the smoking gun, as it precedes the timeout:</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(242, 242, 242); background-color: rgb(33, 32, 33);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 16:14:45 node1.example.dd nova-compute[133470]: ERROR os_brick.initiator.connectors.</span><span style="font-variant-ligatures: no-common-ligatures; color: #212021; background-color: #f2f2f2" class="">iscs</span><span style="font-variant-ligatures: no-common-ligatures" class="">i Command: </span><span style="font-variant-ligatures: no-common-ligatures; color: #212021; background-color: #f2f2f2" class="">iscs</span><span style="font-variant-ligatures: no-common-ligatures" class="">iadm -m discoverydb -o show -P 1</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(242, 242, 242); background-color: rgb(33, 32, 33);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 16:14:45 node1.example.dd nova-compute[133470]: ERROR os_brick.initiator.connectors.</span><span style="font-variant-ligatures: no-common-ligatures; color: #212021; background-color: #f2f2f2" class="">iscs</span><span style="font-variant-ligatures: no-common-ligatures" class="">i Exit code: 21</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(242, 242, 242); background-color: rgb(33, 32, 33);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">May 30 16:14:45 node1.example.dd nova-compute[133470]: ERROR os_brick.initiator.connectors.</span><span style="font-variant-ligatures: no-common-ligatures; color: #212021; background-color: #f2f2f2" class="">iscs</span><span style="font-variant-ligatures: no-common-ligatures" class="">i Stdout: 'SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nNo targets found.\nFIRMWARE:\nNo targets found.\n'</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div><div class="">I have no idea what that means. My guess is that there’s supposed to be a fake iscsi device and it’s not on a local network, but that’s really just a guess.</div><div class=""><br class=""></div></span></div><blockquote type="cite" class=""><div class=""><div class=""><br class="">Also, you might have a look at one of the nova test localrc file as an example:<br class=""><a href="https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_da7/843127/6/check/tempest-integrated-compute-centos-9-stream/da7bebc/controller/logs/local_conf.txt" class="">https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_da7/843127/6/check/tempest-integrated-compute-centos-9-stream/da7bebc/controller/logs/local_conf.txt</a><br class=""></div></div></blockquote><div><br class=""></div>Presumably, this example would need things like the various IP addresses changing to be useful. What would one then do with it?</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><br class="">Michael<br class=""><br class=""></div></div></blockquote>Tim</div><div><br class=""></div></body></html>