Open Stack

At around Sep 21 02:30UTC mirror01.bhs1.ovh.openstack.org became
uncontactable and jobs in the region started to fail.

The server was in an ACTIVE state but uncontactable.  I attempted to
get a console but either a log or url request returned 500 (request
id's below if it helps).

 ... console url show ...
The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-5da4cba2-efe8-4dfb-a8a7-faf490075c89)
 ...  console log show ...
The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-80beb593-b565-42eb-8a97-b2a208e3d865)

I could not figure out how to log into the web console with our
credentials.

I attempted to hard-reboot it, and it currently appears stuck in
HARD_REBOOT.  Thus I have placed nodepool.o.o in the emergency file
and set max-servers for the ovh-bhs1 region to 0

I have left it at this, as hopefully it will be beneficial for both
OVH and us to diagnose the issue since the host was definitely not
expected to disappear.  After this we can restore or rebuild it as
required.

Thanks,

-i