Open Stack

On 27 February 2014 15:55, Derek Higgins <derekh at redhat.com> wrote:
> On 25/02/14 00:08, Robert Collins wrote:
>> Today we had an outage of the tripleo test cloud :(.
>>
>> tl;dr:
>>  - we were down for 14 hours
>>  - we don't know the fundamental cause
>>  - infra were not inconvenienced - yaaay
>>  - its all ok now.
> Looks like we've hit the same problem again tonight, I've
> o rebooted the server
> o fixed up the hostname
> o restarted nova and neutron services on the controller
>
> VM's still not getting IP's, I'm not seeing dhcp requests from them
> coming into dnsmasq, spent some time trying to figure out the problem,
> no luck, I'll pick up in this in a few hours if nobody else has before then.

https://bugs.launchpad.net/tripleo/+bug/1284354 - Neutron has decided
to toggle short and long name again:
| 0d00f56e-ca18-48c9-9552-4a5aad8f507d | L3 agent           |
ci-overcloud-notcompute0-gxezgcvv4v2q.novalocal | xxx   | True
  |
| 2c68c810-7cc5-4547-a106-ddb698ba8245 | DHCP agent         |
ci-overcloud-notcompute0-gxezgcvv4v2q           | :-)   | True
  |
| 4887d74a-6394-4144-a994-104e092e956e | DHCP agent         |
ci-overcloud-notcompute0-gxezgcvv4v2q.novalocal | xxx   | True
  |
| 8d405cc8-c5c8-429a-ae87-a67296d4249e | Open vSwitch agent |
ci-overcloud-notcompute0-gxezgcvv4v2q           | :-)   | True
  |
| b269d9ea-2435-4cd4-a227-42ee93ab9b62 | Metadata agent     |
ci-overcloud-notcompute0-gxezgcvv4v2q.novalocal | xxx   | True
  |
| c7b80d9b-f3e8-4f15-9437-6c5b78bb6af2 | Open vSwitch agent |
ci-overcloud-notcompute0-gxezgcvv4v2q.novalocal | xxx   | True
  |
| dd9f7688-8135-4098-8eb2-23c9f9c6144c | Metadata agent     |
ci-overcloud-notcompute0-gxezgcvv4v2q           | :-)   | True
  |
| e37a5df6-47a1-45a0-bf69-e0d1f0fe9553 | L3 agent           |
ci-overcloud-notcompute0-gxezgcvv4v2q           | :-)   | True
  |

I've moved the l3 router from the dead agent to the live agent and can
ssh into the broker.

Checking new instance connectivity next.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud