Open Stack

>> Also you can use peacemaker and other stuff to reach high availability
  >    Yes, but I'm guessing those need _another_ machine in front of the ones
>   I want to load balance. And if that goes down, EVERYTHING stops working.
>  Unless they are clustered, which require _even more_ machines!

If I had these constraints I would add a loadbalancer-config on the same machine that runs the OpenStack apis.
So add keepalived with haproxy on the 2 “OpenStack controllers”.
Keepalived for a VRRP based HA-IP and haproxy for load-balancing (with the nice addition you can do SSL offloading on haproxy).
This is somewhat similar to what what most vendors (Mirantis/Ubuntu/RedHat) are doing AFAIK.

If you use neutron with the L3 agent you will also have to think about HA of that.
That’s a different story though and I am not really up to speed on the latest and greatest there.

Cheers,
Robert