Open Stack

Hi Arne,

We've had this EXACT same issue.

I don't know of a way to force an update as you are basically pulling the
rug out from under a running instance. I don't know if it is
possible/feasible to update the virsh xml in place and then migrate to get
it to actually use that data. (I think we tried that to no avail.)
dumpxml=>massage cephmons=>import xml

If you find a way, let me know, and that's part of the reason I'm replying
so that I stay on this thread. NOTE: We did this on icehouse. Haven't tried
since upgrading to Juno but I don't note any change therein that would
mitigate this. So I'm guessing Liberty/post-Liberty for a real fix.



On Wed, May 6, 2015 at 12:57 PM, Arne Wiebalck <arne.wiebalck at cern.ch>
wrote:

> Hi,
>
> As we swapped a fraction of our Ceph mon servers between the
> pre-production and production cluster
> -- something we considered to be transparent as the Ceph config points to
> the mon alias--, we ended
> up in a situation where VMs with volumes attached were not able to boot
> (with a probability that matched
> the fraction of the servers moved between the Ceph instances).
>
> We found that the reason for this is the connection_info in
> block_device_mapping which contains the
> IP adresses of the mon servers as extracted by the rbd driver in
> initialize_connection() at the moment
> when the connection is established. From what we see, however, this
> information is not updated as long
> as the connection exists, and will hence be re-applied without checking
> even when the XML is recreated.
>
> The idea to extract the mon servers by IP from the mon map was probably to
> get all mon servers (rather
> than only one from a load-balancer or an alias), but while our current
> scenario may be special, we will face
> a similar problem the day the Ceph mons need to be replaced. And that
> makes it a more general issue.
>
> For our current problem:
> Is there a user-transparent way to force an update of that connection
> information? (Apart from fiddling
> with the database entries, of course.)
>
> For the general issue:
> Would it be possible to simply use the information from the ceph.conf file
> directly (an alias in our case)
> throughout the whole stack to avoid hard-coding IPs that will be obsolete
> one day?
>
> Thanks!
>  Arne
>
> --
> Arne Wiebalck
> CERN IT
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150506/ecc0a46d/attachment.html>