[Openstack-operators] Ceph Implementations

Erik McCormick emccormick at cirrusseven.com
Fri Nov 7 21:33:03 UTC 2014


On Fri, Nov 7, 2014 at 11:43 AM, MailingLists - EWS <
mailinglists at expresswebsystems.com> wrote:

> Since there seems to be a fair amount of people on this list running Ceph
> with Openstack, I wanted to ask what configuration most people are using
> for their Ceph/Openstack configuration.
>
>
>
> We tried installing Ceph on CentOS 6 (storage nodes and nova nodes) but
> discovered that the kernel that is included in RDO (for nova) doesn’t
> include rbd support in qemu-img. No problem we thought, we will just use
> the kernel-lt from elrepo which does include support for rbd. However after
> installing those kernels we discovered that our GRE based networking didn’t
> work. After some additional troubleshooting we determined that the
> kernel-lt didn’t include the backported ovs vport GRE capability that the
> RDO kernel has (it apparently was added in 3.11 but kernel-lt is 3.10).
>
>
You shouldn't need a special kernel unless you're going to do things like
try to mount an rbd image directly to one of your nodes. I'm running my
Icehouse cluster with the stock 2.6 kernel. Same for the ceph nodes if you
run them separately. What you do need are the qemu-kvm and qemu-img
packages from the ceph-extras repo on your compute nodes as the stock
Redhat ones do not have RBD support compiled in. If you're going to run
Juno though, you'll need to go to CentOS 7 as i don't believe there are RDO
packages for 6.5. That eliminates the need for the custom qemu packages.


>
> So we have to decide how we want to implement our cluster and cloud.
>
>
>
> Are you implementing Openstack on Debian/Ubuntu to get around these sorts
> of issues?
>
> Are you running newer mainline kernels (kernel-ml) with CentOS 6?
>
> Or are you running vlan based tenant isolation for your tenant networks?
>
> Are you running custom compiled kernels?
>
> Is there something else I am missing?
>
>
I'm using GRE on Icehouse and VXLAN on Juno. Both work fine. Be aware there
is at least one bug I've run into with Juno and Glance with RBD that will
cause slow performance. It's easily remedied though either by tweaking the
code or setting your chunk size to 8192 instead of 8.

https://bugs.launchpad.net/glance/+bug/1390386


>
>
> Any input from your experience is greatly appreciated.
>
>
>
> Warm regards,
>
>
>
> Tom Walsh
>
> Express Web Systems
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
Cheers,
Erik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20141107/5648e887/attachment.html>


More information about the OpenStack-operators mailing list