Open Stack

On Thu, Oct 1, 2015 at 12:58 AM, Shinobu Kinjo <skinjo at redhat.com> wrote:
> Is there any plan to merge those branches to master?
> Or is there anything needs to be done more?

As I said in the original email, this is unfinished code, and my
message was just to let people know this was underway so that the
patch didn't come as a complete surprise.

John

>
> Shinobu
>
> ----- Original Message -----
> From: "Ben Swartzlander" <ben at swartzlander.org>
> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
> Sent: Saturday, September 26, 2015 9:27:58 AM
> Subject: Re: [openstack-dev] [Manila] CephFS native driver
>
> On 09/24/2015 09:49 AM, John Spray wrote:
>> Hi all,
>>
>> I've recently started work on a CephFS driver for Manila.  The (early)
>> code is here:
>> https://github.com/openstack/manila/compare/master...jcsp:ceph
>
> Awesome! This is something that's been talking about for quite some time
> and I'm pleased to see progress on making it a reality.
>
>> It requires a special branch of ceph which is here:
>> https://github.com/ceph/ceph/compare/master...jcsp:wip-manila
>>
>> This isn't done yet (hence this email rather than a gerrit review),
>> but I wanted to give everyone a heads up that this work is going on,
>> and a brief status update.
>>
>> This is the 'native' driver in the sense that clients use the CephFS
>> client to access the share, rather than re-exporting it over NFS.  The
>> idea is that this driver will be useful for anyone who has such
>> clients, as well as acting as the basis for a later NFS-enabled
>> driver.
>
> This makes sense, but have you given thought to the optimal way to
> provide NFS semantics for those who prefer that? Obviously you can pair
> the existing Manila Generic driver with Cinder running on ceph, but I
> wonder how that wound compare to some kind of ganesha bridge that
> translates between NFS and cephfs. It that something you've looked into?
>
>> The export location returned by the driver gives the client the Ceph
>> mon IP addresses, the share path, and an authentication token.  This
>> authentication token is what permits the clients access (Ceph does not
>> do access control based on IP addresses).
>>
>> It's just capable of the minimal functionality of creating and
>> deleting shares so far, but I will shortly be looking into hooking up
>> snapshots/consistency groups, albeit for read-only snapshots only
>> (cephfs does not have writeable shapshots).  Currently deletion is
>> just a move into a 'trash' directory, the idea is to add something
>> later that cleans this up in the background: the downside to the
>> "shares are just directories" approach is that clearing them up has a
>> "rm -rf" cost!
>
> All snapshots are read-only... The question is whether you can take a
> snapshot and clone it into something that's writable. We're looking at
> allowing for different kinds of snapshot semantics in Manila for Mitaka.
> Even if there's no create-share-from-snapshot functionality a readable
> snapshot is still useful and something we'd like to enable.
>
> The deletion issue sounds like a common one, although if you don't have
> the thing that cleans them up in the background yet I hope someone is
> working on that.
>
>> A note on the implementation: cephfs recently got the ability (not yet
>> in master) to restrict client metadata access based on path, so this
>> driver is simply creating shares by creating directories within a
>> cluster-wide filesystem, and issuing credentials to clients that
>> restrict them to their own directory.  They then mount that subpath,
>> so that from the client's point of view it's like having their own
>> filesystem.  We also have a quota mechanism that I'll hook in later to
>> enforce the share size.
>
> So quotas aren't enforced yet? That seems like a serious issue for any
> operator except those that want to support "infinite" size shares. I
> hope that gets fixed soon as well.
>
>> Currently the security here requires clients (i.e. the ceph-fuse code
>> on client hosts, not the userspace applications) to be trusted, as
>> quotas are enforced on the client side.  The OSD access control
>> operates on a per-pool basis, and creating a separate pool for each
>> share is inefficient.  In the future it is expected that CephFS will
>> be extended to support file layouts that use RADOS namespaces, which
>> are cheap, such that we can issue a new namespace to each share and
>> enforce the separation between shares on the OSD side.
>
> I think it will be important to document all of these limitations. I
> wouldn't let them stop you from getting the driver done, but if I was a
> deployer I'd want to know about these details.
>
>> However, for many people the ultimate access control solution will be
>> to use a NFS gateway in front of their CephFS filesystem: it is
>> expected that an NFS-enabled cephfs driver will follow this native
>> driver in the not-too-distant future.
>
> Okay this answers part of my above question, but how to you expect the
> NFS gateway to work? Ganesha has been used successfully in the past.
>
>> This will be my first openstack contribution, so please bear with me
>> while I come up to speed with the submission process.  I'll also be in
>> Tokyo for the summit next month, so I hope to meet other interested
>> parties there.
>
> Welcome and I look forward you meeting you in Tokyo!
>
> -Ben
>
>
>> All the best,
>> John
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev