<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: small; display: inline !important; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">Hello to the list,</span>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
In OpenStack releases prior to Wallaby, in CephFS native driver Manila backend config there existed a configuration directive <a href="https://docs.openstack.org/manila/wallaby/configuration/shared-file-systems/samples/manila.conf.htm" target="_blank" data-saferedirecturl="https://www.google.com/url?q=https://docs.openstack.org/manila/wallaby/configuration/shared-file-systems/samples/manila.conf.htm&source=gmail&ust=1694861035652000&usg=AOvVaw16LG_WKe4TCGZaULy8RCOA" style="color: rgb(17, 85, 204);" id="OWAa7577b7b-aa13-7a91-98ad-a6ebbc9ec405" class="OWAAutoLink ContentPasted0" data-loopstyle="linkonly">cephfs_volume_path_<wbr>prefix</a> ;</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<blockquote class="gmail_quote ContentPasted0" style="font-family: Arial, Helvetica, sans-serif; font-size: small; margin: 0px 0px 0px 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
# DEPRECATED: The prefix of the cephfs volume path. (string value)<br class="ContentPasted0">
# This option is deprecated for removal since Wallaby.<br class="ContentPasted0">
# Its value may be silently ignored in the future.<br class="ContentPasted0">
# Reason: This option is not used starting with the Nautilus release<br class="ContentPasted0">
# of Ceph.<br class="ContentPasted0">
#cephfs_volume_path_prefix = /volumes</blockquote>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
This volume path prefix was usable (and very useful) to set different prefixes in different backends, pointing to different pools in the same backend Ceph cluster e.g. pools backed by storage devices of different characteristics/technology or Ceph CRUSH rule,
etc</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
The pool selection would be done in CephFS<span class="ContentPasted0"> </span><a href="https://docs.ceph.com/en/quincy/cephfs/file-layouts/" target="_blank" data-saferedirecturl="https://www.google.com/url?q=https://docs.ceph.com/en/quincy/cephfs/file-layouts/&source=gmail&ust=1694861035652000&usg=AOvVaw1JET32GuXs2gkGjZ2dJ5NX" style="color: rgb(17, 85, 204);" id="OWA4f6450f4-21d6-1ead-117c-afbb83097aeb" class="OWAAutoLink ContentPasted0" data-loopstyle="linkonly">via
the use of file layout fattributes</a>, file layout inheritance ensuring that data created in prefix child sub-directories ends up in the correct Ceph pool.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
e.g. two backends using the same Ceph cluster but two different path prefixes hitting different Ceph pools</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<blockquote class="gmail_quote ContentPasted0" style="font-family: Arial, Helvetica, sans-serif; font-size: small; margin: 0px 0px 0px 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
[cephfsnative1]<br class="ContentPasted0">
driver_handles_share_servers = False<br class="ContentPasted0">
share_backend_name = CEPHFS1<br class="ContentPasted0">
share_driver = manila.share.drivers.cephfs.<wbr class="ContentPasted0">driver.CephFSDriver<br class="ContentPasted0">
cephfs_conf_path = /etc/ceph/ceph.conf<br class="ContentPasted0">
cephfs_auth_id = manila<br class="ContentPasted0">
cephfs_cluster_name = ceph<br class="ContentPasted0">
cephfs_volume_path_prefix = /volumes-staging<br class="ContentPasted0">
cephfs_volume_mode = 777<br class="ContentPasted0">
cephfs_enable_snapshots = False</blockquote>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<div class="ContentPasted0"> </div>
<blockquote class="gmail_quote ContentPasted0" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
[cephfsnative1_ec]<br class="ContentPasted0">
driver_handles_share_servers = False<br class="ContentPasted0">
share_backend_name = CEPHFS1_EC<br class="ContentPasted0">
share_driver = manila.share.drivers.cephfs.<wbr class="ContentPasted0">driver.CephFSDriver<br class="ContentPasted0">
cephfs_conf_path = /etc/ceph/ceph.conf<br class="ContentPasted0">
cephfs_auth_id = manila<br class="ContentPasted0">
cephfs_cluster_name = ceph<br class="ContentPasted0">
cephfs_volume_path_prefix = /volumes-ec-staging<br class="ContentPasted0">
cephfs_volume_mode = 777<br class="ContentPasted0">
cephfs_enable_snapshots = False</blockquote>
<div><br class="ContentPasted0">
</div>
<div class="ContentPasted0">However, since Wallaby this config directive looks to have been deprecated and the default of using a path prefix of only /volumes is possible. Trying to use any other prefix in backend configs is ignored.</div>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
Would anyone on the list know why this option was deprecated in Manila code, or was this forced on Manila by upstream Ceph as of Nautilus? Is there a way to get back to an equivalent functionality?</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
Currently using only a default path of /volumes means we have lost all flexibility in defining Manila CephFS share data placement using the native CephFS driver.<br class="ContentPasted0">
<br class="ContentPasted0">
Possibly using share group types+share groups and some pre-created paths in the root CephFS could get to something like equivalency?</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
But these paths would need to correspond to the share group UUID, which will only be known after the share group has been created.<br class="ContentPasted0">
<br class="ContentPasted0">
So not all that flexible a path, since it requires an interaction between users/tenant to communicate the share group ID and Ceph admins to set the correct file layout policy. Putting the path prefix in the backend+share-type removed all of that in a nicely
transparent way.<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
Having just prototyped this, it will work for setting a desired file layout on a pre-defined share group UUID path in the root CephFS, though it's not really ideal or sustainable to be able to do this for dynamically created share groups by users or automation,
which could exist in large numbers...</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);">
<br class="ContentPasted0">
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
Thanks in advance for any advice,</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: small; color: rgb(34, 34, 34); background-color: rgb(255, 255, 255);" class="ContentPasted0">
Paul Browne</div>
<br>
</div>
<div class="elementToProof">
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; font-family: Calibri, Helvetica, sans-serif; color: rgb(0, 0, 0);">
<p style="margin-top:0; margin-bottom:0"></p>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="font-size: 12.8px; color: rgb(0, 0, 0);">*******************</span><br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">Paul Browne</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">Research Computing Platforms</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">University Information Services</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">Roger Needham Building</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">JJ Thompson Avenue</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">University of Cambridge</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">Cambridge</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">United Kingdom</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">E-Mail: </span><a href="mailto:pfb29@cam.ac.uk" target="_blank" style="color: rgb(17, 85, 204);"><span style="color: rgb(0, 0, 0);">pfb29@cam.ac.uk</span></a></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">Tel: 0044-1223-746548</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12.8px; color: rgb(80, 0, 80);">
<span style="color: rgb(0, 0, 0);">*******************</span></div>
<p></p>
</div>
</div>
</div>
</div>
</body>
</html>