Hi all, Just reaching you out with a brief PTG summary for the Manila team. Thanks to everyone who attended and participated in-person and remote! Here's the PTG notes etherpad: https://etherpad.openstack.org/p/manila-ptg-train *High points* - We had a highly productive PTG, albeit shorter than the previous times we've met as a group. It reminded us of the old Design Summit times, although we weren't doing smaller time slots during the Summit week, but rather had a separate time to catch up. - We had ample time to do cross-project sync up; however, there were unavoidable overlaps. - It was also great to have many engineers travel to both the summit and the PTG. The following is a summary of notes organized with action items and owner/s. *Cross project goals* - Summary: - Identified current cross project goals, being them PDF rendering and IPv6 support - For the first case, we rely on the current documentation we have and on the docs team to guide us on the changes we need to do to have PDF rendering support - For the latter, we have been supporting IPv6 on manila for a while now and we have CI enabled to exercise this configuration. - Action: - Work with asettle to understand what it's need to be done to comply with the PDF rendering goal - Create an IPv6 only job in manila - many of our jobs run with 4+6, where we're currently doing IPv6 data path testing. - Owner: Not yet defined *Auto-snapshotting of manila shares* - Summary: - CERN had a use case that required cloud administrator driven automatic snapshots at pre-configured intervals, with pre-configured retention policies. - The team discussed ways the automation can occur with the existing API. - There are some backend specific extra-specs that can allow offloading the responsibility of periodic snapshots, and pruning of snapshots per retention policy. There is a disadvantage here that these snapshots are invisible to manila; and do not count against the project quotas - They also require knowing backend special sauce. - Snapshotting of multiple shares can be achieved with Share Groups - To make this occur without end-user actions, one would need to build tooling with these APIs - Maybe Mistral can be harnessed to do workflow scheduling and automation per business logic. - Action: CERN will explore these options and revert with their thoughts - Owner/s: CERN/Jose Castro Leon/josecastroleon *Manila in k8s* - Summary: - Robert Vasek/gman0 has an initial implementation of the CSI driver on the cloud-provider-openstack repo. We're seeking reviewers to merge it - https://github.com/kubernetes/cloud-provider-openstack/pull/536 - Next set of TODOs: - Robert's application to GSOC is pending, if he gets sponsorship, he might lead some of the following efforts: - e2e testing in the OpenLab CI for the manila-csi-provisioner - Testing with an NFS backend - AZ awareness / Topology based scheduling support - Snapshot support - Volume expansion - NoAuth mode for using the provisioner in single-tenant deployments - Manila differentiates abilities to take snapshots of shares, cloning these snapshots into new shares, mounting snapshots (with access control) and reverting shares to snapshots in place. CephFS supports snapshots, however, does not support rapid cloning - this feature gap exists for OpenStack use case as well. - Plans to build snapshot cloning capability into Manila and generically within the CSI provisioner were discussed. We'll evolve this design in the coming months. - Action Items: - Continue on plan to build and support manila-csi-provisioner (gman0, Tomas Smatena/tsmatena, Goutham Ravi/gouthamr, Tom Barron/tbarron, Victoria Martinez de la Cruz/vkmc, manila community) - Gather user feedback for the usecase for CEPH snapshots (josecastroleon and Xing Yang/xyang - Ask CERN and Huawei if they care about a slower "create-share-from-snapshot" over not having the ability at all. - Owners: as identified above ^ *OpenStack Client and OpenStack SDK* - Summary: - Manila lags in supporting OSC. Work began late in the Stein cycle to bridge this gap: - Specification: https://review.opendev.org/#/c/644218/ - Initial Code: https://review.opendev.org/#/c/642222/ - We have an Outreachy Intern starting with the manila team (Soledad Kuczala/s0ru) - Sofia Enriquez/enriquetaso and vkmc will be mentoring the intern - CERN may offer cycles to cover some of the development work - Amit Oren/amito submitted changes to add manila support to the openstacksdk: - *https://review.opendev.org/#/c/638782/* <https://review.opendev.org/#/c/638782/> - Team agreed to use the existing manilaclient SDK (in python-manilaclient) for the OSC instead of using the openstacksdk right away until we achieve feature parity in the openstacksdk project - Action Items: - Start Working Group to plan/coordinate OSC implementation - Identify minimum set of commands to implement and extended set to achieve parity with existing client - Allow support for using "openstack share" and "manila" to invoke manila CLI commands - the latter is essential to support manila running standalone. - Deprecate the existing manila clients in favor of the newer OpenStack CLI (and corresponding manila bash shell equivalent) - Review the openstacksdk change and identify feature gaps - Owner/s: s0ru, enriquetaso, vkmc, amito, gouthamr, josecastroleon *SemVer in manila releases* - Summary: This was a recap of how we are versioning the manila deliverables as we release them - "manila" gets a major version bump every release - Client releases indicate backwards compatibility by not bumping up the major release version. Minor version (and micro-version) bumps are done per policy based off of what changes are being released - The general guideline to follow is that of pbr versioning notes here: https://docs.openstack.org/pbr/latest/user/semver.html - Defer to the judgement of the OpenStack release team when in doubt - Action items: None - Owner/s: None *Create share from snapshots in another pool or back end* - Summary: - This specification has new owners who explained how they plan to implement this feature - The main concerns for the proposed spec ( https://review.opendev.org/#/c/609537/) were around the user experience and administrator control of the scheduling - Existing scheduling capabilities will be used to determine when a snapshot has to be cloned into a new pool rather than its existing pool. - Users can influence the scheduler's decision by specifying the availability zone or a different share type - Implementation will be two-phased - a driver optimized approach followed by a generic approach - Action Items: - Re-target specification to Train, assign to new owners - Implement driver-driven snapshot cloning to different pool - Owner/s: Douglas Viroel/dviroel, Lucio Seki/lseki, Erlon Cruz/erlon *Security Service password* - Summary: - Manila stores the security service password as plaintext in its database. It also allows all users within the tenant to retrieve the password via the security service API. - https://bugs.launchpad.net/manila/+bug/1817316 - We discussed adding barbican and castellan support to manila and making it a hard dependency by including their clients into the requirements.txt - Appropriate key management software will be a soft dependency, and there will be a recommendation about which key management backend can be used - a soft dependency to manila. - Action Items: - Disallow retrieving password from the security services API - Investigate use of Barbican/Castellan to store security services API password - change representation in the database - Owner/s: Not yet identified *Edge Requirements and active/active* - Summary: - Active-active - Manila share manager service running active/active - this has been a request from a long time, from traditional data center workloads as well as for newer/Edge workloads - Ganesha active/active - this allows datapath high availability for the CephFS-NFS driver - Edge - The goal of edge is to distribute services better, with latency considerations. We rely in deployment tools such as TripleO to do this work. This implementation has started already in some other projects (e.g. Cinder). TripleO supports and edge topology already. It's mainly hyperconverged OpenStack at the edge. - Initial users are telco customers running NFVs, 5G - Other services have been configured to be deployed on the edge - Cinder started on this effort last cycle - Cinder - qualify at least one driver to work in an active-active volume service - Is tooz supported for cinder-volume at the edge? - What tooz backend is being deployed? -> etcd deployed at each edge - Action items: - Introduce a multi-node job in the manila repository to test active-active deployment in the gate - Audit the usage of oslo_concurrency based file locks in the share manager service, replace these with tooz based locks - Design a leader election based workflow for asynchronous/time based tasks occuring within the share manager - Owner/s: gouthamr *OpenStack technical vision* - Summary: - This was a discussion of the TC OpenStack Vision Document: https://governance.openstack.org/tc/reference/technical-vision.html - Tom submitted a proposal that is for review in https://review.openstack.org/#/c/636770/. Based on this we exchanged some ideas regarding the accuracy of this document. - Action items: Reviewers are needed to check this proposal. - Owner/s: tbarron *API Improvements: Pagination and Filtering* - Summary: In the last couple of cycles several bugs were filed wrt pagination and filtering for our API. Precisely, - Can't filter/list resources that have no key=value metadata/extra-specs (https://bugs.launchpad.net/manila/+bug/1782847) - Pagination does not speed up list queries ( https://bugs.launchpad.net/manila/+bug/1795463) - next share_links contains not supported marker field ( https://bugs.launchpad.net/manila/+bug/1819167) - Action items: Look for volunteers wishing to work on the reported bugs. Bug/1795463 is being followed up by carloss. - Owner/s: Each bug has its asignee *CI Improvements: Rationalizing our jobs* - Summary: This discussion focused on the current status of our CI. It's fairly complex and we noticed that we need to simplify it to focus our resources better. Main highlights are: - We test 8 FOSS/first party drivers (Generic DHSS=True, Generic DHSS=False, Container, LVM, ZFSOnLinux, CEPHFS-Native, CEPHFS-NFS, ~~~Dummy~~~) - We test two databases - mysql, postgresql - We run API and scenario tests (together in some cases, individually as "manila-tempest-dsvm-scenario" (Generic driver DHSS=True) - We run python2 and python3 jobs - We run same jobs in manila-tempest-plugin and manila repos - Action items: - Generic driver with DHSS=False will be removed; - the job for Generic driver with DHSS=True can be combined with dsvm-scenario. - Unify tempest api and scenario tests for all drivers, even if that means we must bump timeouts. - Owner/s: gouthamr *Generic Driver improvements* - Summary: - There are lot of issues (even for usage at the gate) wrt the Generic Driver and nobody has enough bandwitch to work on all of them. Some of the known issues are: - It has a single point of failure, since there is no HA for the share servers created by manila - Attach / re-attach issues - timeouts - We need to figure out is there are people using the generic driver, understand better if we need to keep it or if it can be substituted for a different driver. Ben Swartzlander had a "NextGen" generic driver that is stuck in review because there's no migration path to move shares between the two different implementations of the generic driver: - https://review.opendev.org/#/c/511038/ - There are some production users of the Generic Driver that seem to have solved the SPOF (Single point of failure) issues in private forks of manila. If they are interested in sharing the love upstream, the manila community will be very welcoming. - Action items: - Continue to look for alternatives to the generic driver for CI. - Discuss in following upstream Manila meetings. - Owner/s: None *Bugs* - Summary: - Force flag on the snapshot creation API ( https://bugs.launchpad.net/manila/+bug/1811336) Considered to be a doc bug. Need to clarify this in the bug report. Marked as a "low-hanging-fruit" for new volunteers. - Limit share size (https://bugs.launchpad.net/manila/+bug/1811943). The team is interested in this one, we replied to the reporter during the PTG. We need to keep the dicussion going and look for a volunteer to work on this. - Incompatible protocol/access types in the API ( https://bugs.launchpad.net/manila/+bug/1637542) - Action items: None in particular, triaged during the PTG - Owner/s: None Cheers, Victoria