Open Stack

Hi all,

Just reaching you out with a brief PTG summary for the Manila team. Thanks
to everyone who attended and participated in-person and remote!

Here's the PTG notes etherpad:
https://etherpad.openstack.org/p/manila-ptg-train

*High points*
- We had a highly productive PTG, albeit shorter than the previous times
we've met as a group. It reminded us of the old Design Summit times,
although we weren't doing smaller time slots during the Summit week, but
rather had a separate time to catch up.
- We had ample time to do cross-project sync up; however, there were
unavoidable overlaps.
- It was also great to have many engineers travel to both the summit and
the PTG.

The following is a summary of notes organized with action items and owner/s.

*Cross project goals*

   - Summary:


   - Identified current cross project goals, being them PDF rendering and
   IPv6 support


   - For the first case, we rely on the current documentation we have and
   on the docs team to guide us on the changes we need to do to have PDF
   rendering support


   - For the latter, we have been supporting IPv6 on manila for a while now
   and we have CI enabled to exercise this configuration.


   - Action:


   - Work with asettle to understand what it's need to be done to comply
   with the PDF rendering goal


   - Create an IPv6 only job in manila - many of our jobs run with 4+6,
   where we're currently doing IPv6 data path testing.


   - Owner: Not yet defined


*Auto-snapshotting of manila shares*

   -  Summary:


   - CERN had a use case that required cloud administrator driven automatic
   snapshots at pre-configured intervals, with pre-configured retention
   policies.


   - The team discussed ways the automation can occur with the existing
   API.


   - There are some backend specific extra-specs that can allow offloading
   the responsibility of periodic snapshots, and pruning of snapshots per
   retention policy. There is a disadvantage here that these snapshots are
   invisible to manila; and do not count against the project quotas - They
   also require knowing backend special sauce.


   - Snapshotting of multiple shares can be achieved with Share Groups


   - To make this occur without end-user actions, one would need to build
   tooling with these APIs - Maybe Mistral can be harnessed to do workflow
   scheduling and automation per business logic.


   - Action: CERN will explore these options and revert with their thoughts


   - Owner/s: CERN/Jose Castro Leon/josecastroleon


 *Manila in k8s*

   -  Summary:


   - Robert Vasek/gman0 has an initial implementation of the CSI driver on
   the cloud-provider-openstack repo. We're seeking reviewers to merge it


   - https://github.com/kubernetes/cloud-provider-openstack/pull/536


   - Next set of TODOs:


   - Robert's application to GSOC is pending, if he gets sponsorship, he
   might lead some of the following efforts:


   - e2e testing in the OpenLab CI for the manila-csi-provisioner


   - Testing with an NFS backend


   - AZ awareness / Topology based scheduling support


   - Snapshot support


   - Volume expansion


   - NoAuth mode for using the provisioner in single-tenant deployments


   - Manila differentiates abilities to take snapshots of shares, cloning
   these snapshots into new shares, mounting snapshots (with access control)
   and reverting shares to snapshots in place. CephFS supports snapshots,
   however, does not support rapid cloning - this feature gap exists for
   OpenStack use case as well.


   - Plans to build snapshot cloning capability into Manila and generically
   within the CSI provisioner were discussed. We'll evolve this design in the
   coming months.


   - Action Items:


   - Continue on plan to build and support manila-csi-provisioner (gman0,
   Tomas Smatena/tsmatena, Goutham Ravi/gouthamr, Tom Barron/tbarron, Victoria
   Martinez de la Cruz/vkmc, manila community)


   - Gather user feedback for the usecase for CEPH snapshots
   (josecastroleon and Xing Yang/xyang - Ask CERN and Huawei if they care
   about a slower "create-share-from-snapshot" over not having the ability at
   all.


   - Owners: as identified above ^


*OpenStack Client and OpenStack SDK*

   -  Summary:


   - Manila lags in supporting OSC. Work began late in the Stein cycle to
   bridge this gap:


   - Specification: https://review.opendev.org/#/c/644218/


   - Initial Code: https://review.opendev.org/#/c/642222/


   - We have an Outreachy Intern starting with the manila team (Soledad
   Kuczala/s0ru) - Sofia Enriquez/enriquetaso and vkmc will be mentoring the
   intern


   - CERN may offer cycles to cover some of the development work


   - Amit Oren/amito submitted changes to add manila support to the
   openstacksdk:


   -  *https://review.opendev.org/#/c/638782/*
   <https://review.opendev.org/#/c/638782/>


   - Team agreed to use the existing manilaclient SDK (in
   python-manilaclient) for the OSC instead of using the openstacksdk right
   away until we achieve feature parity in the openstacksdk project


   - Action Items:


   - Start Working Group to plan/coordinate OSC implementation


   - Identify minimum set of commands to implement and extended set to
   achieve parity with existing client


   - Allow support for using "openstack share" and "manila" to invoke
   manila CLI commands - the latter is essential to support manila running
   standalone.


   - Deprecate the existing manila clients in favor of the newer OpenStack
   CLI (and corresponding manila bash shell equivalent)


   - Review the openstacksdk change and identify feature gaps


   - Owner/s: s0ru, enriquetaso, vkmc, amito, gouthamr, josecastroleon


*SemVer in manila releases*

   - Summary: This was a recap of how we are versioning the manila
   deliverables as we release them


   - "manila" gets a major version bump every release


   - Client releases indicate backwards compatibility by not bumping up the
   major release version. Minor version (and micro-version) bumps are done per
   policy based off of what changes are being released


   - The general guideline to follow is that of pbr versioning notes here:
   https://docs.openstack.org/pbr/latest/user/semver.html


   - Defer to the judgement of the OpenStack release team when in doubt


   - Action items: None


   - Owner/s: None


*Create share from snapshots in another pool or back end*

   - Summary:


   - This specification has new owners who explained how they plan to
   implement this feature


   - The main concerns for the proposed spec (
   https://review.opendev.org/#/c/609537/) were around the user experience
   and administrator control of the scheduling


   - Existing scheduling capabilities will be used to determine when a
   snapshot has to be cloned into a new pool rather than its existing pool.


   - Users can influence the scheduler's decision by specifying the
   availability zone or a different share type


   - Implementation will be two-phased - a driver optimized approach
   followed by a generic approach


   - Action Items:


   - Re-target specification to Train, assign to new owners


   - Implement driver-driven snapshot cloning to different pool


   - Owner/s: Douglas Viroel/dviroel, Lucio Seki/lseki, Erlon Cruz/erlon


*Security Service password*

   - Summary:


   - Manila stores the security service password as plaintext in its
   database. It also allows all users within the tenant to retrieve the
   password via the security service API.


   - https://bugs.launchpad.net/manila/+bug/1817316


   - We discussed adding barbican and castellan support to manila and
   making it a hard dependency by including their clients into the
   requirements.txt


   - Appropriate key management software will be a soft dependency, and
   there will be a recommendation about which key management backend can be
   used - a soft dependency to manila.


   - Action Items:


   - Disallow retrieving password from the security services API


   - Investigate use of Barbican/Castellan to store security services API
   password - change representation in the database


   - Owner/s: Not yet identified


*Edge Requirements and active/active*

   - Summary:


   - Active-active


   - Manila share manager service running active/active - this has been a
   request from a long time, from traditional data center workloads as well as
   for newer/Edge workloads


   - Ganesha active/active - this allows datapath high availability for the
   CephFS-NFS driver


   - Edge


   - The goal of edge is to distribute services better, with latency
   considerations. We rely in deployment tools such as TripleO to do this
   work. This implementation has started already in some other projects (e.g.
   Cinder). TripleO supports and edge topology already. It's mainly
   hyperconverged OpenStack at the edge.


   - Initial users are telco customers running NFVs, 5G


   - Other services have been configured to be deployed on the edge


   - Cinder started on this effort last cycle


   - Cinder - qualify at least one driver to work in an active-active
   volume service


   - Is tooz supported for cinder-volume at the edge?


   - What tooz backend is being deployed? -> etcd deployed at each edge


   - Action items:


   - Introduce a multi-node job in the manila repository to test
   active-active deployment in the gate


   - Audit the usage of oslo_concurrency based file locks in the share
   manager service, replace these with tooz based locks


   - Design a leader election based workflow for asynchronous/time based
   tasks occuring within the share manager


   - Owner/s: gouthamr


*OpenStack technical vision*

   - Summary:


   - This was a discussion of the TC OpenStack Vision Document:
   https://governance.openstack.org/tc/reference/technical-vision.html


   - Tom submitted a proposal that is for review in
   https://review.openstack.org/#/c/636770/. Based on this we exchanged
   some ideas regarding the accuracy of this document.


   - Action items: Reviewers are needed to check this proposal.


   - Owner/s: tbarron


*API Improvements: Pagination and Filtering*

   - Summary: In the last couple of cycles several bugs were filed wrt
   pagination and filtering for our API. Precisely,


   - Can't filter/list resources that have no key=value
   metadata/extra-specs (https://bugs.launchpad.net/manila/+bug/1782847)


   - Pagination does not speed up list queries (
   https://bugs.launchpad.net/manila/+bug/1795463)


   - next share_links contains not supported marker field (
   https://bugs.launchpad.net/manila/+bug/1819167)


   - Action items: Look for volunteers wishing to work on the reported
   bugs. Bug/1795463 is being followed up by carloss.


   - Owner/s: Each bug has its asignee


*CI Improvements: Rationalizing our jobs*

   - Summary: This discussion focused on the current status of our CI. It's
   fairly complex and we noticed that we need to simplify it to focus our
   resources better. Main highlights are:


   - We test 8 FOSS/first party drivers (Generic DHSS=True, Generic
   DHSS=False, Container, LVM, ZFSOnLinux, CEPHFS-Native, CEPHFS-NFS,
   ~~~Dummy~~~)


   - We test two databases - mysql, postgresql


   - We run API and scenario tests (together in some cases, individually as
   "manila-tempest-dsvm-scenario" (Generic driver DHSS=True)


   - We run python2 and python3 jobs


   - We run same jobs in manila-tempest-plugin and manila repos


   - Action items:


   - Generic driver with DHSS=False will be removed;


   - the job for Generic driver with DHSS=True can be combined with
   dsvm-scenario.


   - Unify tempest api and scenario tests for all drivers, even if that
   means we must bump timeouts.


   - Owner/s: gouthamr


*Generic Driver improvements*

   - Summary:


   - There are lot of issues (even for usage at the gate) wrt the Generic
   Driver and nobody has enough bandwitch to work on all of them. Some of the
   known issues are:


   - It has a single point of failure, since there is no HA for the share
   servers created by manila


   - Attach / re-attach issues - timeouts


   - We need to figure out is there are people using the generic driver,
   understand better if we need to keep it or if it can be substituted for a
   different driver.  Ben Swartzlander had a "NextGen" generic driver that is
   stuck in review because there's no migration path to move shares between
   the two different implementations of the generic driver:


   - https://review.opendev.org/#/c/511038/


   - There are some production users of the Generic Driver that seem to
   have solved the SPOF (Single point of failure) issues in private forks of
   manila. If they are interested in sharing the love upstream, the manila
   community will be very welcoming.


   - Action items:


   - Continue to look for alternatives to the generic driver for CI.


   - Discuss in following upstream Manila meetings.


   - Owner/s: None


*Bugs*

   - Summary:


   - Force flag on the snapshot creation API (
   https://bugs.launchpad.net/manila/+bug/1811336) Considered to be a doc
   bug. Need to clarify this in the bug report. Marked as a
   "low-hanging-fruit" for new volunteers.


   - Limit share size (https://bugs.launchpad.net/manila/+bug/1811943). The
   team is interested in this one, we replied to the reporter during the PTG.
   We need to keep the dicussion going and look for a volunteer to work on
   this.


   - Incompatible protocol/access types in the API (
   https://bugs.launchpad.net/manila/+bug/1637542)


   - Action items: None in particular, triaged during the PTG


   - Owner/s: None


Cheers,

Victoria
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190514/4115dc8b/attachment-0001.html>