Open Stack

Hello Zorillas and all other interested Stackers!

The OpenStack Manila Project Technical Gathering for the Wallaby
development cycle was a successful meeting of project developers, users,
operators and contributors to the rest of OpenStack, Kubernetes, Ceph and
other communities - simply all zorillas and friends!

I would like to thank the PTG organizing committee, (the Kendalls! :D) the
infrastructure team, and everyone that contributed to the discussions. The
following is my summary of the proceedings. I've linked associated
etherpads, and resources to dig in further, and chat with us here, or on
freenode's #openstack-manila!

== Day 1 - Oct 26, 2020, Mon ==

=== Manila Interop testing ===
Topic Etherpad: https://etherpad.opendev.org/p/wallaby-ptg-manila-interop
Discussion Items:
- OSF Foundation Board and Interop Working Group members were apprised
about Manila, its capabilities and problem space
- Manila contributors answered questions regarding user workflows, tempest
integration, vendor coverage and project maturity
- Manila's addition to the Interop refstack suite will allow:
  * OpenStack clouds and distributions to certify their manila APIs for the
"OpenStack Powered" trademark program, as well as,
  * OpenStack Manila third party integrations can claim the "OpenStack
Compatible" label
- A plan was charted to target an upcoming interop advisory with manila as
an "Add-On" component
- Interop team presented their idea for E2E testing for running container
workloads on top of openstack

Actions:
- Vida Haririan (vhari), Liron Kuchlani (lkuchlan) and Goutham Pacha Ravi
(gouthamr) will create the interop test plan for Manila
- Test plan target is the upcoming 2020.10 advisory


=== Manila Retrospective ===
Topic Etherpad: https://etherpad.opendev.org/p/manila-victoria-retrospective

Discussion Items:
  - Victoria Cycle Bug/Documentation Squash days were a big hit. 45% of the
bug backlog was addressed with these squashes - thanks vhari/vkmc!
  - Team's doing a great job of farming low-hanging-fruit; but resolution
is taking longer, we should timebox fixes for Medium prio
low-hanging-fruit; high prio bugs cannot be tagged "low-hanging-fruit"
  - Team recognized the mentorship eagerness, and the opportunity providers
in Victoria/Wallaby - Grace Hopper Celebration, Outreachy, Boston
University Project and other venues
  - We couldn't complete the "migrate to focal fossa goal" - team was short
handed at the end of the release, thanks to the goal champion, Ghanshyam
Mann for ensuring we can make decent progress and target completion in
wallaby
  - New TC tags (kubernetes-in-virt, assert:supports-accessible-upgrade,
erstwhile tc:approved-release) were great, more to come in Wallaby
(vulnerability-managed, supports-api-interoperability)
  - A GREAT DEAL of work was done on manila-tempest-plugin, and the team is
thankful, and is sure users are appreciative of the focus we have on CI.
  - Great PTL leadership - but there's room to grow :D
  - Collaborative review sessions are a hit. Let's keep doing them!

Actions:
  - We'll have a bug squash event closer to Wallaby-1 for bigger bugs that
require API/DB changes as identified in the last Cycle (vkmc/vhari)
  - We'll have Bug/Documentation Squash after Wallaby-3 as well
  - Work on TC tags by Wallaby-1
  - Contributors can actively seek Collaborative Review sessions for the
work they're doing
  - CI flakiness needs to be audited and recurring test failures will be
identified (dviroel) and discussed at an upcoming IRC meeting
  - Backend Feature Support Matrix will be improved and doc bugs will be
filed for other issues identified (dviroel)


=== Cephadm, Triple-O and NFS-Ganesha ===
Topic Etherpad: https://etherpad.opendev.org/p/tripleo-ceph
Discussion Items:
- fultonj/gfidente/fpantano presented three specs to chart the future of
ceph deployment with tripleo. The ceph community will no longer support
ceph-ansible as a deployment tool.
- particularly interesting to the manila team was the fate of nfs-ganesha
("ceph-nfs") which doesn't have adequate support in ceph's new deployment
tooling (cephadm and ceph orch) to cover tripleo's requirements of HA, or
allow deployment flexibility like ceph-ansible used to (ceph-ansible would
allow deploying nfs-ganesha as a standalone service on tripleo hosts)
- the proposal was to retain the portions of ceph-ansible that were used
for nfs-ganesha in tripleo while replacing the use of ceph-ansible with
cephadm/ceph orch.
- tripleo contributors were concerned about the future supportability of
taking over the portion of the setup that ceph ansible does for tripleo
today; they would be more comfortable if cephadm can be fixed in time for
wallaby, or the portions of the code taken from ceph-ansible be in an
independent repo

Actions:
      - pursue cephadm/orch support for nfs-ganesha with ceph-orchestration
team and understand timeline and upgrade implications when substituting the
work
      - advance the work via specs

== Day 2 - Oct 27, 2020, Tue ==

=== Sub teams ===
Topic Etherpad: https://etherpad.opendev.org/p/wallaby-ptg-manila-subteams
Discussion Items:
    - gouthamr/vkmc proposed that client repos (python-manilaclient,
manila-ui) and tempest (manila-tempest-plugin) have their own core teams
    - manila-core can be part of the initial core teams of those repos
    - the idea is to create focussed subteams that can to some degree of
autonomy, govern these projects with the domain knowledge they have, have
independent meetings and bug triages if they wish and dictate a roadmap -
this way, review bandwidth can be adequately shared
    - the sub teams will allow manila core team mentoring efforts and
provide an easier on ramp for new maintainers
    - team agreed that this is a good way forward, and acknowledged that
they would not want to create more maintenance burden or create silos.

Actions:
    - get https://review.opendev.org/758868/ merged
    - to avoid siloing, weekly IRC meeting will call upon subteams to share
updates
    - focussing on a sub project doesn't mean we don't pay attention to
other repos
    - attempt to record mentoring sessions and post it publicly for
posterity

=== Extending shares via the Manila scheduler ===
Topic Etherpad:
https://etherpad.opendev.org/p/share-stuck-in-extend-discussion
Discussion Items:
    - manila share extension currently doesn't go through the scheduler -
they directly hit the share service/host responsible for the share (
https://bugs.launchpad.net/manila/+bug/1855391)
    - this needs to be fixed for two reasons - a) capacity based
calculations are off if scheduler is ignored, b) service health checks are
not accounted for
    - there was a concern this change would break administrators that rely
on this wrong behavior as storage pools fill up
    - this was called out as a corner case, and instead of having a global
way to opt-in/out of using the scheduler for share extensions, we can build
a "force" flag for privileged users that will preserve existing behavior

Actions:
    - haixin will propose changes to the patch based on this feedback

=== Virtio-FS ===
Topic Etherpad:
https://etherpad.opendev.org/p/nova-manila-virtio-fs-support-vptg-october-2020
Discussion Items:
    - virtiofs support has debuted in newer linux kernels and the
libvirt/qemu/kvm communities have adopted it
    - in OpenStack, Nova can be made to use virtio-fs and expose Manila
shares to tenant VMs
    - this provides a user experience improvement for manila users where
mount automation has been a concern for a while, along with the desire to
have strong multi-tenant separation on the network path which only a subset
of manila backends are able to provide directly
    - virtiofs support does not replace DHSS=True or use of gateways like
NFS-Ganesha (e.g.: CephFS, GPFS, etc) since those are still very much
valuable when the consumer isn't nova VMs but baremetals, containers or
non-OpenStack VMs.
    - nova team helped understand the feature submission process and
provided guidance on how to integrate this feature in stages
    - the user experience, data models and cross service RPCs for this can
be modeled around block device mapping, however we don't need to share or
copy the implementation
    - this is surely a large multi-cycle work - nova team suggested that we
begin with only attach/detach workflows in Wallaby

Actions:
    - tbarron will propose a nova spec to allow "nova attach fs" (and
detach) workflows
    - we'll continue to design "boot with share" or "boot from share" APIs,
including scheduler changes through this cycle

=== Share Server Updates ===
Topic Etherpad: https://etherpad.opendev.org/p/share-server-update
Discussion Items:
    - objective is to be able to update security services and share network
subnets after share servers have been provisioned
    - users that want to make these updates have no access to share servers
    - a security service update can impact multiple share networks (and
hence multiple share servers) and in turn multiple share resources, so
there was a concern with granularity of the update
    - operators don't want to be involved with the updates, want users to
be able to make the updates on resources they can see (security services or
share networks)
    - if we allow end users to make these changes, we'll still need to
break down the operation into more granular bits, and provide APIs to
perform validation and multi-step updates

Actions:
    - dviroel will propose spec updates based on the feedback gathered
here: https://review.opendev.org/729292
    - need other DHSS=True driver maintainers to pay attention to these
changes

=== Share Affinity Hints ===
Topic Etherpad: https://etherpad.opendev.org/p/manila-anti-affinity
Discussion Items:
    - need a way to specify affinity or anti-affinity placement hints wrt
existing resources (e.g: shares, share groups)
    - share groups provide a way to indicate affinity, but they may also
encase shares into a construct and make individual operations on shares
harder (you cannot add existing shares to a group or remove shares from a
share group without deleting them, you can only create new members in a
share group)
    -  need a vehicle other than groups or share type extra specs to carry
scheduling decisions that affect single shares
    - this mechanism shouldn't break cloud storage abstraction
    - cinder has had the ability for end users to add placement hints for
affinity without breaking abstractions

Actions:
    - carthaca will propose a specification for this work
    - we'll seek reviews from cinder contributors on the spec

== Day 3 - Oct 28, 2020, Wed ==

=== Responsibility for Code Reviews ===
Topic Etherpad: https://etherpad.opendev.org/p/wallaby-ptg-manila
Discussion Items:
    - a bulk of the day-2 operations that are being enhanced have
significant impact on vendor driver implementations
    - we do have first party "reference" driver support for testing and
designing manila APIs in a generic fashion but, not having vendor driver
maintainer feedback is unfortunate
    - operator feedback is also very much appreciated, and we've made
significant decisions because of it

Actions:
    - proposers for new features that affect drivers must actively seek out
vendor driver maintainers and operators and add them to review. The team
will help identify these individuals
    - we'll track vendor/operator feedback and reviews as part of the
review focus tracking we do through the cycle

=== Acting on Deprecation removals ===
Topic Etherpad: https://etherpad.opendev.org/p/wallaby-ptg-manila
Discussion Items:
    - tbarron proposed https://review.opendev.org/745206/ to remove old
deprecated config options from manila
    - we should be doing this more actively each cycle, however, we held
off from merging the code change above considering the impact to
configuration and deployment tooling
    - driver module maintainers must adjust the deployment tooling/other
usages accordingly asap

Actions:
    - dviroel, carloss and gouthamr will review and merge
https://review.opendev.org/745206/

=== rootwrap-to-privsep migration ===
Topic Etherpad:
https://etherpad.opendev.org/p/manila-migration-from-rootwrap-to-privsep
Discussion Items:
    - this has been selected as a community goal for Wallaby
    - privsep has not been used in manila so far
    - most first party and reference drivers (lvm, zfsonlinux, generic,
container, ceph) use rootwrap heavily
    - some third party drivers also use rootwrap
    - team was concerned with the completion criteria wrt third party
drivers
    - a stretch goal would be to investigate if dropping privileges for
operations by substituting mechanisms is possible

Actions:
    - carloss will work on this goal, no specification is necessary
    - adequate testing, admin/operator documentation will be provided
    - carloss will reach out to the third party driver maintainers to act
on rootwrap removals
    - carloss/team will attack de-escalation/removing use of root privs to
perform an operation, opportunistically

=== Secure RBAC improvements ===
Topic Etherpad:
https://etherpad.opendev.org/p/wallaby-ptg-manila-policy-changes
Discussion Items:
    - manila will need to support splitting of the existing admin, member
roles to system, domain and project admins and members - operators can
define personas with these default roles using keystone's "role scope"
    - an accompanying feature is to provide "reader" role permissions as
well
    - manila accepts the use of "policy.yaml" to drive RBAC, over
"policy.json", but, the default is still "policy.json" and this needs to
change due to several challenges, including the ability to have comments in
the files

Actions:
    - gouthamr will propose a spec for policy refresh across manila
APIs/resources using these new role personas
    - making policy.yaml the default for policy overrides is going to be a
community goal. We're targeting this transition to wallaby-1 in manila
    - we'll attempt to complete this effort in wallaby. there are several
parts, collaboration is welcome!

=== CephFS Updates ===
Topic Etherpad:
https://etherpad.opendev.org/p/wallaby-ptg-manila-cephfs-driver-update
Discussion Items:
    - vkmc provided an update on the cephfs feature roadmap
    - we need changes in cephfs (preferably in ceph-nautilus) to perform
authorization, subvolume cloning, capacity reporting etc. This work is
being tracked through tracker tickets in Ceph's tracking system
    - we stopped using shaman builds for ceph and nfs-ganesha in victoria
cycle - consuming nfs-ganesha from ppa's has introduced new scenario test
failures, vkmc is currently triaging them

Actions:
    - start testing ceph octopus in the wallaby cycle
    - keep momentum on the ceph trackers and switch over from
ceph-volume-client to ceph-mgr and implement create share from snapshot in
the cephfs drivers
    - engage with ceph and nfs-ganesha communities to create a support
matrix across nfs-ganesha versions, manila and cephfs

=== Divisive Language Correction ===
Topic Etherpad: https://etherpad.opendev.org/p/wallaby-ptg-manila
Discussion Items:
    - spotz joined us, and recapped the diversity and inclusion wg's work
so far
    - manila team is determined to help resolve any divisive language and
references in code and documentation
    - the team realizes that unconscious use of negative language is
affecting the community's inherent desire to be welcoming and inclusive
    - no manila architecture or code references currently *need* the use of
the divisive words identified by the working group and the wider openstack
community
    - making changes is important to us, but we recognize it's just part of
the story

Actions:
    - we'll begin with a documentation change to remove the words
identified in https://etherpad.opendev.org/p/divisivelanguage
    - we'll wait to coordinate a change of name for the development/trunk
branch, and other "upstream" changes to databases, git, opendev
    - team actively welcomes any feedback

== Day 5 - Oct 30, 2020, Fri ==

=== Share and Server Migration graduation ===
Topic Etherpad: https://etherpad.opendev.org/p/wallaby-ptg-manila
Discussion Items:
    - we've graduated one major feature through the last couple of cycles,
share migration is next
    - team agreed to keep share server migration as experimental APIs since
they haven't had sufficient soak time
    - no significant changes have been made to share migration migration
APIs since Ocata, leading to believe the APIs are stable and can be
improved via microversions where necessary
    - not a lot of vendor driver implementations for driver optimized
migration, NetApp driver only allows moving within a backend at the moment
    - generic migration improvements have been deprioritized, and need help
in terms of High Availability of the Data service as well as providing a
jobs table to track, and coordinate ongoing migrations
    - tests are being skipped at the gate because of flakiness

Actions:
    - team will get in touch with third party vendors and guage interest in
implementing driver optimized migration (or testing/supporting generic
migration with their drivers) (carloss). Feedback from the vendors will
drive our decision to graduate the share migration APIs in Wallaby.

=== Metadata for all share resources ===
Topic Etherpad:
https://etherpad.opendev.org/p/wallaby-ptg-manila-metadata-controller
Discussion Items:
    - the proposal is to introduce a metadata API controller that can be
inherited by the resource controllers
    - all user facing resources will have metadata capabilities, with
consistent API methods
    - this is expected to benefit automation use cases, manila-csi and the
upcoming virtio-fs effort

Actions:
    - gouthamr will propose the spec for team's review, including
identifying the resources that will be targeted in the Wallaby release
    - share metadata APIs will be improved to adhere to the API SIG spec:
https://specs.openstack.org/openstack/api-sig/guidelines/metadata.html

=== Client Project updates ===
Topic Etherpads: https://etherpad.opendev.org/p/manila-osc-wallaby-ptg and
https://etherpad.opendev.org/p/manila-ui-wallaby-ptg
Discussion Items:
    - the team met with Nicole Chen, Ashley Rodriguez, Mark Tony from
Boston University who will be working on the OpenStackSDK integration for
Manila APIs
    - maaritamm provided an update on the OSC plugin interface
implementation
    - vkmc provided an update on manila-ui feature gaps and the upcoming
Outreachy internship

Actions:
    - for some OSC plugin command implementations, we'll benefit from
integrating with OpenStackSDK rather than manilaclient SDK, so we'll
coordinate that work
    - maaritamm/vkmc will send an email to openstack-discuss to prioritize
the commands left to implement in the OSC plugin
    - vkmc will drive consistency across cinder/manila for the user
messages dashboards that were added in Victoria
    - we'll continue to work on closing the feature gap in manila-ui

=== Manila CSI Wallaby roadmap ===
Topic Document:
https://docs.google.com/document/u/1/d/1YJcv0EROWzYbV_8DM3DHVvDqdhlPhYRV8Npl7latttY/edit#
Discussion Items:
    - users of manila-csi today deploy one driver per share protocol
    - future work on monitoring, common node plugin interactions requires
us to rework the architecture to have one driver multiplex with multiple
share protocol node plugins
    - upgrade impact as well as UI/UX impact was discussed and gman0 has
documented them
    - share metadata is necessary to tag and identify shares

Actions:
    - gman0 has started working on the code patches, and will benefit from
reviews
    - mfedosin will take a look at the upgrade impact and share details for
automation that can be done in the csi-operator
    - mfedosin will test gman0's latest patch to add share metadata via the
storage class (as will folks from CERN) and provide feedback


Thanks for reading thus far, there were a number of items that we couldn't
cover, that you can see on
the PTG Planning etherpad [1], if you own those topics, please add them to
the weekly IRC meeting agenda [2] and
we can go over them. The master meeting minutes document is available as
well [3].

As usual, the whole PTG was recorded and posted on the OpenStack Manila
Youtube channel [4]

We had a great turn out and we heard from a diverse set of contributors,
operators, interns and users from several affiliations and time zones. On
behalf of the OpenStack Manila team, I deeply appreciate your time, and
help in keeping the momentum on the project!

[1] https://etherpad.opendev.org/p/wallaby-ptg-manila-planning
[2] https://wiki.openstack.org/wiki/Manila/Meetings
[3] https://etherpad.opendev.org/p/wallaby-ptg-manila
[4] https://www.youtube.com/playlist?list=PLnpzT0InFrqATBbJ60FesKGYRejnTIoCW
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20201104/e3e703e4/attachment-0001.html>