Open Stack

Hi all,

my team is evaluating the cinder-backup multi backends configuration spec. 
It seems this spec fulfills our needs as it is, so we are considering working on its 
implementation, but we cannot at the moment commit to deliver this feature.

About the improvement on NFS backup driver to allow backups on private NFS
servers, we decided that we won't try to upstream this feature, based on feedback 
that we received.
 
We also won't bring these topics to be discussed on Cinder PTG meeting.

I would like to thank Gorka Eguileor, Brian Rosmaita and Arkady Kanevsky
for your comments in this thread.

Regards,
Daniel Pereira.


From: Gorka Eguileor <geguileo at redhat.com>
Sent: Monday, October 4, 2021 7:23 AM
To: Pereira, Daniel Oliveira <Daniel.Pereira at windriver.com>
Cc: openstack-discuss at lists.openstack.org <openstack-discuss at lists.openstack.org>
Subject: Re: [dev][cinder] Consultation about new cinder-backup features 
 
[Please note: This e-mail is from an EXTERNAL e-mail address]

On 30/09, Daniel de Oliveira Pereira wrote:
> On 06/09/2021 10:28, Gorka Eguileor wrote:
> > [Please note: This e-mail is from an EXTERNAL e-mail address]
> >
> > On 27/08, Daniel de Oliveira Pereira wrote:
> >> Hello everyone,
> >>
> >> We have prototyped some new features on Cinder for our clients, and we
> >> think that they are nice features and good candidates to be part of
> >> upstream Cinder, so we would like to get feedback from OpenStack
> >> community about these features and if you would be willing to accept
> >> them in upstream OpenStack.
> >
> > Hi Daniel,
> >
> > Thank you very much for your willingness to give back!!!
> >
> >
> >>
> >> Our team implemented the following features for cinder-backup service:
> >>
> >>     1. A multi-backend backup driver, that allow OpenStack users to
> >> choose, via API/CLI/Horizon, which backup driver (Ceph or NFS, in our
> >> prototype) will be used during a backup operation to create a new volume
> >> backup.
> >
> > This is a feature that has been discussed before, and e0ne already did
> > some of the prerequisites for it.
> >
> >
> >>     2. An improved NFS backup driver, that allow OpenStack users to back
> >> up their volumes to private NFS servers, providing the NFS hostpath at
> >> runtime via API/CLI/Horizon, while creating the volume backup.
> >>
> >
> > What about the username and password?
>
> Hi Gorka,
>
> thanks for your feedback.
>
> Our prototype doesn't support authentication using username/password,
> since this is a feature that NFS doesn't provide built-in support.
>
> > Can backups be restored from a remote location as well?
>
> Yes, if the location is the one where the backup was originally saved
> (same NFS hostpath), as the backup location is stored on Cinder backups
> table during the backup creation. It doesn't support restoring the
> backup from an arbitrary remote NFS server.
>
> >
> > This sounds like a very cool feature, but I'm not too comfortable with
> > having it in Cinder.
> >
> > The idea is that Cinder provides an abstraction and doesn't let users
> > know about implementation details.
> >
> > With that feature as it is a user could request a backup to an off-site
> > location that could result in congestion in one of the outbound
> > connections.
>
> I think this is a very good point, that we weren't taking into
> consideration in our prototype.
>
> >
> > I can only think of this being acceptable for admin users, and in that
> > case I think it would be best to use the multi-backup destination
> > feature instead.
> >
> > After all, how many times do we have to backup to a different location?
> > Maybe I'm missing a use case.
>
> Our clients have privacy and security concerns with the same NFS server
> being shared by OpenStack tenants to store volume backups, so they
> required cinder-backup to be able to back up volumes to private NFS servers.
>
> >
> > If the community thinks this as a desired feature I would encourage
> > adding it with a policy that disables it by default.
> >
> >
> >> Considering that cinder was configured to use the multi-backend backup
> >> driver, this is how it works:
> >>
> >>     During a volume backup operation, the user provides a "location"
> >> parameter to indicate which backend will be used, and the backup
> >> hostpath, if applicable (for NFS driver), to create the volume backup.
> >> For instance:
> >>
> >>     - Creating a backup using Ceph backend:
> >>     $ openstack volume backup create --name <backup_name> --location
> >> ceph <volume_id>
> >>
> >>     - Creating a backup using the improved NFS backend:
> >>     $ openstack volume backup create --name <backup_name> --location
> >> nfs://my.nfs.server:/backups <volume_id>
> >>
> >>     If the user chooses Ceph backend, the Ceph driver will be used to
> >> create the backup. If the user chooses the NFS backend, the improved NFS
> >> driver, previously mentioned, will be used to create the backup.
> >>
> >>     The backup location, if provided, is stored on Cinder database, and
> >> can be seen fetching the backup details:
> >>     $ openstack volume backup show <backup_name>
> >>
> >> Briefly, this is how the features were implemented:
> >>
> >>     - Cinder API was updated to add an optional location parameter to
> >> "create backup" method. Horizon, and OpenStack and Cinder CLIs were
> >> updated accordingly, to handle the new parameter.
> >>     - Cinder backup controller was updated to handle the backup location
> >> parameter, and a validator for the parameter was implemented using the
> >> oslo config library.
> >>     - Cinder backup object model was updated to add a nullable location
> >> property, so that the backup location could be stored on cinder database.
> >>     - a new backup driver base class, that extends BackupDriver and
> >> accepts a backup context object, was implemented to handle the backup
> >> configuration provided at runtime by the user. This new backup base
> >> class requires that the concrete drivers implement a method to validate
> >> the backup context (similar to BackupDriver.check_for_setup_error)
> >>     - the 2 new backup drivers, previously mentioned, were implemented
> >> using these new backup base class.
> >>     - in BackupManager class, the "service" attribute, that on upstream
> >> OpenStack holds the backup driver class name, was re-implemented as a
> >> factory function that accepts a backup context object and return an
> >> instance of a backup driver, according to the backup driver configured
> >> on cinder.conf file and the backup context provided at runtime by the user.
> >>     - All the backup operations continue working as usual.
> >>
> >
> > When this feature was discussed upstream we liked the idea of
> > implementing this like we do multi-backends for the volume service,
> > adding backup-types.
>
> I found this approved spec [1] (that, I believe, is product of the work
> done by eOne that you mentioned before), but I couldn't find any work
> items in progress related to it.
> Do you know the current status of this spec? Is it ready to be
> implemented or is there some more work to be done until there? If we
> decide to work on its implementation, would be required to review, and
> possibly update, the spec for the current development cycle?
>
> [1]
> https://specs.openstack.org/openstack/cinder-specs/specs/victoria/backup-backends-configuration.html
>

Hi,

I think all that would need to be done regarding the spec is to submit a
patch to move it to the current release directory and fix the formatting
issue of the tables from the "Data model impact" section.

You'll be able to leverage Ivan's work [1] when implementing the
multi-backup feature.

Cheers,
Gorka.

[1]: https://review.opendev.org/c/openstack/cinder/+/630305

>
> >
> > In latest code backup creation operations have been modified to go
> > through the scheduler, so that's a piece that is already implemented.
> >
> >
> >> Could you please let us know your thoughts about these features and if
> >> you would be open to adding them to upstream Cinder? If yes, we would be
> >> willing to submit the specs and work on the upstream implementation, if
> >> they are approved.
> >>
> >> Regards,
> >> Daniel Pereira
> >>
> >
> > I believe you will have the full community's support on the first idea
> > (though probably not on the proposed implementation).
> >
> > I'm not so sure on the second one, iti will most likely depend on the
> > use cases.  Many times the reasons why features are dismissed upstream
> > is because there are no clear use cases that justify the addition of the
> > code.
> >
> > Looking forward to continuing this conversation at the PTG, IRC, in a
> > spec, or through here.
> >
> > Cheers,
> > Gorka.
> >
>