Open Stack

Here is the updated (final) notes from the Trove design summit in
Austin, TX.

Some changes from yesterday's email are highlighted. 

Changes are marked with "::NOTE::"

-amrith


Python3 support

- make python34 a voting test in the gate (was already done, so no
  action required)

- peterstac to ensure that trove-dashboard is python3 enabled and
  enable python34 gate changes in trove-dashboard


Multiple datastores with a single manager

etherpad: trove-newton-summit-multiple-datastores

- needs more discussion


Management client and openstack client

::NOTE:: Change from yesterday's summary

- decided to add quota management to the Trove API and expose it
  through the trove client [flavio, maybe]


Trove upgrades

etherpad: trove-newton-summit-trove-upgrades
spec: https://review.openstack.org/#/c/302416/

- the team was in agreement with the proposal that has been up for
  review.

- pete mackinnon had a specific request that some thought and research
  be put into the area of how nova rebuild may fail, and how a user
  would have to recover from this.


Extend backend persistent storage

etherpad: trove-newton-summit-extensible-backend-storage
spec: https://review.openstack.org/#/c/302952/

- the team was in agreement with the proposal that has been up for
  review.

- need to investigate supporting replicated volumes for DR and
  restoring a new database instance from a volume replica. This
  workflow is not currently possible.


Trove container support

etherpad: trove-newton-summit-container
spec: https://review.openstack.org/307883

- rather than having trove integrate with magnum, we agreed that in
  the short term we would provide containers through the nova lxc or
  lxd drivers

- the spec will need to be rewritten to reflect this, and to identify
  the work involved in doing that

- in the future, trove will have to realize how it will deal with
  clouds where this solution would not work. for example, clouds that
  run kubernetes on native bare metal and don't have nova.


Snapshot as a backup strategy

etherpad: trove-newton-summit-snapshot-as-a-backup-strategy
spec: https://review.openstack.org/#/c/306620/

- the team was in agreement with the proposal that was up for review,
  in principle. some changes are required such as to deal with
  quiescing the database.

- there was discussion of whether the operation of taking the snapshot
  would be taken by the guest agent or the task manager and the
  consensus was that the task manager should be the one generating the
  snapshot.

- it was recommended that the task manager should make a synchronous
  call to the task manager to quiesce the database, and to release it
  after the snapshot was taken. this would avoid the asynchronous
  mechanism based on cast().

- there are two operations here that could cause an interruption in
  service of the database and these relate to the period of time when
  the database is quiesced. it is possible that the operation to
  quiesce the database may take a while to happen (for example if
  there is a long running transaction going on). this could cause the
  call() from the task manager to timeout. Similarly once the database
  is quiesced, the snapshot may take a while. It was suggested that
  there should be mechanisms to place limits on how long either of
  these can take.

- it should be configurable to determine where the snapshot will get
  stored; if we have to actually stream it somewhere. this is not
  necessarily the case for all storage solutions.

- point in time rollback is not a deliverable of this project.


Make it easier to build guest images

etherpad: trove-newton-summit-easier-to-build-images
spec: https://review.openstack.org/#/c/295274/

- the consensus was that we should create a new repository (named
  something like trove-image-builder) and pick up all the elements
  from trove-integration and move them here

- we need to write a new element to install the guestagent code into
  the image

- we need to write a image-builder script that would do the moral
  equivalent of redstack build-image

- that script/tool should allow for creation of both
  development/testing images and images that a customer could use.

- [amrith] to send a note to the ML with the [dib] in the subject line
  and keep the dib folks informed if there are real problems that we
  face with the tool

- pete mackinnon will be working to update and provide elements with
  dib that will generate database images for some databases and
  operating system (CentOS 7)

- there was no consensus on whether the future should be dib based or
  libvirt-customize based

- there was a concern that if we ended up with a different tool in the
  future, there would end up being duplication and redundancy in the
  image-builder tool

- once we get the elements out of trove-integration, if there's not
  much left there we can move it to the trove project and get rid of
  trove-integration altogether

::NOTE:: Change from yesterday's summary below

- in parallel work on libguestfs in parallel with the understanding
  that dib is the priority because it currently supports a number of
  databases and platforms already

- pete to update the spec to do all of this


Trove V2 API

etherpad: trove-newton-summit-v2-api

- need to review and think about this some more

- would like more details and feel that this spec needs to be fleshed
  out some more; for example what will the REST api look like


Modularity guest image and agent

etherpad: trove-newton-summit-modularity-guest-image

- needs more investigation and come up with a more concrete proposal.


Self signed certificates for Trove

etherpad: trove-newton-summit-ssl-self-signed-certificates

Baking self signed certificates onto the image is a non-starter.

Using cloud-init and sending down files at nova boot time is one
thing, but there is no such API on rebuild. Therefore an upgrade of an
instance could lose the CA Certificate.

If there is a multi-region deployment of Trove, access to RabbitMQ is
a challenge anyway and therefore we have to keep that in mind in
handling the certificates.


Superconductor

etherpad: trove-newton-summit-superconductor

- the objective is to get rid of the guest agent on the guest
  instance

- the sense is that we want to lose the backup and restore capability
  as we have it today. We don't necessarily need the swift client to
  be on the guest, we could directly hit the REST endpoint.

- we need to understand better how we will deal with the tracking of
  instance/database state. with the guest agent, there is a heartbeat
  from the guest which the conductor logs into the database. how do we
  want to do this with the superconductor.

- this has implications, for example, with the performance of trove
  list. It would have to go and physically poll every instance if we
  don't cache state in a database.

- the consensus was that we'd write a spec for this and move this
  forward. currently this will be done by Nikhil and Flavio.



Improve guestagent datastore models

etherpad: trove-newton-summit-refactor-user-db-extensions
bug: 1498573

- better to fail fast and add checking which is now in the guestagent
  into the API

- no need for a blue print, sync up with petr's changes

- if you ask in the API call to create a database and a user, if the
  first succeeds and the second fails, cleanup properly

- agreed to move forward with this as a bug fix


Cluster and replication locality

etherpad: trove-newton-summit-locality
spec: https://review.openstack.org/#/c/298994/
code: https://review.openstack.org/#/c/301936/
code: https://review.openstack.org/#/c/301342/
code: https://review.openstack.org/#/c/301342/
code: https://review.openstack.org/#/c/301342/

- currently we don't store the server group associated with a cluster
  or a replica set; when we need to determine which server group an
  instance is part of, we enumerate all server groups and search for
  the instance. this seems less than awesome. peterstac to chat with
  Nova about it. Also check with Sahara, they may have the same issue
  and may have solved it already. [Jack Lauritsen]

- do we default the affinity or anti-affinty? No. If we do, we need to
  expose it to horizon.


What do we do about heat?

etherpad: trove-newton-summit-is-the-heat-on

- [amrith] push a review to do this and we can debate it there.


Graduating datastores

etherpad: trove-newton-summit-graduating-datastores

PROPOSAL: Revert all datastores into a common directory, not having
the experimental and release preview in the path names, and instead
document clearly the level of testing and support in the form of a
readme with each datastore. We could optionally add a LOG.info()
message with each datastore manager and point people to the document.

- if anything, early in O to reduce impact

- peterstac to continue to work on refactoring the scenario tests.


Release calendar

etherpad: none
review: https://review.openstack.org/#/c/311123/2

- found some errors in patch set 1, made patch set 2.

- trove-dashboard freeze related question.

- trove client libraries freeze is set to R-6 which is one week before
  the hard freeze. this gives us a little time to fix any last minute
  snafu's.


Couchbase clusters
PostgreSQL backup and restore
PostgreSQL replication

- for all three, we need to get these reviews prioritized and moved
  forward.


Upgrade Trove RPC API Version [vkmc]

- currently no spec for review

- the proposed spec will address revisioning all of the internal API's


Upgrade from using Keystone v2 API to v3 API

potentially telles


Service tenant neutron port mapping

- spec from Dale (Catalyst IT)


Switch redstack to use neutron by default

- peterstac push up a change



Final summary of projects that we discussed
-------------------------------------------

python3: Victor and Abhishek
- Newton deliverable

multiple datastores with the same manager:
- no action

management client and openstack client:
- no action

Trove upgrades: morgan
- Intending that the feature will be delivered in newton

Extending Trove's storage support: amrith
- Update the spec
- Provide some initial prototype

Trove container support: flavio
- Coming up with a revised spec, ansible playbook,
- No code changes anticipated

Snapshots for cinder backup: Telles
- Delivered feature

guest images: pete, victoria
- Update spec and provide details
- Migrate dib elements to new repo, this is the newton priority
- In parallel do work on libguestfs, trove community has not
  specifically agreed to move to this tool, the sahara project has

trove v2 api   investigation: morgan
- More details

improve code modularity between guest and image: amrith
- More details

self signed certificates: amrith
- Spec and code if time permits

superconductor: nikhil, pete, flavio, amrith
- Finalize the Spec for newton
- Stretch goal is delivery of the feature

refactor user/db extensions: matt
- Code is available for review

locality support clusters and replication: peter
- Code and spec are ready for review

couchbase clusters: petr
- Code for review

postgresql b&r: petr
- Code for review

postgresql replication: petr
- Code for review

Heat:  - amrith
- Propose change set removing heat support in Trove

- to be clear, this relates to Trove's use of heat for provisioning
  instances, not the ability to deploy and orchestrate Trove from
  heat.

Graduate datastores, rearrange in tree:
- Nothing for now

scenario tests refactoring and making voting:
- Work on CI/Scenario tests for multiple databases into non-voting at
least

versioning of oslo.messaging API's: vkmc
- Spec for sure
- Code as well

Keystone API v3: telles
- Investigate and put up a spec
- Code is not (at this time) a goal

Service tenant neutron port mapping
- Spec from Dale Dalees


amrith at amrith-work:~$


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 966 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160429/851b8c28/attachment.pgp>