Hi, I'd like to discuss how we can have DB upgrade migration method (with data manipulation) in placement. --- BackGround ========== https://bugs.launchpad.net/nova/+bug/1803925 * In Rocky, to have nested resource provider feature, we expanded the DB to have root provider id column. * The root provider id shouldn't be None and for root providers it should be the same value of its resource provider id. * In Rocky, the code is build in a backward compatible way doing online migration. * For each request of listing/showing resource providers, we look the root provider id and if it is stale and empty, we assume the resource provider is a root and set the same value as resource provider id. * Those providers that are not called in the Rocky cycle will still have an empty value for the root provider id. * In Stein or later, we want a way to be sure that all the root provider id contains some non-None value. * This is because we have a lot of TODOs in code which we want to clean up once we are sure all the root provider ids have non-None value in the DB. * In Stein, we are already ready use alembic to manage DB schema changes in placement. Question ======== How should we copy the resource provider id to root provider id if the root provider id is None? Options ======= 1. Do it in the alembic script in the same way as the schema expansion * This is done in the https://review.openstack.org/#/c/619126/ and brought several concerns. * We don't want the data manipulation migration to be inter-mixed with schema changes. * For cases skipping one release in an FFU fashion, there would be a lot of rows to be changed. 2. Have two stable version branches for alembic to separate schema changes and data manipulation * Neutron has been managing two branches in alembic already. * https://specs.openstack.org/openstack/neutron-specs/specs/liberty/online-sch... * Developers would specify on which branch to have a new version via some a new CLI something like; `placement-db-manage revision -m "description of revision" (--schema-change|--data-manipulate)` * We should be careful for the dependency management across the two branches. * https://alembic.sqlalchemy.org/en/latest/branches.html 3. Bring the online-migration batch CLI command * Follow the traditional way to manage DB data in Nova * `placement-manage db online-data-migration [--max-count]` * I'm looking into this in https://review.openstack.org/#/c/624942/. 4. Any other ideas? --- Since it looks like it's going to have an impact both on operators who upgrade and developers who add new features in placement, it is nice to seek more ideas and to have a consensus before we go further. Thanks, -- Tetsuro Nakamura <nakamura.tetsuro@lab.ntt.co.jp> NTT Network Service Systems Laboratories TEL:0422 59 6914(National)/+81 422 59 6914(International) 3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan