Open Stack

Hello Balazs:

You are right: a DB migration is not necessary but a DB sanitization. I did
something similar in https://review.opendev.org/c/openstack/neutron/+/789831.
This patch includes the script to, in one shot, modify the whole DB and an
upgrade check to test it.

About the code, if something like the script provided and the upgrade check
is included, that will ensure the DB is correctly migrated in Y release.
That means the code supporting both formats could be removed in Z.

Regards.


On Wed, Sep 22, 2021 at 6:54 PM Balazs Gibizer <balazs.gibizer at est.tech>
wrote:

>
>
> On Wed, Sep 22 2021 at 05:09:17 PM +0200, Rodolfo Alonso Hernandez
> <ralonsoh at redhat.com> wrote:
> > Hello Balazs:
> >
> > About the group_id, I see this is built using the port_id and the
> > qos_rules. We have all of this in the DB and we can build it
> > statically (I think so, maybe I'm very optimistic).
>
> Yes, that is probably doable. But I let Przemek work out the details in
> the patch.
> >
> > About the code, that was something I was thinking about after sending
> > the last mail. For at least two releases, we need to support both RP
> > formats in the DB. If we read only the UUID (old format), then we
> > should convert it and store it in the new format.
> >
> > About the migration, we don't support contract migrations anymore.
> > But this is not true as we have done some migrations that have added
> > new restrictions in the DB schema. In any case, this could be done as
> > an expansion migration. If the code is in place, I don't see any
> > problem of doing this migration with the server running. Each
> > "ml2_port_bindings" register will be updated atomically, while the
> > Neutron server will be able to handle both versions.
> >
>
> If I understand correctly what you described is an online data
> migration where
> 1) neutron does not even need an expand migration as no new field is
> needed in the database as we use the old field both for the old and the
> new data
> 2) neutron server converts between data formats when the data is read
> 3) neutron can drop the conversion code only after every register is
> upgraded this way. As there could be ports that are not touched between
> upgrades we cannot simply say that we are done with the migration after
> waiting a release cycle. I think we have to add an upgrade check in
> Yoga that warns if there are still ports with the old format. And also
> neutron needs to provide a tool for the deployer to trigger the
> conversion of those remaining port before the upgrade to X.
>
> Do I understand your suggestion correct? Do you agree with the above
> solution proposal?
>
> Cheers,
> gibi
>
> > Regards.
> >
> >
> > On Wed, Sep 22, 2021 at 3:44 PM Balazs Gibizer
> > <balazs.gibizer at est.tech> wrote:
> >>
> >>
> >>  On Tue, Sep 21 2021 at 06:30:46 PM +0200, Rodolfo Alonso Hernandez
> >>  <ralonsoh at redhat.com> wrote:
> >>  > Hello Balazs:
> >>  >
> >>  > Sorry for the late reply, I was on PTO.
> >>  >
> >>  > If I'm not wrong, now port['binding:profile']['allocation'] is a
> >> UUID
> >>  > and you need it to be a list of UUIDs. Am I correct?
> >>
> >>  It is a bit more complicated than that. The old value is a single RP
> >>  UUID. the new value is a dict where the key is the group_id and the
> >>  value is the RP UUID fulfilling that group. So the transformation
> >> needs
> >>  to access to the group_id.
> >>  The group_id is a stable UUID generated by neutron server as part of
> >>  the port.resource_request value, but it is not persisted.
> >>
> >>  >
> >>  > To make this change in the DB you should use the Alembic
> >> migrations,
> >>  > as you said. That should ensure all registers are translated. We
> >>  > should also include a sanity check to ensure the DB migration was
> >>  > done correctly.
> >>
> >>  I'm not 100% sure but I think such data migration can only be done
> >> in
> >>  the contract part as it needs to be done while the neutron server is
> >>  down as the old code can only use the old data format while the new
> >>  code can only use the new format. Is it OK to introduce a new
> >> contract
> >>  migration in Yoga in neutron?
> >>
> >>  Cheers,
> >>  gibi
> >>
> >>
> >>  >
> >>  > Is that what you needed? Don't hesitate to ping me in IRC if
> >> needed.
> >>  >
> >>  > Regards.
> >>  >
> >>  > On Fri, Sep 10, 2021 at 6:06 PM Balazs Gibizer
> >>  > <balazs.gibizer at est.tech> wrote:
> >>  >> Hi Neutrinos!
> >>  >>
> >>  >>  We found a technical challenge during implementing the
> >>  >>  port-resource-request-groups API extension[1]. That extension
> >>  >> changes
> >>  >>  the format of the port.resoruce_request as well as the format
> >> of the
> >>  >>  port.binding:profile.allocation. The former is a calculated
> >> field on
> >>  >>  the port so that is easy. However the bindig:profile is
> >> persisted in
> >>  >>  the database so data migration is needed. What is the canonical
> >> way
> >>  >> to
> >>  >>  do such DB data translation in Neutron? Can we translate the
> >> data in
> >>  >>  place during alembic migration? Or should we do some kind of
> >> online
> >>  >>  data migration when the data is translated by neutron when it is
> >>  >> read
> >>  >>  from the db?
> >>  >>
> >>  >>  cheers,
> >>  >>  gibi
> >>  >>
> >>  >>  [1] https://review.opendev.org/c/openstack/neutron/+/805637/5
> >>  >>
> >>  >>
> >>  >>
> >>
> >>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210923/ab52dac9/attachment-0001.htm>