Open Stack

On Wed, Sep 22 2021 at 05:09:17 PM +0200, Rodolfo Alonso Hernandez 
<ralonsoh at redhat.com> wrote:
> Hello Balazs:
> 
> About the group_id, I see this is built using the port_id and the 
> qos_rules. We have all of this in the DB and we can build it 
> statically (I think so, maybe I'm very optimistic).

Yes, that is probably doable. But I let Przemek work out the details in 
the patch.
> 
> About the code, that was something I was thinking about after sending 
> the last mail. For at least two releases, we need to support both RP 
> formats in the DB. If we read only the UUID (old format), then we 
> should convert it and store it in the new format.
> 
> About the migration, we don't support contract migrations anymore. 
> But this is not true as we have done some migrations that have added 
> new restrictions in the DB schema. In any case, this could be done as 
> an expansion migration. If the code is in place, I don't see any 
> problem of doing this migration with the server running. Each 
> "ml2_port_bindings" register will be updated atomically, while the 
> Neutron server will be able to handle both versions.
> 

If I understand correctly what you described is an online data 
migration where
1) neutron does not even need an expand migration as no new field is 
needed in the database as we use the old field both for the old and the 
new data
2) neutron server converts between data formats when the data is read
3) neutron can drop the conversion code only after every register is 
upgraded this way. As there could be ports that are not touched between 
upgrades we cannot simply say that we are done with the migration after 
waiting a release cycle. I think we have to add an upgrade check in 
Yoga that warns if there are still ports with the old format. And also 
neutron needs to provide a tool for the deployer to trigger the 
conversion of those remaining port before the upgrade to X.

Do I understand your suggestion correct? Do you agree with the above 
solution proposal?

Cheers,
gibi

> Regards.
> 
> 
> On Wed, Sep 22, 2021 at 3:44 PM Balazs Gibizer 
> <balazs.gibizer at est.tech> wrote:
>> 
>> 
>>  On Tue, Sep 21 2021 at 06:30:46 PM +0200, Rodolfo Alonso Hernandez
>>  <ralonsoh at redhat.com> wrote:
>>  > Hello Balazs:
>>  >
>>  > Sorry for the late reply, I was on PTO.
>>  >
>>  > If I'm not wrong, now port['binding:profile']['allocation'] is a 
>> UUID
>>  > and you need it to be a list of UUIDs. Am I correct?
>> 
>>  It is a bit more complicated than that. The old value is a single RP
>>  UUID. the new value is a dict where the key is the group_id and the
>>  value is the RP UUID fulfilling that group. So the transformation 
>> needs
>>  to access to the group_id.
>>  The group_id is a stable UUID generated by neutron server as part of
>>  the port.resource_request value, but it is not persisted.
>> 
>>  >
>>  > To make this change in the DB you should use the Alembic 
>> migrations,
>>  > as you said. That should ensure all registers are translated. We
>>  > should also include a sanity check to ensure the DB migration was
>>  > done correctly.
>> 
>>  I'm not 100% sure but I think such data migration can only be done 
>> in
>>  the contract part as it needs to be done while the neutron server is
>>  down as the old code can only use the old data format while the new
>>  code can only use the new format. Is it OK to introduce a new 
>> contract
>>  migration in Yoga in neutron?
>> 
>>  Cheers,
>>  gibi
>> 
>> 
>>  >
>>  > Is that what you needed? Don't hesitate to ping me in IRC if 
>> needed.
>>  >
>>  > Regards.
>>  >
>>  > On Fri, Sep 10, 2021 at 6:06 PM Balazs Gibizer
>>  > <balazs.gibizer at est.tech> wrote:
>>  >> Hi Neutrinos!
>>  >>
>>  >>  We found a technical challenge during implementing the
>>  >>  port-resource-request-groups API extension[1]. That extension
>>  >> changes
>>  >>  the format of the port.resoruce_request as well as the format 
>> of the
>>  >>  port.binding:profile.allocation. The former is a calculated 
>> field on
>>  >>  the port so that is easy. However the bindig:profile is 
>> persisted in
>>  >>  the database so data migration is needed. What is the canonical 
>> way
>>  >> to
>>  >>  do such DB data translation in Neutron? Can we translate the 
>> data in
>>  >>  place during alembic migration? Or should we do some kind of 
>> online
>>  >>  data migration when the data is translated by neutron when it is
>>  >> read
>>  >>  from the db?
>>  >>
>>  >>  cheers,
>>  >>  gibi
>>  >>
>>  >>  [1] https://review.opendev.org/c/openstack/neutron/+/805637/5
>>  >>
>>  >>
>>  >>
>> 
>>