Open Stack

On 06/03/2015 09:35 AM, Rodrigo Barbieri wrote:
> Hello guys,
>
> I would like to bring everyone up to speed on this topic, since we 
> have a weekly meeting tomorrow and I would like to further discuss 
> this, either here or tomorrow at the meeting, since this is something 
> that is a pre-requisite for future features planned for liberty.
>
> We had a discussion on IRC last week about possible improvements to 
> Share Migration concerning the IDs and additional temporary DB row. So 
> far, our conclusion has been that we prefer to have the additional DB 
> row, but we must deal with the fact that current architecture does not 
> expect a Share to have two separate IDs, the "API ID" and the "Driver 
> ID". We have came up with several ways to improve this, and we would 
> like to continue the discussion and decide how we can better improve 
> it thinking about the future features such as Replication and AZ.
>
> Current scenario (as of prototype):
> - Migration creates a totally new share in destination backend, copy 
> data, copy new DB values (such as destination export location) to the 
> original DB entry, and then deletes the new DB entry, and the source 
> physical share. The result is the original DB entry with the new DB 
> values (such as destination export location). In this prototype, the 
> export location is being used as "Driver ID", because it is derived 
> from the "API ID". After migration, the migrated Share has "API ID" X 
> and export location Y, because Y was derived from the temporary DB row 
> created for the destination share.
>
> Proposal 1: Use Private Driver Storage to store "Driver ID". This will 
> require all drivers to follow the guideline as implemented in the 
> generic driver, which manages the volume ID ("Driver ID" for this 
> driver) separate from the "API ID".
>
> Proposal 2: Use additional DB column so we have separate IDs in each 
> column. This will require less effort from drivers, because this 
> column value can be transferred from the temporary DB row to the 
> original DB entry, similar to what is done with the export location 
> column in the prototype. Drivers can manage the value in this column 
> if they want, but if they do not, we can derive from the API ID if we 
> continue to use the approach currently implemented for Share 
> Migration, and keep in mind that for replication or other features, we 
> need to fill this field with a value as if we are creating a new 
> share. This approach also has the disadvantage of being confusing for 
> debugging and require more changes in Manila Core code, but at least 
> this is handled by Manila Core code instead of Driver code.
>
> Additionally, proposal 1 can be mixed with proposal 2, if the Manila 
> Core code attempts to store the "Driver ID" value in Private Share 
> Data instead of column, but we argued that Manila Core should not 
> touch Private Share Data, we have not come to a conclusion on this.
>
> Proposal 3: Create new table "instances" that will be linked to the 
> "API ID", so a share can have several instances, which have their own 
> ID, and only one is considered "Active". This approach sounds very 
> interesting for future features, the admin can find the ID for which 
> instances are in the backend through a "manila share-instances-show 
> <share_id>" command. There has been a lot of discussion regarding how 
> we use the Instance ID, if we provide them directly to drivers as if 
> it was the API ID, or include in a field in the Share object so the 
> driver can continue to use the API ID and reads the Instance ID if it 
> wants (which makes it similar to proposal 1). It was stated that for 
> replication, drivers need to see the instance IDs, so providing the 
> Instance ID as if it was the API ID would not make much sense here. 
> This approach will also require a lot of changes on Manila Core code, 
> and depending on what we decide to do with the Instance ID regarding 
> drivers, may require no changes or minimal changes to drivers.
>
> Proposal 4: Allow drivers to change the "API ID". The 
> advantages/disadvanges of this proposal are not very clear to me, it 
> would fix part of Share Migration problem (not sure if replication 
> would need to do the same), but I see as it breaking the concept that 
> we were trying to migrate a share, it becomes cloning shares and 
> erasing the original, we do not know how it would impact users, and 
> certainly would be much less transparent.
>
> I think that from here we can proceed on expressing our concerns, 
> disadvantages or advantages or each approach, for other features as 
> well (Unfortunately I am familiar with migration only), develop each 
> proposal further with diagrams if that's the case, so we can decide on 
> which one is best for our future features.

I see 2 possible paths forward on the share ID problem for migrations. 
It's not clear that the 2 options I see neatly map to your proposal 
numbers so I'll call them A and B to avoid confusion.

Option A: Shares continue to have 1 ID, but we allow a share's ID to 
change in special cases.
I believe this option is most similar to proposals 1 and 4. The main 
idea is that if we want to avoid introducing separate user-facing and 
driver-facing IDs, and we want to avoid adding special logic to drivers 
to create shares that will be the destination of a migration, then the 
only choice is to create a temporary share for the destination of the 
migration, and to change that ID when the migration is complete, so the 
user doesn't observe any ID change. The advantage to this approach is 
that the database schema wouldn't really need to change much at all, and 
we could avoid making a bunch of changes in the Manila core to deal with 
multiple IDs. The disadvantage is that every driver would need to 
implement a new change_share_id() method which would alter the ID of an 
existing share, either by modifying some objects on the actual storage 
controller (if possible) or by changing some mapping information stored 
in the Manila DB (using driver private share data) to track the share by 
a new ID. Forcing every driver to implement a new feature like in order 
to support migration would create a very high risk that Liberty would 
ship without universal support for migrations.

Option B: Create share instances, allowing a multiple shares to appear 
as one to the user
This option is more similar to proposals 2 and 3. It would involve 
splitting the shares table into 2 database tables (one for shares, one 
for share instances) and giving every share 2 IDs (one for API clients, 
and one for drivers). This could be done in such a way that drivers 
didn't need to change at all, and drivers would simply operate on share 
instances without realizing that multiple share instances might actually 
be presented to the API client as a single share. A feature like this 
would completely solve the share-ID related challenges for migration. It 
could have other advantages too. The downside to this approach is that 
it would change a lot of the core code in Manila around shares and the 
database and the possibility for bugs to creep in would be significant. 
Also a large change like this would make it hard to backport bugfixes to 
earlier releases.

I'm in favor of option B currently because I feel the risks of migration 
not working universally outweigh the risks of the code changes required 
to implement share instances. I'm also convinced that share instances 
can benefit other proposed features, such as share replication. I can 
update the design of replication to work with share-instances and I 
believe the resulting design will be easier to implement rather than harder.

-Ben


>
> -- 
> Rodrigo Barbieri
> Computer Scientist
> Federal University of São Carlos
> (11) 96889 3412 <tel:%2811%29%2096889%203412>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150617/1bb0b1a5/attachment.html>