<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<br>
<div class="moz-cite-prefix">On 06/03/2015 09:35 AM, Rodrigo
Barbieri wrote:<br>
</div>
<blockquote
cite="mid:CADMdNi+GgO0w5OJDHonEqGmoS3OzQFJe=+o6F4MW+oju7HmZ8g@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>Hello guys,<br>
<br>
</div>
<div>I would like to bring everyone up to speed on this
topic, since we have a weekly meeting tomorrow and I would
like to further discuss this, either here or tomorrow at
the meeting, since this is something that is a
pre-requisite for future features planned for liberty.</div>
<div><br>
</div>
We had a discussion on IRC last week about possible
improvements to Share Migration concerning the IDs and
additional temporary DB row. So far, our conclusion has been
that we prefer to have the additional DB row, but we must
deal with the fact that current architecture does not expect
a Share to have two separate IDs, the "API ID" and the
"Driver ID". We have came up with several ways to improve
this, and we would like to continue the discussion and
decide how we can better improve it thinking about the
future features such as Replication and AZ.<br>
<br>
</div>
Current scenario (as of prototype): <br>
</div>
- Migration creates a totally new share in destination backend,
copy data, copy new DB values (such as destination export
location) to the original DB entry, and then deletes the new DB
entry, and the source physical share. The result is the original
DB entry with the new DB values (such as destination export
location). In this prototype, the export location is being used
as "Driver ID", because it is derived from the "API ID". After
migration, the migrated Share has "API ID" X and export location
Y, because Y was derived from the temporary DB row created for
the destination share.<br>
<br>
Proposal 1: Use Private Driver Storage to store "Driver ID".
This will require all drivers to follow the guideline as
implemented in the generic driver, which manages the volume ID
("Driver ID" for this driver) separate from the "API ID".
<div><br>
</div>
<div>Proposal 2: Use additional DB column so we have separate
IDs in each column. This will require less effort from
drivers, because this column value can be transferred from the
temporary DB row to the original DB entry, similar to what is
done with the export location column in the prototype. Drivers
can manage the value in this column if they want, but if they
do not, we can derive from the API ID if we continue to use
the approach currently implemented for Share Migration, and
keep in mind that for replication or other features, we need
to fill this field with a value as if we are creating a new
share. This approach also has the disadvantage of being
confusing for debugging and require more changes in Manila
Core code, but at least this is handled by Manila Core code
instead of Driver code.</div>
<div><br>
</div>
<div>Additionally, proposal 1 can be mixed with proposal 2, if
the Manila Core code attempts to store the "Driver ID" value
in Private Share Data instead of column, but we argued that
Manila Core should not touch Private Share Data, we have not
come to a conclusion on this. </div>
<div><br>
</div>
<div>Proposal 3: Create new table "instances" that will be
linked to the "API ID", so a share can have several instances,
which have their own ID, and only one is considered "Active".
This approach sounds very interesting for future features, the
admin can find the ID for which instances are in the backend
through a "manila share-instances-show <share_id>"
command. There has been a lot of discussion regarding how we
use the Instance ID, if we provide them directly to drivers as
if it was the API ID, or include in a field in the Share
object so the driver can continue to use the API ID and reads
the Instance ID if it wants (which makes it similar to
proposal 1). It was stated that for replication, drivers need
to see the instance IDs, so providing the Instance ID as if it
was the API ID would not make much sense here. This approach
will also require a lot of changes on Manila Core code, and
depending on what we decide to do with the Instance ID
regarding drivers, may require no changes or minimal changes
to drivers. </div>
<div><br>
</div>
<div>Proposal 4: Allow drivers to change the "API ID". The
advantages/disadvanges of this proposal are not very clear to
me, it would fix part of Share Migration problem (not sure if
replication would need to do the same), but I see as it
breaking the concept that we were trying to migrate a share,
it becomes cloning shares and erasing the original, we do not
know how it would impact users, and certainly would be much
less transparent. <br>
<br clear="all">
<div>
<div>I think that from here we can proceed on expressing our
concerns, disadvantages or advantages or each approach,
for other features as well (Unfortunately I am familiar
with migration only), develop each proposal further with
diagrams if that's the case, so we can decide on which one
is best for our future features. </div>
</div>
</div>
</div>
</blockquote>
<br>
I see 2 possible paths forward on the share ID problem for
migrations. It's not clear that the 2 options I see neatly map to
your proposal numbers so I'll call them A and B to avoid confusion.<br>
<br>
Option A: Shares continue to have 1 ID, but we allow a share's ID to
change in special cases.<br>
I believe this option is most similar to proposals 1 and 4. The main
idea is that if we want to avoid introducing separate user-facing
and driver-facing IDs, and we want to avoid adding special logic to
drivers to create shares that will be the destination of a
migration, then the only choice is to create a temporary share for
the destination of the migration, and to change that ID when the
migration is complete, so the user doesn't observe any ID change.
The advantage to this approach is that the database schema wouldn't
really need to change much at all, and we could avoid making a bunch
of changes in the Manila core to deal with multiple IDs. The
disadvantage is that every driver would need to implement a new
change_share_id() method which would alter the ID of an existing
share, either by modifying some objects on the actual storage
controller (if possible) or by changing some mapping information
stored in the Manila DB (using driver private share data) to track
the share by a new ID. Forcing every driver to implement a new
feature like in order to support migration would create a very high
risk that Liberty would ship without universal support for
migrations.<br>
<br>
Option B: Create share instances, allowing a multiple shares to
appear as one to the user<br>
This option is more similar to proposals 2 and 3. It would involve
splitting the shares table into 2 database tables (one for shares,
one for share instances) and giving every share 2 IDs (one for API
clients, and one for drivers). This could be done in such a way that
drivers didn't need to change at all, and drivers would simply
operate on share instances without realizing that multiple share
instances might actually be presented to the API client as a single
share. A feature like this would completely solve the share-ID
related challenges for migration. It could have other advantages
too. The downside to this approach is that it would change a lot of
the core code in Manila around shares and the database and the
possibility for bugs to creep in would be significant. Also a large
change like this would make it hard to backport bugfixes to earlier
releases.<br>
<br>
I'm in favor of option B currently because I feel the risks of
migration not working universally outweigh the risks of the code
changes required to implement share instances. I'm also convinced
that share instances can benefit other proposed features, such as
share replication. I can update the design of replication to work
with share-instances and I believe the resulting design will be
easier to implement rather than harder.<br>
<br>
-Ben<br>
<br>
<br>
<blockquote
cite="mid:CADMdNi+GgO0w5OJDHonEqGmoS3OzQFJe=+o6F4MW+oju7HmZ8g@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div><br>
-- <br>
<div>Rodrigo Barbieri
<div>Computer Scientist</div>
<div>Federal University of São Carlos</div>
<div><a moz-do-not-send="true"
href="tel:%2811%29%2096889%203412"
value="+5511968893412" target="_blank">(11) 96889
3412</a></div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>