[Nova] Suggestion needed for detach-boot-volume design
Hi Nova, Happy New Year! I've been working on detach-boot-volume[1] in Stein, we got the initial design merged and while implementing we have meet some new problems and now I'm amending the spec to cover these new problems[2]. The thing I want to discuss for wider opinion is that in the initial design, we planned to support detach root volume for only STOPPED and SHELVED/SHELVE_OFFLOADED instances. But then we found out that we allowed to detach volumes for RESIZED/PAUSED/SOFT_DELETED instances as well. Should we allow detaching root volume for instances in these status too? Cases like RESIZE could be complicated for the revert resize action, and it also seems unnecesary. Thoughts? BR, [1] https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/volume... [2] https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/volume... Kevin Zheng
On 1/2/2019 2:57 AM, Zhenyu Zheng wrote:
I've been working on detach-boot-volume[1] in Stein, we got the initial design merged and while implementing we have meet some new problems and now I'm amending the spec to cover these new problems[2].
[2] is https://review.openstack.org/#/c/619161/
The thing I want to discuss for wider opinion is that in the initial design, we planned to support detach root volume for only STOPPED and SHELVED/SHELVE_OFFLOADED instances. But then we found out that we allowed to detach volumes for RESIZED/PAUSED/SOFT_DELETED instances as well. Should we allow detaching root volume for instances in these status too? Cases like RESIZE could be complicated for the revert resize action, and it also seems unnecesary.
The full set of allowed states for attaching and detaching are here: https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4187 https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4297 Concerning those other states: RESIZED: There might be a case for attaching/detaching volumes based on flavor during a resize, but I'm not sure about the root volume in that case (that really sounds more like rebuild with a new image to me, which is a different blueprint). I'm also not sure how much people know about the ability to do this or what the behavior is on revert if you have changed the volumes while the server is resized. If we consider that when a user reverts a resize, they want to go back to the way things were for the root disk image, then I would think we should not allow changing out the root volume while resized. PAUSED: First, I'm not sure how much anyone uses the pause API (or suspend for that matter) although most of the virt drivers implement it. At one point you could attach volumes to suspended servers as well, but because libvirt didn't support it that was removed from the API (yay for non-discoverable backend-specific API behavior changes): https://review.openstack.org/#/c/83505/ Anyway, swapping the root volume on a paused instance seems dangerous to me, so until someone really has a good use case for it, then I think we should avoid that one as well. SOFT_DELETED: I really don't understand the use case for attaching/detaching volumes to/from a (soft) deleted server. If the server is deleted and only hanging around because it hasn't been reclaimed yet, there are really no guarantees that this would work, so again, I would just skip this one for the root volume changes. If the user really wants to play with the volumes attached to a soft deleted server, they should restore it first. So in summary, I think we should just not support any of those other states for attach/detach root volumes and only focus on stopped or shelved instances. -- Thanks, Matt
On Fri, 4 Jan 2019 09:50:46 -0600, Matt Riedemann <mriedemos@gmail.com> wrote:
On 1/2/2019 2:57 AM, Zhenyu Zheng wrote:
I've been working on detach-boot-volume[1] in Stein, we got the initial design merged and while implementing we have meet some new problems and now I'm amending the spec to cover these new problems[2].
[2] is https://review.openstack.org/#/c/619161/
The thing I want to discuss for wider opinion is that in the initial design, we planned to support detach root volume for only STOPPED and SHELVED/SHELVE_OFFLOADED instances. But then we found out that we allowed to detach volumes for RESIZED/PAUSED/SOFT_DELETED instances as well. Should we allow detaching root volume for instances in these status too? Cases like RESIZE could be complicated for the revert resize action, and it also seems unnecesary.
The full set of allowed states for attaching and detaching are here:
https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4187
https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4297
Concerning those other states:
RESIZED: There might be a case for attaching/detaching volumes based on flavor during a resize, but I'm not sure about the root volume in that case (that really sounds more like rebuild with a new image to me, which is a different blueprint). I'm also not sure how much people know about the ability to do this or what the behavior is on revert if you have changed the volumes while the server is resized. If we consider that when a user reverts a resize, they want to go back to the way things were for the root disk image, then I would think we should not allow changing out the root volume while resized.
Yeah, if someone attaches/detaches a regular volume while the instance is in VERIFY_RESIZE state and then reverts the resize, I assume we probably don't attempt to change or restore anything with the volume attachments to put them back to how they were attached before the resize. But as you point out, the situation does seem different regarding a root volume. If a user changes that while in VERIFY_RESIZE and reverts the resize, and we leave the root volume alone, then they end up with a different root disk image than they had before the resize. Which seems weird. I agree it seems better not to allow this for now and come back to it later if people start asking for it.
PAUSED: First, I'm not sure how much anyone uses the pause API (or suspend for that matter) although most of the virt drivers implement it. At one point you could attach volumes to suspended servers as well, but because libvirt didn't support it that was removed from the API (yay for non-discoverable backend-specific API behavior changes):
https://review.openstack.org/#/c/83505/
Anyway, swapping the root volume on a paused instance seems dangerous to me, so until someone really has a good use case for it, then I think we should avoid that one as well.
SOFT_DELETED: I really don't understand the use case for attaching/detaching volumes to/from a (soft) deleted server. If the server is deleted and only hanging around because it hasn't been reclaimed yet, there are really no guarantees that this would work, so again, I would just skip this one for the root volume changes. If the user really wants to play with the volumes attached to a soft deleted server, they should restore it first.
So in summary, I think we should just not support any of those other states for attach/detach root volumes and only focus on stopped or shelved instances.
Again, agree, I think we should just not allow the other states for the initial implementation and revisit later if it turns out people need these. -melanie
Thanks alot for the replies, lets wait for some more comments, and I will update the follow-up spec about this within two days. On Sat, Jan 5, 2019 at 7:37 AM melanie witt <melwittt@gmail.com> wrote:
On Fri, 4 Jan 2019 09:50:46 -0600, Matt Riedemann <mriedemos@gmail.com> wrote:
On 1/2/2019 2:57 AM, Zhenyu Zheng wrote:
I've been working on detach-boot-volume[1] in Stein, we got the initial design merged and while implementing we have meet some new problems and now I'm amending the spec to cover these new problems[2].
[2] is https://review.openstack.org/#/c/619161/
The thing I want to discuss for wider opinion is that in the initial design, we planned to support detach root volume for only STOPPED and SHELVED/SHELVE_OFFLOADED instances. But then we found out that we allowed to detach volumes for RESIZED/PAUSED/SOFT_DELETED instances as well. Should we allow detaching root volume for instances in these status too? Cases like RESIZE could be complicated for the revert resize action, and it also seems unnecesary.
The full set of allowed states for attaching and detaching are here:
https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4187
https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4297
Concerning those other states:
RESIZED: There might be a case for attaching/detaching volumes based on flavor during a resize, but I'm not sure about the root volume in that case (that really sounds more like rebuild with a new image to me, which is a different blueprint). I'm also not sure how much people know about the ability to do this or what the behavior is on revert if you have changed the volumes while the server is resized. If we consider that when a user reverts a resize, they want to go back to the way things were for the root disk image, then I would think we should not allow changing out the root volume while resized.
Yeah, if someone attaches/detaches a regular volume while the instance is in VERIFY_RESIZE state and then reverts the resize, I assume we probably don't attempt to change or restore anything with the volume attachments to put them back to how they were attached before the resize. But as you point out, the situation does seem different regarding a root volume. If a user changes that while in VERIFY_RESIZE and reverts the resize, and we leave the root volume alone, then they end up with a different root disk image than they had before the resize. Which seems weird.
I agree it seems better not to allow this for now and come back to it later if people start asking for it.
PAUSED: First, I'm not sure how much anyone uses the pause API (or suspend for that matter) although most of the virt drivers implement it. At one point you could attach volumes to suspended servers as well, but because libvirt didn't support it that was removed from the API (yay for non-discoverable backend-specific API behavior changes):
https://review.openstack.org/#/c/83505/
Anyway, swapping the root volume on a paused instance seems dangerous to me, so until someone really has a good use case for it, then I think we should avoid that one as well.
SOFT_DELETED: I really don't understand the use case for attaching/detaching volumes to/from a (soft) deleted server. If the server is deleted and only hanging around because it hasn't been reclaimed yet, there are really no guarantees that this would work, so again, I would just skip this one for the root volume changes. If the user really wants to play with the volumes attached to a soft deleted server, they should restore it first.
So in summary, I think we should just not support any of those other states for attach/detach root volumes and only focus on stopped or shelved instances.
Again, agree, I think we should just not allow the other states for the initial implementation and revisit later if it turns out people need these.
-melanie
Thanks all for the feedback, I have update the spec to be more clear about the scope: https://review.openstack.org/#/c/619161/ On Mon, Jan 7, 2019 at 4:37 PM Zhenyu Zheng <zhengzhenyulixi@gmail.com> wrote:
Thanks alot for the replies, lets wait for some more comments, and I will update the follow-up spec about this within two days.
On Sat, Jan 5, 2019 at 7:37 AM melanie witt <melwittt@gmail.com> wrote:
On Fri, 4 Jan 2019 09:50:46 -0600, Matt Riedemann <mriedemos@gmail.com> wrote:
On 1/2/2019 2:57 AM, Zhenyu Zheng wrote:
I've been working on detach-boot-volume[1] in Stein, we got the initial design merged and while implementing we have meet some new problems and now I'm amending the spec to cover these new problems[2].
[2] is https://review.openstack.org/#/c/619161/
The thing I want to discuss for wider opinion is that in the initial design, we planned to support detach root volume for only STOPPED and SHELVED/SHELVE_OFFLOADED instances. But then we found out that we allowed to detach volumes for RESIZED/PAUSED/SOFT_DELETED instances as well. Should we allow detaching root volume for instances in these status too? Cases like RESIZE could be complicated for the revert
resize
action, and it also seems unnecesary.
The full set of allowed states for attaching and detaching are here:
https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4187
https://github.com/openstack/nova/blob/8ef3d253a/nova/compute/api.py#L4297
Concerning those other states:
RESIZED: There might be a case for attaching/detaching volumes based on flavor during a resize, but I'm not sure about the root volume in that case (that really sounds more like rebuild with a new image to me, which is a different blueprint). I'm also not sure how much people know about the ability to do this or what the behavior is on revert if you have changed the volumes while the server is resized. If we consider that when a user reverts a resize, they want to go back to the way things were for the root disk image, then I would think we should not allow changing out the root volume while resized.
Yeah, if someone attaches/detaches a regular volume while the instance is in VERIFY_RESIZE state and then reverts the resize, I assume we probably don't attempt to change or restore anything with the volume attachments to put them back to how they were attached before the resize. But as you point out, the situation does seem different regarding a root volume. If a user changes that while in VERIFY_RESIZE and reverts the resize, and we leave the root volume alone, then they end up with a different root disk image than they had before the resize. Which seems weird.
I agree it seems better not to allow this for now and come back to it later if people start asking for it.
PAUSED: First, I'm not sure how much anyone uses the pause API (or suspend for that matter) although most of the virt drivers implement it. At one point you could attach volumes to suspended servers as well, but because libvirt didn't support it that was removed from the API (yay for non-discoverable backend-specific API behavior changes):
https://review.openstack.org/#/c/83505/
Anyway, swapping the root volume on a paused instance seems dangerous to me, so until someone really has a good use case for it, then I think we should avoid that one as well.
SOFT_DELETED: I really don't understand the use case for attaching/detaching volumes to/from a (soft) deleted server. If the server is deleted and only hanging around because it hasn't been reclaimed yet, there are really no guarantees that this would work, so again, I would just skip this one for the root volume changes. If the user really wants to play with the volumes attached to a soft deleted server, they should restore it first.
So in summary, I think we should just not support any of those other states for attach/detach root volumes and only focus on stopped or shelved instances.
Again, agree, I think we should just not allow the other states for the initial implementation and revisit later if it turns out people need these.
-melanie
participants (3)
-
Matt Riedemann
-
melanie witt
-
Zhenyu Zheng