<div dir="ltr">I wonder if the best solution here is to change the uniqueness constraint in our tables (as has been mentioned in several other OpenStack-related database discussions) so that deleted images do not conflict with active images with the same uuid. As I recall, the best practice suggestion was to key the uniqueness constraint off the pair <uuid, deleted>. In this scheme, deleted is no longer a simple boolean, but rather either 0 (not deleted) or equal to the auto-increment row id (deleted).<div>
<br></div><div style>Does that make sense?</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 16, 2013 at 8:29 AM, Fei Long Wang <span dir="ltr"><<a href="mailto:flwang@cn.ibm.com" target="_blank">flwang@cn.ibm.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>
<p><font face="sans-serif">Hi stackers,</font><font size="3" face="sans-serif"><br>
</font><font face="sans-serif"><br>
Now I'm running into an issue about reusing image id in Glance. Based on current design, the image id can be specified during create image by end user. But now there is a problem, which is tracked by defect </font><a href="https://bugs.launchpad.net/glance/+bug/1176978" target="_blank"><font size="1" color="#0000FF" face="Arial"><u>https://bugs.launchpad.net/glance/+bug/1176978</u></font></a><font face="sans-serif">. In summary, if user want to port an image from VMware vCenter which has been used by many VMs. So when user port the image into Glance, it would be nice if the original image id with UUID format from vCenter can be used. But if the image is removed and recreated again, he will run into current issue. I have discussed it with flaper87 in Glance IRC channel, below is the chat log. FYI. And I also attached the patch, please help review it. Any comment/suggestion is welcome. Thanks.</font><font size="3" face="sans-serif"><br>
</font><font face="sans-serif"><br>
Chat log between flaper87 and me.<br>
=============================</font><font color="#16569E" face="serif"><br>
(07:46:34 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> btw, may I get your thought about this defect? </font><a href="https://bugs.launchpad.net/glance/+bug/1176978" target="_blank"><font size="3" color="#0000FF" face="serif"><u>https://bugs.launchpad.net/glance/+bug/1176978</u></font></a><font color="#16569E" face="serif"><br>
(07:47:48 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> about if the image id should be reused after the image is deleted</font><font color="#A82F2F" face="serif"><br>
(07:49:18 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> flwang: it shouldn't, images are never *truly* deleted, see </font><a href="https://github.com/openstack/glance/blob/master/glance/db/sqlalchemy/models.py#L65" target="_blank"><font size="3" color="#0000FF" face="serif"><u>https://github.com/openstack/glance/blob/master/glance/db/sqlalchemy/models.py#L65</u></font></a><font color="#16569E" face="serif"><br>
(07:49:27 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> I see</font><font color="#A82F2F" face="serif"><br>
(07:49:39 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> that's the BaseModel</font><font color="#16569E" face="serif"><br>
(07:49:43 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> I know it's just marked 1 in the deleted column</font><font color="#A82F2F" face="serif"><br>
(07:49:45 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> and Image inherits from it</font><font color="#A82F2F" face="serif"><br>
(07:49:52 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> correct</font><font color="#A82F2F" face="serif"><br>
(07:50:21 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> there were some discussions about completely delelting those images (cleaning up the db)</font><font color="#16569E" face="serif"><br>
(07:50:23 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> now there is a scenario</font><font color="#A82F2F" face="serif"><br>
(07:50:34 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> but nothing has been decided yet</font><font color="#A82F2F" face="serif"><br>
(07:51:28 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> TBH, I don't like the idea of people passing the id on the create command, but that's another story for another time</font><font color="#16569E" face="serif"><br>
(07:51:30 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> user is porting some images from vmware vcenter, and then these images are deleted</font><font color="#A82F2F" face="serif"><br>
(07:51:49 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> flwang: not sure I understand</font><font color="#A82F2F" face="serif"><br>
(07:52:00 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> flwang: you mean, import image from vmware into glance</font><font color="#16569E" face="serif"><br>
(07:52:04 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> but next, user want to add these images again with the image id given by vcenter</font><font color="#A82F2F" face="serif"><br>
(07:52:05 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> then delete image in glance</font><font color="#16569E" face="serif"><br>
(07:52:09 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> yes</font><font color="#16569E" face="serif"><br>
(07:53:20 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> anyway, if user want to re-register an image with its given id from other place, he will run into problem based on current code</font><font color="#A82F2F" face="serif"><br>
(07:54:10 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> flwang: yep, I guess passing the ID makes sense when you've 3K services pointing to that specific ID and you want them to use glance</font><font color="#16569E" face="serif"><br>
(07:54:35 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> exactly</font><font color="#A82F2F" face="serif"><br>
(07:55:19 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> mmh, interesting. I'm trying to find the bp / discussion about "really deleting" images. I can't find it</font><font color="#16569E" face="serif"><br>
(07:58:04 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> for short term, I want to reuse the id</font><font color="#16569E" face="serif"><br>
(07:59:03 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> when create new image, glance will try to check if there is a dead image if there is a given image id</font><font color="#16569E" face="serif"><br>
(07:59:14 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> if yes, reuse id</font><font color="#16569E" face="serif"><br>
(07:59:55 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> any concern?</font><font color="#A82F2F" face="serif"><br>
(08:00:01 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> might make sense. It would be cool if you could write this to the mailing list so that other folks can comment. Make sure your point the use-cases and ideas there</font><font color="#062585" face="serif"><br>
(08:00:19 AM)</font><font size="3" color="#062585" face="serif"> </font><font size="3" color="#062585" face="serif"><b>***flaper87</b></font><font size="3" face="serif"> is still trying to find that stuff</font><font color="#A82F2F" face="serif"><br>
(08:00:22 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> arrgh</font><font color="#A82F2F" face="serif"><br>
(08:00:24 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> internet is sooo big</font><font color="#16569E" face="serif"><br>
(08:02:23 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> good suggestion, I will send it to mailing list. Actually, I have worked out the patch, and I think it works fine.</font><font color="#A82F2F" face="serif"><br>
(08:03:57 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> If you have some code to show, I'd suggest to either push it somewhere in GH or submitting the patch (thing is that this change, most likely, requires a blueprint so, I'd rather get some feedback before going down that road)</font><font color="#16569E" face="serif"><br>
(08:13:45 AM)</font><font size="3" color="#16569E" face="serif"> </font><font size="3" color="#16569E" face="serif"><b>flwang:</b></font><font size="3" face="serif"> I'm going to submit the patch as the fix of this bug, make sense?</font><font color="#A82F2F" face="serif"><br>
(08:14:30 AM)</font><font size="3" color="#A82F2F" face="serif"> </font><font size="3" color="#A82F2F" face="serif"><b>flaper87:</b></font><font size="3" face="serif"> mmh, yeah, that makes sense. Expect some discussions, though. </font><font size="3" face="sans-serif"><br>
</font><font face="sans-serif"><br>
=============================================================================</font><font size="3" face="sans-serif"><br>
</font><font face="sans-serif"><br>
diff -rupN glance-unpatched/glance/db/sqlalchemy/api.py glance/glance/db/sqlalchemy/api.py<br>
--- glance-unpatched/glance/db/sqlalchemy/api.py 2013-04-17 19:45:48.175132054 -0500<br>
+++ glance/glance/db/sqlalchemy/api.py 2013-04-17 19:46:03.905136572 -0500<br>
@@ -235,7 +235,34 @@ def wrap_db_error(f):<br>
<br>
def image_create(context, values):<br>
"""Create an image from the values dictionary."""<br>
- return _image_update(context, values, None, False)<br>
+ image_id = values.get('id', None)<br>
+ if image_id:<br>
+ session = get_session()<br>
+ with session.begin():<br>
+ # The create image request comes with an ID, since these are<br>
+ # supposed to be universal IDs, this may mean that a user is<br>
+ # trying to re-add an image that he had previously deleted.<br>
+ # Let's let him do that, by "undeleting" the image.<br>
+ try:<br>
+ dead_image = _image_get(context,<br>
+ image_id,<br>
+ session=session,<br>
+ force_show_deleted=True)<br>
+ LOG.debug("Dead Image: %s" % dead_image.deleted)<br>
+ if dead_image.deleted:<br>
+ LOG.info("Image '%s' had been deleted before "<br>
+ "and will be reactivated." % image_id)<br>
+ dead_image.undelete(session=session)<br>
+ except exception.NotFound:<br>
+ LOG.debug("Adding new image with user provided "<br>
+ "UUID '%s'" % image_id)<br>
+ # Not really a failure, this just means the user provided a<br>
+ # UUID for the very first time, still let's null it out<br>
+ # for _image+_update() because even if the user provided<br>
+ # an ID, this method does not expect the ID here when creating<br>
+ # new images<br>
+ image_id = None<br>
+ return _image_update(context, values, image_id, False)<br>
<br>
<br>
def image_update(context, image_id, values, purge_props=False):<br>
diff -rupN glance-unpatched/glance/db/sqlalchemy/models.py glance/glance/db/sqlalchemy/models.py<br>
--- glance-unpatched/glance/db/sqlalchemy/models.py 2013-04-27 19:45:48.175132054 -0500<br>
+++ glance/glance/db/sqlalchemy/models.py 2013-04-17 19:46:03.918174498 -0500<br>
@@ -66,6 +66,13 @@ class ModelBase(object):<br>
self.deleted_at = timeutils.utcnow()<br>
self.save(session=session)<br>
<br>
+ def undelete(self, session=None):<br>
+ """Undelete this object"""<br>
+ self.deleted = False<br>
+ self.deleted_at = None<br>
+ self.status = 'active'<br>
+ self.save(session=session)<br>
+<br>
def update(self, values):<br>
"""dict.update() behaviour."""<br>
for k, v in values.iteritems():</font><font size="3" face="sans-serif"><br>
<br>
<br>
</font><font face="sans-serif"><br>
Thanks & Best regards,<br>
Fei Long Wang (王飞龙)<br>
--------------------------------------------------<br>
Scrum Master, Cloud Solutions and OpenStack Development<br>
Tel: 8610-82450513 | T/L: 905-0513 <br>
Email: <a href="mailto:flwang@cn.ibm.com" target="_blank">flwang@cn.ibm.com</a><br>
China Systems & Technology Laboratory in Beijing<br>
--------------------------------------------------</font><tt><font>_______________________________________________<br>
Mailing list: </font></tt><tt><font><a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a></font></tt><tt><font><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : </font></tt><tt><font><a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a></font></tt><tt><font><br>
More help : </font></tt><tt><font><a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a></font></tt><tt><font><br>
</font></tt></p></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>