Open Stack

> I might be confused now or confused when I did that review, because I
> thought it was reducing download concurrency rather than upload
> concurrency. Are you talking about https://review.openstack.org/#/c/46479/ ?

    My bad ... yes it was downloads and not uploads. But I was referring
to concurrency/consitency issues in general, not just w.r.t. uploads.

    Zhi sent a link to the FSM blueprint in another mail so I'll take a
look at that. If I can manage to digest that then it should (hopefully)
solve our problems.

--
Koo


On Sun, Jan 26, 2014 at 05:34:47PM -0800, Mark Washenberger wrote:
> On Sun, Jan 26, 2014 at 4:37 PM, David Koo <kpublicmail at gmail.com> wrote:
> 
> >
> > > Perhaps there is still a bug where an image is getting stuck in 'saving'
> > or
> > > some other state when a PUT fails?
> >
> >     Yes, that's precisely the problem.
> >
> 
> We should definitely fix that, thanks for pointing it out!
> 
> 
> >
> >     Of course, one could argue that that if an upload fails the user
> > should be able to continue trying until the upload succeeds! But in that
> > case the image status should probably be reset to "queued" rather than
> > stay at "saving".
> >
> 
> That's exactly my argument so I'd like to see it go back to 'queued'.
> Nothing except the status has substantially changed during an upload that
> fails due to either the client or the underlying store, so it is easy to
> just revert the status and leave the image in a state where the user can
> reattempt the upload.
> 
> 
> >
> >     But this makes me a little uneasy because our
> > consistency/concurrency handling seems a little weak at the moment (am I
> > right?). If we were to have a more complicated state machine then we
> > would need much stronger consistency guarantees when there are
> > simultaneous uploads in progress (where some fail and some succeed)!
> >
> 
> +1 to less complicated state machines :-)
> 
> This is part of what the current work on the import task is designed to
> accomplish. When you use import, an image effectively has only two states,
> 'active' and nonexistent.
> 
> 
> 
> >
> >     Is there any work on this (concurrency/consistency) front? I
> > remember seeing some patches related to caching of simultaneous
> > downloads of an image file where issues related to concurrent update of
> > image metadata were addressed but IIRC it was -1ed because it reduced
> > concurrency.
> 
> 
> I might be confused now or confused when I did that review, because I
> thought it was reducing download concurrency rather than upload
> concurrency. Are you talking about https://review.openstack.org/#/c/46479/ ?
> 
> 
> >
> >     So do we bring back the 'killed' state or should we shoot for a more
> > complicated/powerful state machine?
> >
> 
> I think we can get by with trying to simplify the state that is involved
> and fixing any bugs with our state management. Is there a specific problem
> you're seeing with the
> 
> 
> >
> > --
> > Koo
> >
> >
> > On Sun, Jan 26, 2014 at 06:36:36AM -0800, Mark Washenberger wrote:
> > > It does not seem very ReSTful--or very usable, for that matter--for a
> > > resource to be permanently modified when you a PUT fails. So I don't
> > think
> > > we need the 'killed' status. It was purposefully left out of v2 images,
> > > which is not just a reskin of v1.
> > >
> > > Perhaps there is still a bug where an image is getting stuck in 'saving'
> > or
> > > some other state when a PUT fails?
> > >
> > >
> > > On Sun, Jan 26, 2014 at 5:10 AM, David Koo <kpublicmail at gmail.com>
> > wrote:
> > >
> > > >
> > > > Hi Fei,
> > > >
> > > >     Thanks for the confirmation.
> > > >
> > > > > I think you're right. The 'killed' status should be set in method
> > > > upload()
> > > > > if there is an upload failure, see
> > > > >
> > > >
> > https://github.com/openstack/glance/blob/master/glance/common/utils.py#L244
> > > >
> > > > I think you meant:
> > > >
> > > >
> > > >
> > https://github.com/openstack/glance/blob/master/glance/api/v1/upload_utils.py#L244
> > > >
> > > > (the safe_kill() call) right?
> > > >
> > > > --
> > > > Koo
> > > >
> > > >
> > > > > ------------------ Original ------------------
> > > > > From:  "David Koo"<kpublicmail at gmail.com>;
> > > > > Date:  Jan 26, 2014
> > > > > To:  "OpenStack Development Mailing
> > > > > List"<openstack-dev at lists.openstack.org>;
> > > > > Subject:  [openstack-dev] [Glance] Is the 'killed' state ever set in
> > v2?
> > > > >
> > > > > Hi All,
> > > > >
> > > > > While trying to work on a bug I was trying to simulate some image
> > > > > download failures and found that apparently the 'killed' state is
> > never
> > > > > set using v2 APIs.
> > > > >
> > > > > If I understand correctly, a file upload goes to
> > > > > api.v2.image_data.ImageDataController.upload and goes all the way to
> > > > > store.ImageProxy.set_data which proceeds to write to the backend
> > store.
> > > > >
> > > > > If the backend store raises an exception it is simply propagated all
> > the
> > > > > way up. The notifier re-encodes the exceptions (which is the bug I
> > was
> > > > > looking at) but doesn't do anything about the image status.
> > > > >
> > > > > Nowhere does the image status seem to get set to 'killed'.
> > > > >
> > > > > Before I log a bug I just wanted to confirm with everybody whether or
> > > > > not I've missed out on something.
> > > > >
> > > > > Thanks.
> > > > >
> > > > > --
> > > > > Koo
> > > >
> > > > _______________________________________________
> > > > OpenStack-dev mailing list
> > > > OpenStack-dev at lists.openstack.org
> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > > >
> >
> > > _______________________________________________
> > > OpenStack-dev mailing list
> > > OpenStack-dev at lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >

> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev