Open Stack

Slawek,

did you change Swift proxy-server.conf file too?

On Wed, Oct 1, 2014 at 11:26 PM, Sławek Kapłoński <slawek at kaplonski.pl>
wrote:

> Hello,
>
> Thanks for Your help but it not helps. I checked that for sure on each
> swift
> node there is a lot of free space. What can confirm that is fact that when
> I
> try to create image with size about 1.7GB and I have
> swift_store_large_object_size set to 1GB than there is error (always after
> send first chunk to swift (200MB). When I only change
> swift_store_large_object_size to 2GB and restart glance-api than the same
> image is created correctly (it is then in one big object).
>
> ---
> Best regards
> Sławek Kapłoński
> slawek at kaplonski.pl
>
> Dnia wtorek, 30 września 2014 22:28:11 Timur Nurlygayanov pisze:
> > Hi Slawek,
> >
> > we faced the same error and this is issue with Swift.
> > We can see 100% disk usage on the Swift node during the file upload and
> > looks like Swift can't send info about status of the file loading in
> time.
> >
> > On our environments we found the workaround for this issue:
> > 1. Set  swift_store_large_object_size = 200 in glance.conf.
> > 2. Add to Swift proxy-server.conf:
> >
> > [DEFAULT]
> > ...
> > node_timeout = 90
> >
> > Probably we can set this value as default value for this parameter
> instead
> > of '30'?
> >
> >
> > Regards,
> > Timur
> >
> >
> > On Tue, Sep 30, 2014 at 7:41 PM, Sławek Kapłoński <slawek at kaplonski.pl>
> >
> > wrote:
> > > Hello,
> > >
> > > I can't find that upload from was previous logs but I now try to upload
> > > same image once again. In glance there was exactly same error. In swift
> > > logs I have:
> > >
> > > Sep 30 17:35:10 127.0.0.1 proxy-server X.X.X.X Y.Y.Y.Y
> > > 30/Sep/2014/15/35/10 HEAD
> /v1/AUTH_7ef5a7661ccd4c069e3ad387a6dceebd/glance
> > > HTTP/1.0 204
> > > Sep 30 17:35:16 127.0.0.1 proxy-server X.X.X.X Y.Y.Y.Y
> > > 30/Sep/2014/15/35/16 PUT /v1/AUTH_7ef5a7661ccd4c069e3ad387a6dcee
> > > bd/glance/fa5dfe09-74f5-4287-9852-d2f1991eebc0-00001 HTTP/1.0 201 - -
> > >
> > > Best regards
> > > Slawek Kaplonski
> > >
> > > W dniu 2014-09-30 17:03, Kuo Hugo napisał(a):
> > >> Hi ,
> > >>
> > >> Could you please post the log of related requests in Swift's log ???
> > >>
> > >> Thanks // Hugo
> > >>
> > >> 2014-09-30 22:20 GMT+08:00 Sławek Kapłoński <slawek at kaplonski.pl>:
> > >>  Hello,
> > >>
> > >>> I'm using openstack havana release and glance with swift backend.
> > >>> Today I found that I have problem when I create image with url in
> > >>> "--copy-from" when image is bigger than my
> > >>> "swift_store_large_object_size" because then glance is trying to
> > >>> split image to chunks with size given in
> > >>> "swift_store_large_object_chunk_size" and when try to upload first
> > >>> chunk to swift I have error:
> > >>>
> > >>> 2014-09-30 15:05:29.361 18023 ERROR glance.store.swift [-] Error
> > >>> during chunked upload to backend, deleting stale chunks
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift Traceback
> > >>> (most recent call last):
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
> > >>> "/usr/lib/python2.7/dist-packages/glance/store/swift.py", line 384,
> > >>> in add
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
> > >>>
> > >>>  content_length=content_length)
> > >>>
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
> > >>> "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1234,
> > >>> in put_object
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
> > >>>
> > >>>  response_dict=response_dict)
> > >>>
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
> > >>> "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1143,
> > >>> in _retry
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
> > >>>
> > >>>  reset_func(func, *args, **kwargs)
> > >>>
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift   File
> > >>> "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1215,
> > >>> in _default_reset
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift     %
> > >>> (container, obj))
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
> > >>> ClientException: put_object('glance',
> > >>> '9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no
> > >>> ability to reset contents for reupload.
> > >>> 2014-09-30 15:05:29.361 18023 TRACE glance.store.swift
> > >>> 2014-09-30 15:05:29.362 18023 ERROR glance.store.swift [-] Failed
> > >>> to add object to Swift.
> > >>> Got error from Swift: put_object('glance',
> > >>> '9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no
> > >>> ability to reset contents for reupload.
> > >>> 2014-09-30 15:05:29.362 18023 ERROR glance.api.v1.upload_utils [-]
> > >>> Failed to upload image 9f56ccec-deeb-4020-95ba-ca7bf1170056
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>> Traceback (most recent call last):
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>>
> > >>>  File
> > >>>
> > >>> "/usr/lib/python2.7/dist-packages/glance/api/v1/upload_utils.py",
> > >>> line 101, in upload_data_to_store
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>>
> > >>>  store)
> > >>>
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>>
> > >>>  File "/usr/lib/python2.7/dist-packages/glance/store/__init__.py",
> > >>>
> > >>> line 333, in store_add_to_backend
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>>
> > >>>  (location, size, checksum, metadata) = store.add(image_id, data,
> > >>>
> > >>> size)
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>>
> > >>>  File "/usr/lib/python2.7/dist-packages/glance/store/swift.py",
> > >>>
> > >>> line 447, in add
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>>
> > >>>  raise glance.store.BackendException(msg)
> > >>>
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils
> > >>> BackendException: Failed to add object to Swift.
> > >>> 2014-09-30 15:05:29.362 18023 TRACE glance.api.v1.upload_utils Got
> > >>> error from Swift: put_object('glance',
> > >>> '9f56ccec-deeb-4020-95ba-ca7bf1170056-00001', ...) failure and no
> > >>> ability to reset contents for reupload.
> > >>>
> > >>> Does someone of You got same error and know what is solution of it?
> > >>> I was searching about that in google but I not found anything what
> > >>> could solve my problem.
> > >>>
> > >>> --
> > >>> Best regards
> > >>> Sławek Kapłoński
> > >>> slawek at kaplonski.pl
> > >>>
> > >>> _______________________________________________
> > >>> Mailing list:
> > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
> > >>> Post to     : openstack at lists.openstack.org
> > >>> Unsubscribe :
> > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack [1]
> > >>
> > >> Links:
> > >> ------
> > >> [1] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> > >
> > > --
> > > Pozdrawiam
> > > Sławek Kapłoński
> > > slawek at kaplonski.pl
> > >
> > > _______________________________________________
> > > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/
> > > openstack
> > > Post to     : openstack at lists.openstack.org
> > > Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/
> > > openstack
> > > _______________________________________________
> > > OpenStack-dev mailing list
> > > OpenStack-dev at lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Timur,
QA Engineer
OpenStack Projects
Mirantis Inc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141002/d0a45b85/attachment.html>