[Openstack] ownCloud and openstack
Tom Fifield
tom at openstack.org
Mon Jul 14 04:36:36 UTC 2014
On 13/07/14 13:48, Remo Mattei wrote:
> As far as I know max size is 5gb into swift unless you setup a way to
> split it
5GB is the default limit for a single request. However, swift provides a
couple of ways to upload larger single objects:
http://docs.openstack.org/developer/swift/overview_large_objects.html
OwnCloud should probably implement one of these methods to allow objects
greater than the default limit.
Regards,
Tom
> Il giorno Jul 12, 2014, alle ore 16:42, Jason Bishop
> <jason.bishop at gmail.com <mailto:jason.bishop at gmail.com>> ha scritto:
>
>>
>> Hi Frank, I have to say ownCloud is really nicely done. i have been
>> in need of a file share which has native linux client support and a
>> really robust storage backend. ownCloud with swift meets the
>> requirements. I do have a comment which mirrors Hugo's comments I
>> think. Is it feasible to have the concept of users in ownCloud more
>> closely match up with swift's concept? It would be nice if the
>> backend storage for each user was stored in that user's swift account.
>> I'm sure it will add complexity to user management and authentication
>> but the advantages of separation of user data may offset that.
>>
>> I did some testing on ubuntu trusty against an ubuntu trusty icehouse
>> openstack. I updated the config file and didn't get it quite right
>> the first time. The error in the admin gui told me what the problem
>> was (I had wrong string in for the region attribute) so that worked nice.
>>
>> I stood up an instance of trusty and installed the owncloud client and
>> copied some files into the shared directory.
>>
>> a 2gb file worked fine. when i tried a 7gb file I got this error
>> message, so I think the code in ownCloud does not handle > 5GB files
>> in swift. It also looks like it keeps retrying the upload over and
>> over. i'll watch it to see if it eventually gives up and errors out.
>>
>> Fatal webdav Exception: #9 {main} 2014-07-12T16:11:43+00:00
>> Fatal webdav Exception: #8 /var/www/html/owncloud/remote.php(52):
>> require_once('/var/www/html/o...') 2014-07-12T16:11:43+00:00
>> Fatal webdav Exception: #7
>> /var/www/html/owncloud/apps/files/appinfo/remote.php(61):
>> Sabre\DAV\Server->exec() 2014-07-12T16:11:43+00:00
>> FatalwebdavException: #6
>> /var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(214):
>> Sabre\DAV\Server->invokeMethod('PUT',
>> '7gigafile-chunk...')2014-07-12T16:11:43+00:00
>> FatalwebdavException: #5
>> /var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(474):
>> call_user_func(Array, '7gigafile-chunk...')2014-07-12T16:11:43+00:00
>> FatalwebdavException: #4 [internal function]:
>> Sabre\DAV\Server->httpPut('7gigafile-chunk...')2014-07-12T16:11:43+00:00
>> FatalwebdavException: #3
>> /var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(900):
>> Sabre\DAV\Server->createFile('7gigafile-chunk...', Resource id #17,
>> NULL)2014-07-12T16:11:43+00:00
>> FatalwebdavException: #2
>> /var/www/html/owncloud/3rdparty/sabre/dav/lib/Sabre/DAV/Server.php(1647):
>> OC_Connector_Sabre_Directory->createFile('7gigafile-chunk...',
>> Resource id #17)2014-07-12T16:11:43+00:00
>> FatalwebdavException: #1
>> /var/www/html/owncloud/lib/private/connector/sabre/directory.php(76):
>> OC_Connector_Sabre_File->put(Resource id #17)2014-07-12T16:11:43+00:00
>> FatalwebdavSabre\DAV\Exception: Could not rename part file assembled
>> from chunks2014-07-12T16:11:43+00:00
>> Errorwebdav\OC\Files\Filesystem::rename() failed2014-07-12T16:11:43+00:00
>> ErrorobjectstoreCould not create object: Client error response [status
>> code] 413 [reason phrase] Request Entity Too Large [url]
>> http://openstack-swiftproxy.stanford.edu:8080/v1/AUTH_78043e536430433fae0318b0ac1040ff/owncloud2/urn:oid:1163
>>
>> I will continue testing and let you know if I find anything else. I
>> am still using just http connection for keystone and swift services.
>> I will pull proper https certificates and check ssl plumbing is
>> working fine.
>>
>> After that my next steps will probably be authentication and LDAP related.
>>
>> Jason
>>
>>
>>
>> On Mon, Jul 7, 2014 at 9:09 AM, Frank Karlitschek <frank at owncloud.com
>> <mailto:frank at owncloud.com>> wrote:
>>
>> Hi Hugo,
>>
>>
>> thanks a lot for the recommendations. This is super helpful.
>> We will look into the upload problem you found.
>>
>>
>> Cheers
>> Frank
>>
>>
>> > On 07.07.2014, at 05:09, Kuo Hugo <tonytkdk at gmail.com
>> <mailto:tonytkdk at gmail.com>> wrote:
>> >
>> > Hi Frank,
>> >
>> > I had a try with it on my box here. Most of OwnCloud's features
>> are done by OwnCloud host. Not too much functions rely on Swift's
>> feature. It simply uses CRUD methods.
>> >
>> > [Suggestions]
>> > • Multiple containers/accounts for Swift as primary
>> storage. The performance will degrade while over 1million within a
>> container.
>> > • Since all metadata is stored in the DB on OwnCloud host.
>> And OwnCloud remaps the filename as object_id in Swift. That would
>> be good to store some mapping metadata in Swift for safety. If the
>> DB crashed, no chance to get the proper data from Swift. User has
>> no knowledge about object name "urn:oid:153".
>> > • For large object(100MB+), OwnCloud chunks it into pieces
>> and upload each segment to Swift as single object. Swift provide
>> two methods for handling large object DLO/SLO. It might be good to
>> leverage it for reduce some loading on OwnCloud. And more
>> consistency of how large object been stored in Swift.
>> > • Data traffic passthrough the OwnCloud server will cause
>> the bottleneck/loading on OwnCloud host. To upload data to Swift
>> API from client app directly may bring you some benefits.
>> > • OwnCloud can have the shared link from Swift directly
>> via TempURL feature.
>> > • The Object expired feature of Swift is a plus for
>> OwnCloud to use.
>> > • Pre-create folders for user in objectstore as primary
>> storage mode.
>> > • When downloading file, seems OwnCloud will cache it
>> somewhere. Once the entire object were loaded into OwnCloud then
>> sending to user. To streaming it is an ideally way to do so from
>> my perspective.
>> >
>> > Renaming is easy on OwnCloud by overwriting the value in DB.
>> > Thumbnail is good idea.
>> > To edit txt file on Web is awesome.
>> >
>> > I got a problem to upload a 470MB object from OSX client app. It
>> just showed me timeout but no any further clues here.
>> >
>> > Cheers // Hugo
>> >
>> >
>> >
>> >
>> > 2014-07-04 10:31 GMT+08:00 John Dickinson <me at not.mn
>> <mailto:me at not.mn>>:
>> > That's fantastic news! Hearing more companies and applications
>> supporting Swift is great for the community, and it gives us as
>> contributors yet another use case to reference.
>> >
>> > I work with Hugo, so I'll peek over his shoulder to check it
>> out. Meanwhile, if there's anything I can do to help answer
>> questions or get involved in the community (from contributing to
>> getting issues addressed), please let me know.
>> >
>> > --John
>> >
>> >
>> > > On Jul 3, 2014, at 6:01 PM, Frank Karlitschek
>> <frank at owncloud.com <mailto:frank at owncloud.com>> wrote:
>> > >
>> > > Hi everyone,
>> > >
>> > >
>> > > Tom Fifield recommended that I send a message to this list.
>> > >
>> > > As some of you might know we at ownCloud just added Swift
>> support as primary storage to ownCloud. This will be released as
>> part of our upcoming release 7 in about 3 weeks.
>> > >
>> > > The code can be seen here.
>> > > https://github.com/owncloud/core/pull/8383
>> > >
>> > >
>> > > ownCloud is a fully open source solution that can be used a
>> self hosted Dropbox or Google Drive alternative. The new Swift
>> backend is an important step for us to mask ownCloud more
>> scalable, easier to deploy in bigger setups and integrate it
>> better into OpenStack.
>> > >
>> > > Tom suggested to ask here for testers who are willing to try
>> ownCloud 7 together with Swift.
>> > > We just announced the RC1 a few minutes ago:
>> > >
>> http://mailman.owncloud.org/pipermail/announcements/2014-July/000049.html
>> > >
>> > >
>> > > So it would be great if you guys could give it a test and give
>> some feedback.
>> > >
>> > >
>> > > Thanks a lot
>> > >
>> > >
>> > > Cheers
>> > > Frank
>> > > _______________________________________________
>> > > Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> > > Post to : openstack at lists.openstack.org
>> <mailto:openstack at lists.openstack.org>
>> > > Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >
>> > _______________________________________________
>> > Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> > Post to : openstack at lists.openstack.org
>> <mailto:openstack at lists.openstack.org>
>> > Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >
>>
>>
>> _______________________________________________
>> Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openstack at lists.openstack.org
>> <mailto:openstack at lists.openstack.org>
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>> !DSPAM:1,53c1cac3265432135880153!
>> _______________________________________________
>> Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openstack at lists.openstack.org
>> <mailto:openstack at lists.openstack.org>
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>> !DSPAM:1,53c1cac3265432135880153!
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
More information about the Openstack
mailing list