Open Stack

hi Saverio, all

we've just found the solution. The issue is in the way swift manages 
files bigger than 5 GB, as explained here:

http://docs.openstack.org/developer/swift/overview_large_objects.html

Swift segments big files and puts them in a new container called 
<name_of_container>_segments. So it is necessary to give read access to 
the _segment container too in order to make external tenants access it:

swift post mycontainer -r 'ext_tenant:*'
swift post mycontainer_segments -r 'ext_tenant:*'

now the ext_tenant user can read and download big files.

cheers,

             Alberto




Il 12/05/2016 17:29, Saverio Proto ha scritto:
> Hello,
>
> I got this working, I was actually missing the storage-url in my client config.
>
> Related to this setup I am experiencing a weird problem with large
> objects when reading from another tenant.
>
> Only when I read objects from a container where I have read-only
> access, if the object is larger than 10GB I am not able to read the
> object.
>
> I used the rclone client that has a very covenient option --dump-bodies.
>
> When I list the objects in the container I receive a JSON structure
> with all the objects in the container. All objects that I can read
> have data that makes sense. Some files that are bigger than 10Gb have
> a size of 0 bytes.
>
> example:
>
> {"hash": "d41d8cd98f00b204e9800998ecf8427e", "last_modified":
> "2016-05-10T15:12:44.233710", "bytes": 0, "name":
> "eng/googlebooks-eng-all-3gram-20120701-li.gz", "content_type":
> "application/octet-stream"}
>
> or sometimes the bytes size is just random and wrong.
>
> When I try to read these objects I get a 403.
>
> I tried both with swiftclient and rclone and I have the same problem.
>
> Of course if I use the container in my tenant where I have read and
> write, I can successfully read and write all the large objects. This
> only happens when reading a large object shared across tenants.
>
> Did you maybe try to work with such large objects in your setup ? does
> it work for you ?
>
> Saverio
>
>
>
> 2016-05-03 14:34 GMT+02:00 Wijngaarden, Pieter van
> <pieter.van.wijngaarden at philips.com>:
>> Hi Saverio,
>>
>>
>>
>> Yes, in the end I was able to get it working! The issue was related to my
>> proxy server pipeline config (filter:authtoken). I did not find pointers to
>> updated documentation though.
>>
>>
>>
>> When I had updated the [filter:authtoken] configuration in
>> /etc/swift/proxy-server.conf, everything worked. In my case the values
>> auth_uri and auth_url were not configured correctly:
>>
>>
>>
>> [filter:authtoken]
>>
>> paste.filter_factory = keystonemiddleware.auth_token:filter_factory
>>
>> auth_uri = https://<keystone_public_IP>:5443
>>
>> auth_url = http://<keystone_internal_IP>:35357
>>
>> auth_plugin = password
>>
>> project_name = service
>>
>> project_domain_id = default
>>
>> user_domain_id = default
>>
>> username = swift
>>
>> password = XXXXXXXXXXXXXXXXXXXXX
>>
>>
>>
>> I don’t know why that meant that regular token validation worked, but
>> cross-tenant did not
>>
>>
>>
>> (unfortunately it’s a test cluster so I don’t have history on what it was
>> before I changed it :( )
>>
>>
>>
>> What works for me now (using python-swiftclient) is the following. I hope
>> that the text formatting survives in the email:
>>
>>
>>
>> 1.       A user with complete ownership over the account (say account X)
>> executes
>>
>> a.  swift post <container> --read-acl ‘<project-name>:<username>’
>>
>> b.      or
>>
>> c.  swift post <container> --read-acl ‘<project-name>:*>’
>>
>> 2.       A user in the <project-name> account can now list the container and
>> get objects in the container by doing:
>>
>> a.  swift list <container> --os-storage-url <storage URL for account X>
>> --os-auth-token <normal token>
>>
>> b.      or
>>
>> c.  swift download <container> <object> --os-storage-url <same as above>
>> --os-auth-token <same as above>
>>
>>
>>
>> Note that you can review the full storage URL for an account by doing swift
>> stat -v.
>>
>>
>>
>> In this case, the user in step 2 is not able to do anything else in account
>> X besides do object listing in the container and get its objects, which is
>> what I was aiming for. What does not work for me is if I set the read-acl to
>> ‘<project-name>’ only, even though that should work according to the
>> documentation. If you want to allow all users in another project read access
>> to a container, use ‘<project-name>:*’ as the read-ACL.
>>
>>
>>
>> I hope this helps!
>>
>>
>>
>> With kind regards,
>>
>> Pieter van Wijngaarden
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Saverio Proto [mailto:zioproto at gmail.com]
>> Sent: dinsdag 3 mei 2016 12:44
>> To: Wijngaarden, Pieter van <pieter.van.wijngaarden at philips.com>
>> Cc: openstack-operators at lists.openstack.org
>> Subject: Re: [Openstack-operators] Swift ACL's together with Keystone (v3)
>> integration
>>
>>
>>
>> Hello Pieter,
>>
>>
>>
>> I did run into the same problem today. Did you find pointers to more updated
>> documentation ? Were you able to configure the cross tenant read ACL ?
>>
>>
>>
>> thank you
>>
>>
>>
>> Saverio
>>
>>
>>
>>
>>
>> 2016-04-20 13:48 GMT+02:00 Wijngaarden, Pieter van
>>
>> <pieter.van.wijngaarden at philips.com>:
>>
>>> Hi all,
>>> I’m playing around with a Swift cluster (Liberty) and cannot get the
>>> Swift ACL’s to work. My objective is to give users from one project
>>> (and thus Swift account?) selective access to specific containers in
>>> another project.
>>> According to
>>> http://docs.openstack.org/developer/swift/middleware.html#keystoneauth
>>> , the swift/keystoneauth plugin should support cross-tenant (now
>>> cross-project) ACL’s by setting the read-acl of a container to something
>>> like:
>>> swift post <containername> --read-acl '<projectname>:<username>'
>>> Using a project name instead of a UUID should be supported if all
>>> projects are in the default domain.
>>> But if I set this for a user in a different project / different swift
>>> account, it doesn’t seem to work. The last reference to Swift
>>> container ACL’s from the archives is somewhere in 2011..
>>> I have found a few Swift ACL examples / tutorials online, but they are
>>> all outdated or appear to use special / proprietary middleware. Does
>>> anybody have (or can anybody create) an example that is up-to-date for
>>> OpenStack Liberty or later, and shows container ACL’s together with
>>> Keystone integration?
>>> What I would like to do:
>>> - I have a bunch of users and projects in Keystone, and thus a bunch
>>> of (automatically created) Swift accounts
>>> - I would like to allow one specific user in a project (say project X)
>>> to access a container from a different project (Y)
>>> - And/or, I would like to allow all users in project X to access one
>>> specific container in project Y.
>>> Both these options should include listing the objects in the
>>> container, but exclude listing all containers in the other account.
>>> I hope there is someone who can help, thanks a lot in advance!
>>> With kind regards,
>>> Pieter van Wijngaarden
>>> System Architect
>>> Digital Pathology Solutions
>>> Philips Healthcare
>>> Veenpluis 4-6, Building QY-2.006, 5684 PC Best
>>> Tel: +31 6 2958 6736, Email: pieter.van.wijngaarden at philips.com
>>>    ________________________________
>>> The information contained in this message may be confidential and
>>> legally protected under applicable law. The message is intended solely
>>> for the addressee(s). If you are not the intended recipient, you are
>>> hereby notified that any use, forwarding, dissemination, or
>>> reproduction of this message is strictly prohibited and may be
>>> unlawful. If you are not the intended recipient, please contact the
>>> sender by return e-mail and destroy all copies of the original message.
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operator
>>> s
>>
>> ________________________________
>> The information contained in this message may be confidential and legally
>> protected under applicable law. The message is intended solely for the
>> addressee(s). If you are not the intended recipient, you are hereby notified
>> that any use, forwarding, dissemination, or reproduction of this message is
>> strictly prohibited and may be unlawful. If you are not the intended
>> recipient, please contact the sender by return e-mail and destroy all copies
>> of the original message.
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-- 
  Alberto Colla
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|GARR - The Italian Academic and Research Network    |
|Via dei Tizii, 6 I-00185 Rome - Italy               |
|office: +39 06 4962 2301 mobile: +39 340 46 24 581  |
|Email: alberto.colla at garr.it     http://www.garr.it |
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~