Open Stack

Hello,

I got this working, I was actually missing the storage-url in my client config.

Related to this setup I am experiencing a weird problem with large
objects when reading from another tenant.

Only when I read objects from a container where I have read-only
access, if the object is larger than 10GB I am not able to read the
object.

I used the rclone client that has a very covenient option --dump-bodies.

When I list the objects in the container I receive a JSON structure
with all the objects in the container. All objects that I can read
have data that makes sense. Some files that are bigger than 10Gb have
a size of 0 bytes.

example:

{"hash": "d41d8cd98f00b204e9800998ecf8427e", "last_modified":
"2016-05-10T15:12:44.233710", "bytes": 0, "name":
"eng/googlebooks-eng-all-3gram-20120701-li.gz", "content_type":
"application/octet-stream"}

or sometimes the bytes size is just random and wrong.

When I try to read these objects I get a 403.

I tried both with swiftclient and rclone and I have the same problem.

Of course if I use the container in my tenant where I have read and
write, I can successfully read and write all the large objects. This
only happens when reading a large object shared across tenants.

Did you maybe try to work with such large objects in your setup ? does
it work for you ?

Saverio



2016-05-03 14:34 GMT+02:00 Wijngaarden, Pieter van
<pieter.van.wijngaarden at philips.com>:
> Hi Saverio,
>
>
>
> Yes, in the end I was able to get it working! The issue was related to my
> proxy server pipeline config (filter:authtoken). I did not find pointers to
> updated documentation though.
>
>
>
> When I had updated the [filter:authtoken] configuration in
> /etc/swift/proxy-server.conf, everything worked. In my case the values
> auth_uri and auth_url were not configured correctly:
>
>
>
> [filter:authtoken]
>
> paste.filter_factory = keystonemiddleware.auth_token:filter_factory
>
> auth_uri = https://<keystone_public_IP>:5443
>
> auth_url = http://<keystone_internal_IP>:35357
>
> auth_plugin = password
>
> project_name = service
>
> project_domain_id = default
>
> user_domain_id = default
>
> username = swift
>
> password = XXXXXXXXXXXXXXXXXXXXX
>
>
>
> I don’t know why that meant that regular token validation worked, but
> cross-tenant did not
>
>
>
> (unfortunately it’s a test cluster so I don’t have history on what it was
> before I changed it :( )
>
>
>
> What works for me now (using python-swiftclient) is the following. I hope
> that the text formatting survives in the email:
>
>
>
> 1.       A user with complete ownership over the account (say account X)
> executes
>
> a.  swift post <container> --read-acl ‘<project-name>:<username>’
>
> b.      or
>
> c.  swift post <container> --read-acl ‘<project-name>:*>’
>
> 2.       A user in the <project-name> account can now list the container and
> get objects in the container by doing:
>
> a.  swift list <container> --os-storage-url <storage URL for account X>
> --os-auth-token <normal token>
>
> b.      or
>
> c.  swift download <container> <object> --os-storage-url <same as above>
> --os-auth-token <same as above>
>
>
>
> Note that you can review the full storage URL for an account by doing swift
> stat -v.
>
>
>
> In this case, the user in step 2 is not able to do anything else in account
> X besides do object listing in the container and get its objects, which is
> what I was aiming for. What does not work for me is if I set the read-acl to
> ‘<project-name>’ only, even though that should work according to the
> documentation. If you want to allow all users in another project read access
> to a container, use ‘<project-name>:*’ as the read-ACL.
>
>
>
> I hope this helps!
>
>
>
> With kind regards,
>
> Pieter van Wijngaarden
>
>
>
>
>
> -----Original Message-----
> From: Saverio Proto [mailto:zioproto at gmail.com]
> Sent: dinsdag 3 mei 2016 12:44
> To: Wijngaarden, Pieter van <pieter.van.wijngaarden at philips.com>
> Cc: openstack-operators at lists.openstack.org
> Subject: Re: [Openstack-operators] Swift ACL's together with Keystone (v3)
> integration
>
>
>
> Hello Pieter,
>
>
>
> I did run into the same problem today. Did you find pointers to more updated
> documentation ? Were you able to configure the cross tenant read ACL ?
>
>
>
> thank you
>
>
>
> Saverio
>
>
>
>
>
> 2016-04-20 13:48 GMT+02:00 Wijngaarden, Pieter van
>
> <pieter.van.wijngaarden at philips.com>:
>
>> Hi all,
>
>>
>
>> I’m playing around with a Swift cluster (Liberty) and cannot get the
>
>> Swift ACL’s to work. My objective is to give users from one project
>
>> (and thus Swift account?) selective access to specific containers in
>> another project.
>
>>
>
>> According to
>
>> http://docs.openstack.org/developer/swift/middleware.html#keystoneauth
>
>> , the swift/keystoneauth plugin should support cross-tenant (now
>
>> cross-project) ACL’s by setting the read-acl of a container to something
>> like:
>
>>
>
>> swift post <containername> --read-acl '<projectname>:<username>'
>
>>
>
>> Using a project name instead of a UUID should be supported if all
>
>> projects are in the default domain.
>
>>
>
>> But if I set this for a user in a different project / different swift
>
>> account, it doesn’t seem to work. The last reference to Swift
>
>> container ACL’s from the archives is somewhere in 2011..
>
>>
>
>> I have found a few Swift ACL examples / tutorials online, but they are
>
>> all outdated or appear to use special / proprietary middleware. Does
>
>> anybody have (or can anybody create) an example that is up-to-date for
>
>> OpenStack Liberty or later, and shows container ACL’s together with
>
>> Keystone integration?
>
>>
>
>> What I would like to do:
>
>> - I have a bunch of users and projects in Keystone, and thus a bunch
>
>> of (automatically created) Swift accounts
>
>> - I would like to allow one specific user in a project (say project X)
>
>> to access a container from a different project (Y)
>
>> - And/or, I would like to allow all users in project X to access one
>
>> specific container in project Y.
>
>> Both these options should include listing the objects in the
>
>> container, but exclude listing all containers in the other account.
>
>>
>
>> I hope there is someone who can help, thanks a lot in advance!
>
>>
>
>> With kind regards,
>
>> Pieter van Wijngaarden
>
>> System Architect
>
>> Digital Pathology Solutions
>
>> Philips Healthcare
>
>>
>
>> Veenpluis 4-6, Building QY-2.006, 5684 PC Best
>
>> Tel: +31 6 2958 6736, Email: pieter.van.wijngaarden at philips.com
>
>>
>
>>
>
>>
>
>>
>
>>   ________________________________
>
>> The information contained in this message may be confidential and
>
>> legally protected under applicable law. The message is intended solely
>
>> for the addressee(s). If you are not the intended recipient, you are
>
>> hereby notified that any use, forwarding, dissemination, or
>
>> reproduction of this message is strictly prohibited and may be
>
>> unlawful. If you are not the intended recipient, please contact the
>
>> sender by return e-mail and destroy all copies of the original message.
>
>>
>
>> _______________________________________________
>
>> OpenStack-operators mailing list
>
>> OpenStack-operators at lists.openstack.org
>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operator
>
>> s
>
>>
>
>
> ________________________________
> The information contained in this message may be confidential and legally
> protected under applicable law. The message is intended solely for the
> addressee(s). If you are not the intended recipient, you are hereby notified
> that any use, forwarding, dissemination, or reproduction of this message is
> strictly prohibited and may be unlawful. If you are not the intended
> recipient, please contact the sender by return e-mail and destroy all copies
> of the original message.