<div dir="ltr">Wow, so I glanced quickly at the github project to try and get some context, but I think I actually start getting *more* confused when I see "swift" in the same class name as "file-system" ;)<div>
<br></div><div style>I'd like you (or maybe vaidy, hi vaidy!) to correct me if I'm wrong, but this hadoop integration will *not* access the filesystem of the object servers directly? Everything will happen on a pool of processing boxes that will talk to swift via HTTP - same as any other client?</div>
<div style><br></div><div style>In that case, the error message is just a leaky abstraction showing through. HDFS probably has permission errors that it tries to helpfully map back to file system constructs which just don't apply when you're trying to "simulate" a file system on object storage. You'll have to get down the the HTTP to understand what's causing the error. Presumably a 401 from Swift, so access to swift logs would be helpful.</div>
<div style><br></div><div style>OTOH, if we're * actually* talking about filesystem permissions; then I'm totally confused. But ACL's definitely won't help. They're just a row sitting in a sqlite database - probably on a totally different server from where the one replica of this object is sitting on the filesystem. Nothing you can set in the api will change the filesystem permissions of the directory structure or files on the object servers.</div>
<div style><br></div><div style>Maybe do you have some more overview info on the general approach? I don't really have any Hadoop experience, so maybe it'd be better if there's a hadoop expert out there that also has some experience with swift and can help get you on the right track...</div>
<div style><br></div><div style>-Clay</div><div style><br></div><div style><br></div><div style><br></div><div style>In </div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Apr 26, 2013 at 1:11 AM, Shashank Sahni <span dir="ltr"><<a href="mailto:shredder12@gmail.com" target="_blank">shredder12@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi everyone,<br><br></div>I've been experimenting with using Swift(1.8) as Hadoop's DFS backend. <br>
<br><a href="https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration" target="_blank">https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration</a><br>
<br></div>After a few glitches, I'm able to create/access/delete objects/containers using hadoop's cli fs tool. But whenever I'm trying to run the job it fails with the following error.<br><br>ERROR security.UserGroupInformation: PriviledgedActionException
as:dharmesh cause:java.io.IOException: The ownership/permissions on the
staging directory
swift://hadooptest.rackspace/tmp/app/mapred/staging/dharmesh/.staging is
not as expected. It is owned by and permissions are rwxrwxrwx. The
directory must be owned by the submitter dharmesh or by dharmesh and
permissions must be rwx------<br><br></div>Note that, the local system username is "dharmesh" and the openstack account and associated tenant is "dharmesh" too.<br><br></div>I tried setting the permissions by creating "tmp" container using "swift post -r 'dharmesh:dharmesh'", but unfortunately ended up with the same result. Is there an other way to set rwx ACLs in swift?<br>
<div><div><br><div><div><div><div>--<br>Shashank Sahni<br></div>
</div></div></div></div></div></div>
<br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div>