[openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

Gil Vernik GILV at il.ibm.com
Thu Aug 28 17:28:55 UTC 2014


Hi Michael,

I have an update to  this patch with  temp auth authentication also, but 
it's not yet submitted.
I am not aware of v3 support.

All the best,
Gil.



From:   Michael McCune <mimccune at redhat.com>
To:     "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev at lists.openstack.org>
Date:   28/08/2014 08:14 PM
Subject:        Re: [openstack-dev] [sahara] Notes on developing Sahara 
Spark EDP to work with swift:// paths



hi Gil,

that's cool about the patch to Spark, has there been any talk about 
upgrading that patch to include Keystone v3 operations?

----- Original Message -----
> Hi,
> 
> In case this is helpful for you, this is the patch i submitted to Spark
> about Swift and Spark integration ( about to be merged )
> https://github.com/apache/spark/pull/1010
> 
> I sent information about this patch to this mailing list about two 
months
> ago.
> 
> All the best,
> Gil.
> 
> 
> 
> 
> 
> From:   Trevor McKay <tmckay at redhat.com>
> To:     OpenStack Development Mailing List
> <openstack-dev at lists.openstack.org>
> Date:   28/08/2014 06:22 PM
> Subject:        [openstack-dev] [sahara] Notes on developing Sahara 
Spark
> EDP to work with swift:// paths
> 
> 
> 
> Hi folks,
> 
>   I've updated this etherpad with notes from an investigation of
> Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra
> repo.
> 
>   Following the notes there, I was able to access swift:// paths from
> Spark jobs on a Spark standalone cluster launched from Sahara and then
> fixed up by hand.
> 
>   Comments welcome.  This is a POC at this point imho, we have work to
> do to fully integrate this into Sahara.
> 
> https://etherpad.openstack.org/p/sahara_spark_edp
> 
> Best,
> 
> Trevor
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140828/d10bab11/attachment.html>


More information about the OpenStack-dev mailing list