[openstack-dev] [sahara] Notes on developing Sahara Spark EDP to work with swift:// paths

Trevor McKay tmckay at redhat.com
Thu Aug 28 15:20:13 UTC 2014


Hi folks,

  I've updated this etherpad with notes from an investigation of
Spark/Swift and the hadoop-openstack plugin carried in the sahara-extra
repo.
  
  Following the notes there, I was able to access swift:// paths from
Spark jobs on a Spark standalone cluster launched from Sahara and then
fixed up by hand.

  Comments welcome.  This is a POC at this point imho, we have work to
do to fully integrate this into Sahara.

https://etherpad.openstack.org/p/sahara_spark_edp

Best,

Trevor




More information about the OpenStack-dev mailing list