Open Stack

Cool, thank you for wrapping up design summit session!

On Tue, Jun 3, 2014 at 7:24 PM, Trevor McKay <tmckay at redhat.com> wrote:
> Hi folks,
>
>   Here is a summary of priorities from Summit and some action items for
> high the high priority issues.  The link to the pad is here:
>
> https://etherpad.openstack.org/p/juno-summit-sahara-edp
>
>   We really did not have any leftover questions from summit, but we need
> investigation and development work in several areas.  Please respond with any
> comments/questions, and feel free to work on action items :)
>
> High priority
>     Fix hive support
>     minimal EDP for spark via spark plugin (may be possible w Oozie)
>     Design pluggable job model and investigate Spark / Storm integration
>
> Medium priority
>     Error reporting improvements
>
> Low priority
>     Raw Oozie workflows
>     coordinated jobs
>     preparation tags for workflows
>     files and archives tags (need clear use cases)
>     streamline copying of job binaries from swift to hdfs (dscp)
>
> Action items for high priority issues:
>     Hive:
>         We need to flesh out existing blueprints:
>             https://blueprints.launchpad.net/sahara/+spec/hive-vanilla2-support
>             https://blueprints.launchpad.net/sahara/+spec/hive-integration-tests
>         Additional blueprint needed for swift support in Hive
>         Hive is not fully implemented in HDP plugin
>
>     Investigate Spark job execution via Oozie
>         Underway.  Produce a blueprint after initial investigation.
>
>     Design a pluggable job model
>         We need to review current EDP and think about how operations can be
>         abstracted.  For instance, what are the essential operations on a job,
>         and where has the Oozie implementation leaked knowledge into the EDP code?
>
>         Can we develop an abstraction that maps to other models in a believable way?
>            storm, spark, scalding, others
>
>         How will the UI deal with a pluggable job model assuming we have one?
>
>
>
>
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Principal Software Engineer
Mirantis Inc.