Open Stack

Hi folks,

  Here is a summary of priorities from Summit and some action items for
high the high priority issues.  The link to the pad is here:

https://etherpad.openstack.org/p/juno-summit-sahara-edp

  We really did not have any leftover questions from summit, but we need
investigation and development work in several areas.  Please respond with any 
comments/questions, and feel free to work on action items :)

High priority
    Fix hive support
    minimal EDP for spark via spark plugin (may be possible w Oozie)
    Design pluggable job model and investigate Spark / Storm integration

Medium priority
    Error reporting improvements

Low priority
    Raw Oozie workflows
    coordinated jobs
    preparation tags for workflows
    files and archives tags (need clear use cases)
    streamline copying of job binaries from swift to hdfs (dscp)

Action items for high priority issues:
    Hive:
        We need to flesh out existing blueprints:
            https://blueprints.launchpad.net/sahara/+spec/hive-vanilla2-support
            https://blueprints.launchpad.net/sahara/+spec/hive-integration-tests
        Additional blueprint needed for swift support in Hive
        Hive is not fully implemented in HDP plugin

    Investigate Spark job execution via Oozie
        Underway.  Produce a blueprint after initial investigation.

    Design a pluggable job model
        We need to review current EDP and think about how operations can be
        abstracted.  For instance, what are the essential operations on a job,
        and where has the Oozie implementation leaked knowledge into the EDP code?

        Can we develop an abstraction that maps to other models in a believable way?
           storm, spark, scalding, others

        How will the UI deal with a pluggable job model assuming we have one?