Open Stack

Inlined.


On Mon, Jan 20, 2014 at 8:15 AM, Matthew Farrellee <matt at redhat.com> wrote:

> (inline, trying to make this readable by a text-only mail client that
> doesn't use tabs to indicate quoting)
>
> On 01/20/2014 02:50 AM, Andrey Lazarev wrote:
>
>          ------
>>         FIX - @rest.get('/jobs/config-hints/__<job_type>') - should move
>> to
>>         GET /plugins/<plugin_name>/<__plugin_version>, similar to
>>         get_node_processes
>>         and get_required_image_tags
>>         ------
>>         Not sure if it should be plugin specific right now. EDP uses it
>>         to show some
>>         configs to users in the dashboard. it's just a cosmetic thing.
>>         Also when user
>>         starts define some configs for some job he might not define
>>         cluster yet and
>>         thus plugin to run this job. I think we should leave it as is
>>         and leave only
>>         abstract configs like Mapper/Reducer class and allow users to
>>         apply any
>>         key/value configs if needed.
>>
>>
>>     FYI, the code contains comments suggesting it should be plugin
>> specific.
>>
>>     https://github.com/openstack/__savanna/blob/master/savanna/_
>> _service/edp/workflow_creator/__workflow_factory.py#L179
>>     <https://github.com/openstack/savanna/blob/master/savanna/
>> service/edp/workflow_creator/workflow_factory.py#L179>
>>
>>     IMHO, the EDP should have no plugin specific dependencies.
>>
>>     If it currently does, we should look into why and see if we can't
>>     eliminate this entirely.
>>
>> [AL] EDP uses plugins in two ways:
>> 1. for HDFS user
>> 2. for config hints
>> I think both items should not be plugin specific on EDP API level. But
>> implementation should go to plugin and call plugin API for result.
>>
>
> In fact they are both plugin specific. The user is forced to click through
> a plugin selection (when launching a job on transient cluster) or the
> plugin selection has already occurred (when launching a job on an existing
> cluster).
>
> Since the config is something that is plugin specific, you might not have
> hbase hints from vanilla but you would from hdp, and you already have
> plugin information whenever you ask for a hint, my view that this be under
> the /plugins namespace is growing stronger.
>

[AL] Disagree. They are plugin specific, but EDP itself could have
additional plugin-independent logic inside. Now config hints return EDP
properties (like mapred.input.dir) as well as plugin-specific properties.
Placing it under /plugins namespace will give a vision that it is fully
plugin specific.

I like to see EDP API fully plugin independent and in one workspace. If
core side needs some information internally it can easily go into the
plugin.


> Best,
>
>
> matt
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140120/bde98257/attachment.html>