Open Stack

Inlined.

Thanks,
Andrew.


On Sun, Jan 19, 2014 at 7:50 AM, Matthew Farrellee <matt at redhat.com> wrote:

> On 01/16/2014 08:10 AM, Alexander Ignatov wrote:
>
>> Matthew,
>>
>> I'm ok with proposed solution. Some comments/thoughts below:
>>
>> ---------
>> FIX - @rest.post_file('/plugins/<plugin_name>/<version>/
>> convert-config/<name>')
>> - this is an RPC call, made only by a client to do input validation,
>> move to POST /validations/plugins/:name/:version/check-config-import
>> ---------
>> AFAIR, this rest call was introduced not only for validation.
>> The main idea was to create method which converts plugin specific config
>> for cluster creation to savanna's cluster template [1]. So maybe we may
>> change
>> this rest call to: /plugins/convert-config/<name> and include all need
>> fields
>> to "data". Anyway we have to know Hortonworks guys opinion. Currently HDP
>> plugin implements this method only.
>>
>
> The case of converting savanna cluster templates to plugin specific can be
> done internally, i.e. w/o exposing an API call. The Savanna API should talk
> savanna cluster templates only.
>
> ACAICT, that leaves the validation justification for exposing it, so
> possibly a move to a /validations namespace.
>

[AL] +1 on moving to /validations


>
>  ------
>> REMOVE - @rest.put('/node-group-templates/<node_group_template_id>') -
>> Not Implemented
>> REMOVE - @rest.put('/cluster-templates/<cluster_template_id>') - Not
>> Implemented
>> ------
>> Disagree with that. Samsung people did great job in both
>> savanna/savanna-dashboard
>> to make this implemented [2], [3]. We should leave and support these
>> calls in savanna.
>>
>
> Absolutely. Now that they're implemented they should not be removed.
>
>
>  ------
>> CONSIDER rename /jobs -> /job-templates (consistent w/ cluster-templates
>> & clusters)
>> CONSIDER renaming /job-executions to /jobs
>> -------
>> Good idea!
>>
>> ------
>> FIX - @rest.get('/jobs/config-hints/<job_type>') - should move to
>> GET /plugins/<plugin_name>/<plugin_version>, similar to
>> get_node_processes
>> and get_required_image_tags
>> ------
>> Not sure if it should be plugin specific right now. EDP uses it to show
>> some
>> configs to users in the dashboard. it's just a cosmetic thing. Also when
>> user
>> starts define some configs for some job he might not define cluster yet
>> and
>> thus plugin to run this job. I think we should leave it as is and leave
>> only
>> abstract configs like Mapper/Reducer class and allow users to apply any
>> key/value configs if needed.
>>
>
> FYI, the code contains comments suggesting it should be plugin specific.
>
> https://github.com/openstack/savanna/blob/master/savanna/
> service/edp/workflow_creator/workflow_factory.py#L179
>
> IMHO, the EDP should have no plugin specific dependencies.
>
> If it currently does, we should look into why and see if we can't
> eliminate this entirely.
>
> [AL] EDP uses plugins in two ways:
1. for HDFS user
2. for config hints
I think both items should not be plugin specific on EDP API level. But
implementation should go to plugin and call plugin API for result.


>
>  -----
>> CONSIDER REMOVING, MUST ALWAYS UPLOAD TO Swift FOR /job-binaries
>> -----
>> Disagree. It was discussed before starting EDP implementation that there
>> are
>> a lot of OS installations which don't have Swift deployed, and ability to
>> run
>> jobs using savanna internal db is a good option in this case. But yes,
>> Swift
>> is more preferred. Waiting for Trevor's and maybe Nadya's comments here
>> under
>> this section.
>>
>
> While it's true that you can deploy OS w/o Swift, it's ok for us to start
> preferring deployments w/ Swift.
>
> [AL] +1 on change default value in horizon, but don't change anything in
API.


>
> Best,
>
>
> matt
>
>  ----
>> REMOVE - @rest.get('/job-executions/<job_execution_id>/refresh-status')
>> - refresh
>> and return status - GET should not side-effect, status is part of details
>> and
>> updated periodically, currently unused
>> ----
>> This call goes to Oozie directly to ask it about job status. It allows
>> not to wait
>> too long when periodic task will update status JobExecution object in
>> Savanna.
>> The current GET asks status of JobExecution from savanna-db. I think we
>> can
>> leave this call, it might be useful for external clients.
>>
>> ----
>> REMOVE - @rest.get('/job-executions/<job_execution_id>/cancel') - cancel
>> job-execution - GET should not side-effect, currently unused,
>> use DELETE /job/executions/<job_execution_id>
>> ----
>> Disagree. We have to leave this call. This methods stops job executing on
>> the
>> Hadoop cluster but doesn't remove all its related info from savanna-db.
>> DELETE removes it completely.
>>
>> [1] http://docs.openstack.org/developer/savanna/devref/
>> plugin.spi.html#convert-config-plugin-name-version-template-name-cluster-
>> template-create
>> [2] https://blueprints.launchpad.net/savanna/+spec/modifying-
>> cluster-template
>> [3] https://blueprints.launchpad.net/savanna/+spec/modifying-
>> node-group-template
>>
>> Regards,
>> Alexander Ignatov
>>
>>
>>
>> On 14 Jan 2014, at 21:24, Matthew Farrellee <matt at redhat.com> wrote:
>>
>>  https://blueprints.launchpad.net/savanna/+spec/v2-api
>>>
>>> I've finished a review of the v1.0 and v1.1 APIs with an eye to making
>>> them more consistent and RESTful.
>>>
>>> Please use this thread to comment on my suggestions for v1.0 & v1.1, or
>>> to make further suggestions.
>>>
>>> Best,
>>>
>>>
>>> matt
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140119/b8deca85/attachment.html>