Open Stack

On 01/16/2014 08:10 AM, Alexander Ignatov wrote:
> Matthew,
>
> I'm ok with proposed solution. Some comments/thoughts below:
>
> ---------
> FIX - @rest.post_file('/plugins/<plugin_name>/<version>/convert-config/<name>')
> - this is an RPC call, made only by a client to do input validation,
> move to POST /validations/plugins/:name/:version/check-config-import
> ---------
> AFAIR, this rest call was introduced not only for validation.
> The main idea was to create method which converts plugin specific config
> for cluster creation to savanna's cluster template [1]. So maybe we may change
> this rest call to: /plugins/convert-config/<name> and include all need fields
> to "data". Anyway we have to know Hortonworks guys opinion. Currently HDP
> plugin implements this method only.

The case of converting savanna cluster templates to plugin specific can 
be done internally, i.e. w/o exposing an API call. The Savanna API 
should talk savanna cluster templates only.

ACAICT, that leaves the validation justification for exposing it, so 
possibly a move to a /validations namespace.


> ------
> REMOVE - @rest.put('/node-group-templates/<node_group_template_id>') - Not Implemented
> REMOVE - @rest.put('/cluster-templates/<cluster_template_id>') - Not Implemented
> ------
> Disagree with that. Samsung people did great job in both savanna/savanna-dashboard
> to make this implemented [2], [3]. We should leave and support these calls in savanna.

Absolutely. Now that they're implemented they should not be removed.


> ------
> CONSIDER rename /jobs -> /job-templates (consistent w/ cluster-templates & clusters)
> CONSIDER renaming /job-executions to /jobs
> -------
> Good idea!
>
> ------
> FIX - @rest.get('/jobs/config-hints/<job_type>') - should move to
> GET /plugins/<plugin_name>/<plugin_version>, similar to get_node_processes
> and get_required_image_tags
> ------
> Not sure if it should be plugin specific right now. EDP uses it to show some
> configs to users in the dashboard. it's just a cosmetic thing. Also when user
> starts define some configs for some job he might not define cluster yet and
> thus plugin to run this job. I think we should leave it as is and leave only
> abstract configs like Mapper/Reducer class and allow users to apply any
> key/value configs if needed.

FYI, the code contains comments suggesting it should be plugin specific.

https://github.com/openstack/savanna/blob/master/savanna/service/edp/workflow_creator/workflow_factory.py#L179

IMHO, the EDP should have no plugin specific dependencies.

If it currently does, we should look into why and see if we can't 
eliminate this entirely.


> -----
> CONSIDER REMOVING, MUST ALWAYS UPLOAD TO Swift FOR /job-binaries
> -----
> Disagree. It was discussed before starting EDP implementation that there are
> a lot of OS installations which don't have Swift deployed, and ability to run
> jobs using savanna internal db is a good option in this case. But yes, Swift
> is more preferred. Waiting for Trevor's and maybe Nadya's comments here under
> this section.

While it's true that you can deploy OS w/o Swift, it's ok for us to 
start preferring deployments w/ Swift.


Best,


matt

> ----
> REMOVE - @rest.get('/job-executions/<job_execution_id>/refresh-status') - refresh
> and return status - GET should not side-effect, status is part of details and
> updated periodically, currently unused
> ----
> This call goes to Oozie directly to ask it about job status. It allows not to wait
> too long when periodic task will update status JobExecution object in Savanna.
> The current GET asks status of JobExecution from savanna-db. I think we can
> leave this call, it might be useful for external clients.
>
> ----
> REMOVE - @rest.get('/job-executions/<job_execution_id>/cancel') - cancel
> job-execution - GET should not side-effect, currently unused,
> use DELETE /job/executions/<job_execution_id>
> ----
> Disagree. We have to leave this call. This methods stops job executing on the
> Hadoop cluster but doesn't remove all its related info from savanna-db.
> DELETE removes it completely.
>
> [1] http://docs.openstack.org/developer/savanna/devref/plugin.spi.html#convert-config-plugin-name-version-template-name-cluster-template-create
> [2] https://blueprints.launchpad.net/savanna/+spec/modifying-cluster-template
> [3] https://blueprints.launchpad.net/savanna/+spec/modifying-node-group-template
>
> Regards,
> Alexander Ignatov
>
>
>
> On 14 Jan 2014, at 21:24, Matthew Farrellee <matt at redhat.com> wrote:
>
>> https://blueprints.launchpad.net/savanna/+spec/v2-api
>>
>> I've finished a review of the v1.0 and v1.1 APIs with an eye to making them more consistent and RESTful.
>>
>> Please use this thread to comment on my suggestions for v1.0 & v1.1, or to make further suggestions.
>>
>> Best,
>>
>>
>> matt
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>