<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, Jan 19, 2014 at 7:53 AM, Matthew Farrellee <span dir="ltr"><<a href="mailto:matt@redhat.com" target="_blank">matt@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 01/16/2014 09:19 PM, Andrey Lazarev wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
My 5 cents:<br>
<br>
------<br>
REMOVE - @rest.put('/node-group-<u></u>templates/<node_group_<u></u>template_id>') -<br>
Not Implemented<br>
REMOVE - @rest.put('/cluster-templates/<u></u><cluster_template_id>') - Not<br>
Implemented<br>
------<br>
Disagree with that. Samsung people did great job in both<br>
savanna/savanna-dashboard<br>
to make this implemented [2], [3]. We should leave and support these<br>
calls in savanna.<br>
------<br>
[AL] Agree with Alexander. Ability to modify templates is very useful<br>
feature.<br>
<br>
<br>
----<br>
REMOVE - @rest.get('/job-executions/<<u></u>job_execution_id>/refresh-<u></u>status')<br>
- refresh<br>
and return status - GET should not side-effect, status is part of<br>
details and<br>
updated periodically, currently unused<br>
----<br>
This call goes to Oozie directly to ask it about job status. It allows<br>
not to wait<br>
too long when periodic task will update status JobExecution object in<br>
Savanna.<br>
The current GET asks status of JobExecution from savanna-db. I think we can<br>
leave this call, it might be useful for external clients.<br>
----<br>
[AL] Agree that GET shouldn't have side effect (or at least documented<br>
side effect). I think it could be generic PUT on<br>
'/job-executions/<job_<u></u>execution_id>' which can refresh status or cancel<br>
job on hadoop side.<br>
</blockquote>
<br>
>From what I can tell, this endpoint is not exposed by the savannaclient or used directly from the horizon plugin.<br>
<br>
I imagine that having a "savanna-api, please go faster" call is enticing, but if we're not using it yet, let's make sure we have a well defined need before adding/keeping it.<br></blockquote><div> </div>
<div>[AL] I like to disable 'periodic' in dev environment. And this is the only way to update job status without periodic. </div><div>So, I vote on adding it to savannaclient and to horizon.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
----<br>
REMOVE - @rest.get('/job-executions/<<u></u>job_execution_id>/cancel') - cancel<br>
job-execution - GET should not side-effect, currently unused,<br>
use DELETE /job/executions/<job_<u></u>execution_id><br>
----<br>
Disagree. We have to leave this call. This methods stops job executing<br>
on the<br>
Hadoop cluster but doesn't remove all its related info from savanna-db.<br>
DELETE removes it completely.<br>
----<br>
[AL] We need 'cancel'. Vote on generic PUT (see previous item).<br>
</blockquote>
<br>
AFAICT, this is also not used. Where is the need?<br></blockquote><div><br></div><div>[AL] I can easily imagine scenario where canceling is useful.</div><div><br></div><div>Both features give some benefit, but not extremely needed. So, it is a question of priorities. My vote is on leaving both of them. </div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
Best,<br>
<br>
<br>
matt<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks,<br>
Andrew.<br>
<br>
<br>
On Thu, Jan 16, 2014 at 5:10 AM, Alexander Ignatov<br>
<<a href="mailto:aignatov@mirantis.com" target="_blank">aignatov@mirantis.com</a> <mailto:<a href="mailto:aignatov@mirantis.com" target="_blank">aignatov@mirantis.com</a>><u></u>> wrote:<br>
<br>
Matthew,<br>
<br>
I'm ok with proposed solution. Some comments/thoughts below:<br>
<br>
---------<br>
FIX -<br>
@rest.post_file('/plugins/<<u></u>plugin_name>/<version>/<u></u>convert-config/<name>')<br>
- this is an RPC call, made only by a client to do input validation,<br>
move to POST /validations/plugins/:name/:<u></u>version/check-config-import<br>
---------<br>
AFAIR, this rest call was introduced not only for validation.<br>
The main idea was to create method which converts plugin specific config<br>
for cluster creation to savanna's cluster template [1]. So maybe we<br>
may change<br>
this rest call to: /plugins/convert-config/<name> and include all<br>
need fields<br>
to "data". Anyway we have to know Hortonworks guys opinion.<br>
Currently HDP<br>
plugin implements this method only.<br>
<br>
------<br>
REMOVE - @rest.put('/node-group-<u></u>templates/<node_group_<u></u>template_id>')<br>
- Not Implemented<br>
REMOVE - @rest.put('/cluster-templates/<u></u><cluster_template_id>') - Not<br>
Implemented<br>
------<br>
Disagree with that. Samsung people did great job in both<br>
savanna/savanna-dashboard<br>
to make this implemented [2], [3]. We should leave and support these<br>
calls in savanna.<br>
<br>
------<br>
CONSIDER rename /jobs -> /job-templates (consistent w/<br>
cluster-templates & clusters)<br>
CONSIDER renaming /job-executions to /jobs<br>
-------<br>
Good idea!<br>
<br>
------<br>
FIX - @rest.get('/jobs/config-hints/<u></u><job_type>') - should move to<br>
GET /plugins/<plugin_name>/<<u></u>plugin_version>, similar to<br>
get_node_processes<br>
and get_required_image_tags<br>
------<br>
Not sure if it should be plugin specific right now. EDP uses it to<br>
show some<br>
configs to users in the dashboard. it's just a cosmetic thing. Also<br>
when user<br>
starts define some configs for some job he might not define cluster<br>
yet and<br>
thus plugin to run this job. I think we should leave it as is and<br>
leave only<br>
abstract configs like Mapper/Reducer class and allow users to apply any<br>
key/value configs if needed.<br>
<br>
-----<br>
CONSIDER REMOVING, MUST ALWAYS UPLOAD TO Swift FOR /job-binaries<br>
-----<br>
Disagree. It was discussed before starting EDP implementation that<br>
there are<br>
a lot of OS installations which don't have Swift deployed, and<br>
ability to run<br>
jobs using savanna internal db is a good option in this case. But<br>
yes, Swift<br>
is more preferred. Waiting for Trevor's and maybe Nadya's comments<br>
here under<br>
this section.<br>
<br>
----<br>
REMOVE -<br>
@rest.get('/job-executions/<<u></u>job_execution_id>/refresh-<u></u>status') - refresh<br>
and return status - GET should not side-effect, status is part of<br>
details and<br>
updated periodically, currently unused<br>
----<br>
This call goes to Oozie directly to ask it about job status. It<br>
allows not to wait<br>
too long when periodic task will update status JobExecution object<br>
in Savanna.<br>
The current GET asks status of JobExecution from savanna-db. I think<br>
we can<br>
leave this call, it might be useful for external clients.<br>
<br>
----<br>
REMOVE - @rest.get('/job-executions/<<u></u>job_execution_id>/cancel') - cancel<br>
job-execution - GET should not side-effect, currently unused,<br>
use DELETE /job/executions/<job_<u></u>execution_id><br>
----<br>
Disagree. We have to leave this call. This methods stops job<br>
executing on the<br>
Hadoop cluster but doesn't remove all its related info from savanna-db.<br>
DELETE removes it completely.<br>
<br>
[1]<br>
<a href="http://docs.openstack.org/developer/savanna/devref/plugin.spi.html#convert-config-plugin-name-version-template-name-cluster-template-create" target="_blank">http://docs.openstack.org/<u></u>developer/savanna/devref/<u></u>plugin.spi.html#convert-<u></u>config-plugin-name-version-<u></u>template-name-cluster-<u></u>template-create</a><br>
[2]<br>
<a href="https://blueprints.launchpad.net/savanna/+spec/modifying-cluster-template" target="_blank">https://blueprints.launchpad.<u></u>net/savanna/+spec/modifying-<u></u>cluster-template</a><br>
[3]<br>
<a href="https://blueprints.launchpad.net/savanna/+spec/modifying-node-group-template" target="_blank">https://blueprints.launchpad.<u></u>net/savanna/+spec/modifying-<u></u>node-group-template</a><br>
<br>
Regards,<br>
Alexander Ignatov<br>
<br>
<br>
<br>
On 14 Jan 2014, at 21:24, Matthew Farrellee <<a href="mailto:matt@redhat.com" target="_blank">matt@redhat.com</a><br>
<mailto:<a href="mailto:matt@redhat.com" target="_blank">matt@redhat.com</a>>> wrote:<br>
<br>
> <a href="https://blueprints.launchpad.net/savanna/+spec/v2-api" target="_blank">https://blueprints.launchpad.<u></u>net/savanna/+spec/v2-api</a><br>
><br>
> I've finished a review of the v1.0 and v1.1 APIs with an eye to<br>
making them more consistent and RESTful.<br>
><br>
> Please use this thread to comment on my suggestions for v1.0 &<br>
v1.1, or to make further suggestions.<br>
><br>
> Best,<br>
><br>
><br>
> matt<br>
><br>
> ______________________________<u></u>_________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<mailto:<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.<u></u>openstack.org</a>><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
<br>
______________________________<u></u>_________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<mailto:<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.<u></u>openstack.org</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
<br>
<br>
<br>
______________________________<u></u>_________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
<br>
</blockquote>
<br>
<br>
______________________________<u></u>_________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.<u></u>org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-dev</a><br>
</blockquote></div><br></div></div>