<div dir="ltr">Started a new Neutron-specific thread so we can get some stuff turned into improvements in Neutron from this: <a href="http://lists.openstack.org/pipermail/openstack-dev/2017-May/117112.html">http://lists.openstack.org/pipermail/openstack-dev/2017-May/117112.html</a></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 19, 2017 at 1:05 PM, Zane Bitter <span dir="ltr"><<a href="mailto:zbitter@redhat.com" target="_blank">zbitter@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 19/05/17 15:06, Kevin Benton wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Don't even get me started on Neutron.[2]<br>
</blockquote>
<br>
It seems to me the conclusion to that thread was that the majority of<br>
your issues stemmed from the fact that we had poor documentation at the<br>
time. A major component of the complaints resulted from you<br>
misunderstanding the difference between networks/subnets in Neutron.<br>
</blockquote>
<br></span>
It's true that I was completely off base as to what the various primitives in Neutron actually do. (Thanks for educating me!) The implications for orchestration are largely unchanged though. It's a giant pain that we have to infer implicit dependencies between stuff to get them to create/delete in the right order, pretty much independently of what that stuff does.<br>
<br>
So knowing now that a Network is a layer-2 network segment and a Subnet is... effectively a glorified DHCP address pool, I understand better why it probably seemed like a good idea to hook stuff up magically. But at the end of the day, I still can't create a Port until a Subnet exists, I still don't know what Subnet a Port will be attached to (unless the user specifies it explicitly using the --fixed-ip option... regardless of whether they actually specify a fixed IP), and I have no way in general of telling which Subnets can be deleted before a given Port is and which will fail to delete until the Port disappears.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
There are some legitimate issues in there about the extra routes<br>
extension being replace-only and the routers API not accepting a list of<br>
interfaces in POST. However, it hardly seems that those are worthy of<br>
"Don't even get me started on Neutron."<br>
</blockquote>
<br>
</span><a href="https://launchpad.net/bugs/1626607" rel="noreferrer" target="_blank">https://launchpad.net/bugs/162<wbr>6607</a><br>
<a href="https://launchpad.net/bugs/1442121" rel="noreferrer" target="_blank">https://launchpad.net/bugs/144<wbr>2121</a><br>
<a href="https://launchpad.net/bugs/1626619" rel="noreferrer" target="_blank">https://launchpad.net/bugs/162<wbr>6619</a><br>
<a href="https://launchpad.net/bugs/1626630" rel="noreferrer" target="_blank">https://launchpad.net/bugs/162<wbr>6630</a><br>
<a href="https://launchpad.net/bugs/1626634" rel="noreferrer" target="_blank">https://launchpad.net/bugs/162<wbr>6634</a><span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
It would be nice if you could write up something about current gaps that<br>
would make Heat's life easier, because a large chunk of that initial<br>
email is incorrect and linking to it as a big list of "issues" is<br>
counter-productive.<br>
</blockquote>
<br></span>
Yes, agreed. I wish I had a clean thread to link to. It's a huge amount of work to research it all though.<br>
<br>
cheers,<br>
Zane.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
On Fri, May 19, 2017 at 7:36 AM, Zane Bitter <<a href="mailto:zbitter@redhat.com" target="_blank">zbitter@redhat.com</a><br></span><div><div class="h5">
<mailto:<a href="mailto:zbitter@redhat.com" target="_blank">zbitter@redhat.com</a>>> wrote:<br>
<br>
On 18/05/17 20:19, Matt Riedemann wrote:<br>
<br>
I just wanted to blurt this out since it hit me a few times at the<br>
summit, and see if I'm misreading the rooms.<br>
<br>
For the last few years, Nova has pushed back on adding<br>
orchestration to<br>
the compute API, and even define a policy for it since it comes<br>
up so<br>
much [1]. The stance is that the compute API should expose<br>
capabilities<br>
that a higher-level orchestration service can stitch together<br>
for a more<br>
fluid end user experience.<br>
<br>
<br>
I think this is a wise policy.<br>
<br>
One simple example that comes up time and again is allowing a<br>
user to<br>
pass volume type to the compute API when booting from volume<br>
such that<br>
when nova creates the backing volume in Cinder, it passes<br>
through the<br>
volume type. If you need a non-default volume type for boot from<br>
volume,<br>
the way you do this today is first create the volume with said<br>
type in<br>
Cinder and then provide that volume to the compute API when<br>
creating the<br>
server. However, people claim that is bad UX or hard for users to<br>
understand, something like that (at least from a command line, I<br>
assume<br>
Horizon hides this, and basic users should probably be using Horizon<br>
anyway right?).<br>
<br>
<br>
As always, there's a trade-off between simplicity and flexibility. I<br>
can certainly understand the logic in wanting to make the simple<br>
stuff simple. But users also need to be able to progress from simple<br>
stuff to more complex stuff without having to give up and start<br>
over. There's a danger of leading them down the garden path.<br>
<br>
While talking about claims in the scheduler and a top-level<br>
conductor<br>
for cells v2 deployments, we've talked about the desire to eliminate<br>
"up-calls" from the compute service to the top-level controller<br>
services<br>
(nova-api, nova-conductor and nova-scheduler). Build retries is<br>
one such<br>
up-call. CERN disables build retries, but others rely on them,<br>
because<br>
of how racy claims in the computes are (that's another story and why<br>
we're working on fixing it). While talking about this, we asked,<br>
"why<br>
not just do away with build retries in nova altogether? If the<br>
scheduler<br>
picks a host and the build fails, it fails, and you have to<br>
retry/rebuild/delete/recreate from a top-level service."<br>
<br>
<br>
(FWIW Heat does this for you already.)<br>
<br>
But during several different Forum sessions, like user API<br>
improvements<br>
[2] but also the cells v2 and claims in the scheduler sessions,<br>
I was<br>
hearing about how operators only wanted to expose the base IaaS<br>
services<br>
and APIs and end API users wanted to only use those, which means any<br>
improvements in those APIs would have to be in the base APIs (nova,<br>
cinder, etc). To me, that generally means any orchestration<br>
would have<br>
to be baked into the compute API if you're not using Heat or<br>
something<br>
similar.<br>
<br>
<br>
The problem is that orchestration done inside APIs is very easy to<br>
do badly in ways that cause lots of downstream pain for users and<br>
external orchestrators. For example, Nova already does some<br>
orchestration: it creates a Neutron port for a server if you don't<br>
specify one. (And then promptly forgets that it has done so.) There<br>
is literally an entire inner platform, an orchestrator within an<br>
orchestrator, inside Heat to try to manage the fallout from this.<br>
And the inner platform shares none of the elegance, such as it is,<br>
of Heat itself, but is rather a collection of cobbled-together hacks<br>
to deal with the seemingly infinite explosion of edge cases that we<br>
kept running into over a period of at least 5 releases.<br>
<br>
The get-me-a-network thing is... better, but there's no provision<br>
for changes after the server is created, which means we have to<br>
copy-paste the Nova implementation into Heat to deal with update.[1]<br>
Which sounds like a maintenance nightmare in the making. That seems<br>
to be a common mistake: to assume that once users create something<br>
they'll never need to touch it again, except to delete it when<br>
they're done.<br>
<br>
Don't even get me started on Neutron.[2]<br>
<br>
Any orchestration that is done behind-the-scenes needs to be done<br>
superbly well, provide transparency for external orchestration tools<br>
that need to hook in to the data flow, and should be developed in<br>
consultation with potential consumers like Shade and Heat.<br>
<br>
Am I missing the point, or is the pendulum really swinging away from<br>
PaaS layer services which abstract the dirty details of the<br>
lower-level<br>
IaaS APIs? Or was this always something people wanted and I've just<br>
never made the connection until now?<br>
<br>
<br>
(Aside: can we stop using the term 'PaaS' to refer to "everything<br>
that Nova doesn't do"? This habit is not helping us to communicate<br>
clearly.)<br>
<br>
cheers,<br>
Zane.<br>
<br>
[1] <a href="https://review.openstack.org/#/c/407328/" rel="noreferrer" target="_blank">https://review.openstack.org/#<wbr>/c/407328/</a><br>
<<a href="https://review.openstack.org/#/c/407328/" rel="noreferrer" target="_blank">https://review.openstack.org/<wbr>#/c/407328/</a>><br>
[2]<br>
<a href="http://lists.openstack.org/pipermail/openstack-dev/2014-April/032098.html" rel="noreferrer" target="_blank">http://lists.openstack.org/pip<wbr>ermail/openstack-dev/2014-Apri<wbr>l/032098.html</a><br>
<<a href="http://lists.openstack.org/pipermail/openstack-dev/2014-April/032098.html" rel="noreferrer" target="_blank">http://lists.openstack.org/pi<wbr>permail/openstack-dev/2014-Apr<wbr>il/032098.html</a>><br>
<br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br></div></div>
<<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@<wbr>lists.openstack.org?subject:un<wbr>subscribe</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
<<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cg<wbr>i-bin/mailman/listinfo/opensta<wbr>ck-dev</a>><span class=""><br>
<br>
<br>
<br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
<br>
</span></blockquote><div class="HOEnZb"><div class="h5">
<br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
</div></div></blockquote></div><br></div>