<div dir="ltr">Steve,<div><br></div><div>In Newton, we upgraded the Heat templates to use lbaasv2 [1]. However, it seems the k8s external load balancer still work with lbaasv1. If Kolla doesn't use the external load balancer feature, it should be fine.</div><div><br></div><div>[1] <a href="https://review.openstack.org/#/c/314060/">https://review.openstack.org/#/c/314060/</a></div><div><br></div><div>Best regards,</div><div>Hongbin<br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 28, 2016 at 8:06 PM, Steven Dake (stdake) <span dir="ltr"><<a href="mailto:stdake@cisco.com" target="_blank">stdake@cisco.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="white" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri">Fantastic!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri">Quick semi-related question. Will Magnum Newton be using lbaasv2? That is what we have implemented in Kolla.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri">Regards<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri">-steve<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt;font-family:calibri"><u></u> <u></u></span></p>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:4.5pt solid rgb(181,196,223);padding:0in 0in 0in 4pt;margin-left:3.75pt;margin-right:0in">
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0in 0in">
<p class="MsoNormal"><b><span style="font-family:calibri;color:black">From: </span>
</b><span style="font-family:calibri;color:black">Ton Ngo <<a href="mailto:ton@us.ibm.com" target="_blank">ton@us.ibm.com</a>><span class="gmail-"><br>
<b>Reply-To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><br>
</span><b>Date: </b>Tuesday, September 27, 2016 at 10:58 PM<span class="gmail-"><br>
<b>To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><br>
</span><span class="gmail-"><b>Subject: </b>Re: [openstack-dev] [magnum] Fedora Atomic image that supports kubernetes external load balancer (for stable/mitaka)<u></u><u></u></span></span></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p>Thanks Steve. We indeed have been using the image built by Yolanda's DIB elements and things have been stable. Dane and I have resolved the problems with the load balancer at least for the LBaaS v1. For LBaaS v2, we need to build a new image with Kubernetes
1.3 and we just got one built today.<br>
Ton,<br>
<br>
<img width="16" height="16" src="cid:image001.gif@01D219AA.99CCB470" alt="nactive hide details for "Steven Dake (stdake)" ---09/27/2016 10:18:07 PM"><span style="color:rgb(66,66,130)">"Steven Dake (stdake)" ---09/27/2016 10:18:07 PM---Dane, I’ve
heard Yolanda has done good work on making disk image builder build fedora atomic properl</span></p><div><div class="gmail-h5"><br>
<br>
<span style="font-size:10pt;color:rgb(95,95,95)">From: </span><span style="font-size:10pt">"Steven Dake (stdake)" <<a href="mailto:stdake@cisco.com" target="_blank">stdake@cisco.com</a>></span><br>
<span style="font-size:10pt;color:rgb(95,95,95)">To: </span><span style="font-size:10pt">"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>></span><br>
<span style="font-size:10pt;color:rgb(95,95,95)">Date: </span><span style="font-size:10pt">09/27/2016 10:18 PM</span><br>
<span style="font-size:10pt;color:rgb(95,95,95)">Subject: </span><span style="font-size:10pt">Re: [openstack-dev] [magnum] Fedora Atomic image that supports kubernetes external load balancer (for stable/mitaka)</span><u></u><u></u></div></div><p></p><div><div class="gmail-h5">
<div class="MsoNormal">
<hr size="2" width="100%" noshade style="color:rgb(128,145,165)" align="left">
</div>
<p class="MsoNormal"><br>
<br>
<br>
<span style="font-family:calibri">Dane,</span><br>
<br>
<span style="font-family:calibri">I’ve heard Yolanda has done good work on making disk image builder build fedora atomic properly consistently. This may work better than the current image building tools available with atomic if you need to roll your own. Might
try pinging her on irc for advice if you get jammed up here. Might consider consulting tango as well as I handed off my knowledge in this area to him first and he has distributed to the rest of the Magnum core reviewer team. I’m not sure if tango and Yolanda
have synced on this – recommend checking with them.</span><br>
<br>
<span style="font-family:calibri">Seems important to have a working atomic image for both Mitaka and Newton.</span><br>
<br>
<span style="font-family:calibri">Regards</span><br>
<span style="font-family:calibri">-steve</span><br>
<br>
<br>
<b><span style="font-family:calibri">From: </span></b><span style="font-family:calibri">"Dane Leblanc (leblancd)" <<a href="mailto:leblancd@cisco.com" target="_blank">leblancd@cisco.com</a>><b><br>
Reply-To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><b><br>
Date: </b>Thursday, September 8, 2016 at 2:18 PM<b><br>
To: </b>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><b><br>
Subject: </b>[openstack-dev] [magnum] Fedora Atomic image that supports kubernetes external load balancer (for stable/mitaka)</span><br>
<br>
<span style="font-family:calibri">Does anyone have a pointer to a Fedora Atomic image that works with stable/mitaka Magnum, and supports the kubernetes external load balancer feature [1]?</span><br>
<br>
<span style="font-family:calibri">I’m trying to test the kubernetes external load balancer feature with stable/mitaka Magnum. However, when I try to bring up a load-balanced service, I’m seeing these errors in the kube-controller-manager logs:</span>
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1in"><i><span style="font-family:tahoma">E0907 16:26:54.375286 1 servicecontroller.go:173] Failed to process service delta. Retrying: failed to create external load balancer for service default/nginx-service:
<span style="color:red">SubnetID is required</span></span></i><u></u><u></u></p>
<p class="MsoNormal"><br>
<span style="font-family:calibri">I verified that I have the subnet-id field set in the [LoadBalancer] section in /etc/sysconfig/kube_openstack_<wbr>config.</span><br>
<br>
<span style="font-family:calibri">I’ve tried this using the following Fedora Atomic images from [2]:</span>
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1in"><span style="font-family:calibri">fedora-21-atomic-5.qcow2</span><br>
<span style="font-family:calibri">fedora-21-atomic-6.qcow2</span><br>
<span style="font-family:calibri">fedora-atomic-latest.qcow2</span><u></u><u></u></p>
<p class="MsoNormal"><br>
<span style="font-family:calibri">According to the Magnum external load balancer blueprint [3], there were 3 patches in kubernetes that are required to get the OpenStack provider plugin to work in kubernetes:</span>
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1in"><a href="https://github.com/GoogleCloudPlatform/kubernetes/pull/12203" target="_blank"><span style="font-family:calibri">https://github.com/<wbr>GoogleCloudPlatform/<wbr>kubernetes/pull/12203</span></a><u><span style="font-family:calibri;color:blue"><br>
</span></u><a href="https://github.com/GoogleCloudPlatform/kubernetes/pull/12262" target="_blank"><span style="font-family:calibri">https://github.com/<wbr>GoogleCloudPlatform/<wbr>kubernetes/pull/12262</span></a><u><span style="font-family:calibri;color:blue"><br>
</span></u><a href="https://github.com/GoogleCloudPlatform/kubernetes/pull/12288" target="_blank"><span style="font-family:calibri">https://github.com/<wbr>GoogleCloudPlatform/<wbr>kubernetes/pull/12288</span></a><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:calibri">The first of these patches, “Pass SubnetID to vips.Create()”, is apparently necessary to fix the “SubnetID is required” error shown above.</span><br>
<br>
<span style="font-family:calibri">According to the Magnum external load balancer blueprint [3], the fedora-21-atomic-6 image should include the above 3 fixes:</span>
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1in"><span style="font-family:calibri">“<i>Our work-around is to use our own custom Kubernetes build (version 1.0.4 + 3 fixes) until the fixes are released. This is in image fedora-21-atomic-6.qcow2</i>”</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:calibri">However, I’m still seeing the “SubnetID is required” errors with this image downloaded from [2]. Here are the kube versions I’m seeing with this image:</span>
<u></u><u></u></p>
<p class="MsoNormal" style="margin-left:1in"><span style="font-family:calibri">[minion@k8-64n4bna2v6-0-<wbr>ffukgho7n7tf-kube-master-<wbr>fif5b6pivdmy sysconfig]$ rpm -qa | grep kube</span><br>
<span style="font-family:calibri">kubernetes-node-1.2.0-0.15.<wbr>alpha6.gitf0cd09a.fc23.x86_64</span><br>
<span style="font-family:calibri">kubernetes-1.2.0-0.15.alpha6.<wbr>gitf0cd09a.fc23.x86_64</span><br>
<span style="font-family:calibri">kubernetes-client-1.2.0-0.15.<wbr>alpha6.gitf0cd09a.fc23.x86_64</span><br>
<span style="font-family:calibri">kubernetes-master-1.2.0-0.15.<wbr>alpha6.gitf0cd09a.fc23.x86_64</span><br>
<span style="font-family:calibri">[minion@k8-64n4bna2v6-0-<wbr>ffukgho7n7tf-kube-master-<wbr>fif5b6pivdmy sysconfig]$</span><u></u><u></u></p>
<p class="MsoNormal" style="margin-bottom:12pt"><br>
<span style="font-family:calibri">Does anyone have a pointer to a Fedora Atomic image that contains the 3 kubernetes fixes listed earlier (and works with stable/mitaka)?</span><br>
<br>
<span style="font-family:calibri">Thanks!</span><br>
<span style="font-family:calibri">-Dane</span><br>
<br>
<span style="font-family:calibri">[1] </span><a href="http://kubernetes.io/docs/user-guide/services/#type-loadbalancer" target="_blank"><span style="font-family:calibri">http://kubernetes.io/docs/<wbr>user-guide/services/#type-<wbr>loadbalancer</span></a><br>
<span style="font-family:calibri">[2] </span><a href="https://fedorapeople.org/groups/magnum/" target="_blank"><span style="font-family:calibri">https://fedorapeople.org/<wbr>groups/magnum/</span></a><br>
<span style="font-family:calibri">[3] </span><a href="https://blueprints.launchpad.net/magnum/+spec/external-lb" target="_blank"><span style="font-family:calibri">https://blueprints.launchpad.<wbr>net/magnum/+spec/external-lb</span></a><br>
<tt><span style="font-size:10pt">______________________________<wbr>______________________________<wbr>______________</span></tt><span style="font-size:10pt;font-family:"courier new""><br>
<tt>OpenStack Development Mailing List (not for usage questions)</tt><br>
<tt>Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a></tt><br>
<tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a></tt><br>
</span><br>
<br>
<u></u><u></u></p>
</div></div></blockquote>
</div>
</div>
<br>______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br></blockquote></div><br></div></div></div>