<div dir="ltr">Hi Serg,<div><br></div><div>We were indeed hitting that bug, but the cert wasn't self-signed. It was easier for us to manually patch the Ubuntu Cloud package of Murano with the stable/mitaka fix linked in that bug report than trying to debug where OpenSSL/python/requests/etc was going awry.</div><div><br></div><div>We might redeploy Murano strictly using virtualenv's and pip so we stay on the latest stable patches.</div><div><br></div><div>Thanks,</div><div>Joe</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Sep 26, 2016 at 11:03 PM, Serg Melikyan <span dir="ltr"><<a href="mailto:smelikyan@mirantis.com" target="_blank">smelikyan@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Joe,<br>
<span class=""><br>
>Also, is it safe to say that communication between agent/engine only, and will only, happen during app deployment?<br>
<br>
</span>murano-agent & murano-engine keep active connection to the Rabbit MQ<br>
broker but message exchange happens only during deployment of the app.<br>
<span class=""><br>
>One thing we just ran into, though, was getting the agent/engine rmq config to work with SSL<br>
<br>
</span>We had related bug fixed in Newton, can you confirm that you are *not*<br>
hitting bug #1578421 [0]<br>
<br>
References:<br>
[0] <a href="https://bugs.launchpad.net/murano/+bug/1578421" rel="noreferrer" target="_blank">https://bugs.launchpad.net/<wbr>murano/+bug/1578421</a><br>
<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
On Mon, Sep 26, 2016 at 1:43 PM, Andrew Woodward <<a href="mailto:xarses@gmail.com">xarses@gmail.com</a>> wrote:<br>
> In Fuel we deploy haproxy to all of the nodes that are part of the<br>
> VIP/endpoint service (This is usually part of the controller role) Then the<br>
> vips (internal or public) can be active on any member of the group.<br>
> Corosync/Pacemaker is used to move the VIP address (as apposed to<br>
> keepalived) in our case both haproxy, and the vip live in a namespace and<br>
> haproxy is always running on all of these nodes bound to 0/0.<br>
><br>
> In the case of murano-rabbit we take the same approach as we do for galera,<br>
> all of the members are listed in the balancer, but with the others as<br>
> backup's this makes them inactive until the first node is down. This allow<br>
> the vip to move to any of the proxies in the cluster, and continue to direct<br>
> traffic to the same node util that rabbit instance is also un-available<br>
><br>
> isten mysqld<br>
> bind <a href="http://192.168.0.2:3306" rel="noreferrer" target="_blank">192.168.0.2:3306</a><br>
> mode tcp<br>
> option httpchk<br>
> option tcplog<br>
> option clitcpka<br>
> option srvtcpka<br>
> stick on dst<br>
> stick-table type ip size 1<br>
> timeout client 28801s<br>
> timeout server 28801s<br>
> server node-1 <a href="http://192.168.0.4:3307" rel="noreferrer" target="_blank">192.168.0.4:3307</a> check port 49000 inter 20s fastinter 2s<br>
> downinter 2s rise 3 fall 3<br>
> server node-3 <a href="http://192.168.0.6:3307" rel="noreferrer" target="_blank">192.168.0.6:3307</a> backup check port 49000 inter 20s fastinter<br>
> 2s downinter 2s rise 3 fall 3<br>
> server node-4 <a href="http://192.168.0.5:3307" rel="noreferrer" target="_blank">192.168.0.5:3307</a> backup check port 49000 inter 20s fastinter<br>
> 2s downinter 2s rise 3 fall 3<br>
><br>
> listen murano_rabbitmq<br>
> bind <a href="http://10.110.3.3:55572" rel="noreferrer" target="_blank">10.110.3.3:55572</a><br>
> balance roundrobin<br>
> mode tcp<br>
> option tcpka<br>
> timeout client 48h<br>
> timeout server 48h<br>
> server node-1 <a href="http://192.168.0.4:55572" rel="noreferrer" target="_blank">192.168.0.4:55572</a> check inter 5000 rise 2 fall 3<br>
> server node-3 <a href="http://192.168.0.6:55572" rel="noreferrer" target="_blank">192.168.0.6:55572</a> backup check inter 5000 rise 2 fall 3<br>
> server node-4 <a href="http://192.168.0.5:55572" rel="noreferrer" target="_blank">192.168.0.5:55572</a> backup check inter 5000 rise 2 fall 3<br>
><br>
><br>
> On Fri, Sep 23, 2016 at 7:30 AM Mike Lowe <<a href="mailto:jomlowe@iu.edu">jomlowe@iu.edu</a>> wrote:<br>
>><br>
>> Would you mind sharing an example snippet from HA proxy config? I had<br>
>> struggled in the past with getting this part to work.<br>
>><br>
>><br>
>> > On Sep 23, 2016, at 12:13 AM, Serg Melikyan <<a href="mailto:smelikyan@mirantis.com">smelikyan@mirantis.com</a>><br>
>> > wrote:<br>
>> ><br>
>> > Hi Joe,<br>
>> ><br>
>> > I can share some details on how murano is configured as part of the<br>
>> > default Mirantis OpenStack configuration and try to explain why it's<br>
>> > done in that way as it's done, I hope it helps you in your case.<br>
>> ><br>
>> > As part of Mirantis OpenStack second instance of the RabbitMQ is<br>
>> > getting deployed specially for the murano, but it's configuration is<br>
>> > different than for the RabbitMQ instance used by the other OpenStack<br>
>> > components.<br>
>> ><br>
>> > Why to use separate instance of the RabbitMQ?<br>
>> > 1. Prevent possibility to get access to the RabbitMQ supporting<br>
>> > whole cloud infrastructure by limiting access on the networking level<br>
>> > rather than rely on authentication/authorization<br>
>> > 2. Prevent possibility of DDoS by limiting access on the<br>
>> > networking level to the infrastructure RabbitMQ<br>
>> ><br>
>> > Given that second RabbitMQ instance is used only for the murano-agent<br>
>> > <-> murano-engine communications and murano-agent is running on the<br>
>> > VMs we had to make couple of changes in the deployment of the RabbitMQ<br>
>> > (bellow I am referencing RabbitMQ as RabbitMQ instance used by Murano<br>
>> > for m-agent <-> m-engine communications):<br>
>> ><br>
>> > 1. RabbitMQ is not clustered, just separate instance running on each<br>
>> > controller node<br>
>> > 2. RabbitMQ is exposed on the Public VIP where all OpenStack APIs are<br>
>> > exposed<br>
>> > 3. It's has different port number than default<br>
>> > 4. HAProxy is used, RabbitMQ is hidden behind it and HAProxy is always<br>
>> > pointing to the RabbitMQ on the current primary controller<br>
>> ><br>
>> > Note: How murano-agent is working? Murano-engine creates queue with<br>
>> > uniq name and put configuration tasks to that queue which are later<br>
>> > getting picked up by murano-agent when VM is booted and murano-agent<br>
>> > is configured to use created queue through cloud-init.<br>
>> ><br>
>> > #1 Clustering<br>
>> ><br>
>> > * Given that per 1 app deployment from we create 1-N VMs and send 1-M<br>
>> > configuration tasks, where in most of the cases N and M are less than<br>
>> > 3.<br>
>> > * Even if app deployment will be failed due to cluster failover it's<br>
>> > can be always re-deployed by the user.<br>
>> > * Controller-node failover most probably will lead to limited<br>
>> > accessibility of the Heat, Nova & Neutron API and application<br>
>> > deployment will fail regardless of the not executing configuration<br>
>> > task on the VM.<br>
>> ><br>
>> > #2 Exposure on the Public VIP<br>
>> ><br>
>> > One of the reasons behind choosing RabbitMQ as transport for<br>
>> > murano-agent communications was connectivity from the VM - it's much<br>
>> > easier to implement connectivity *from* the VM than *to* VM.<br>
>> ><br>
>> > But even in the case when you are connecting to the broker from the VM<br>
>> > you should have connectivity and public interface where all other<br>
>> > OpenStack APIs are exposed is most natural way to do that.<br>
>> ><br>
>> > #3 Different from the default port number<br>
>> ><br>
>> > Just to avoid confusion from the RabbitMQ used for the infrastructure,<br>
>> > even given that they are on the different networks.<br>
>> ><br>
>> > #4 HAProxy<br>
>> ><br>
>> > In case of the default Mirantis OpenStack configuration is used mostly<br>
>> > to support non-clustered RabbitMQ setup and exposure on the Public<br>
>> > VIP, but also helpful in case of more complicated setups.<br>
>> ><br>
>> > P.S. I hope my answers helped, let me know if I can cover something in<br>
>> > more details.<br>
>> > --<br>
>> > Serg Melikyan, Development Manager at Mirantis, Inc.<br>
>> > <a href="http://mirantis.com" rel="noreferrer" target="_blank">http://mirantis.com</a> | <a href="mailto:smelikyan@mirantis.com">smelikyan@mirantis.com</a><br>
>> ><br>
>> > ______________________________<wbr>_________________<br>
>> > OpenStack-operators mailing list<br>
>> > <a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.<wbr>openstack.org</a><br>
>> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-operators</a><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> OpenStack-operators mailing list<br>
>> <a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.<wbr>openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-operators</a><br>
><br>
> --<br>
> Andrew Woodward<br>
> Mirantis<br>
<br>
<br>
<br>
--<br>
Serg Melikyan, Development Manager at Mirantis, Inc.<br>
<a href="http://mirantis.com" rel="noreferrer" target="_blank">http://mirantis.com</a> | <a href="mailto:smelikyan@mirantis.com">smelikyan@mirantis.com</a><br>
</div></div></blockquote></div><br></div>