<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" id="owaParaStyle">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Its probably captured in summit notes from 5-3 years ago. Nothing specific I can point at without going through a lot of archaeology.<br>
<br>
Yeah, deploying the users databases on top of kubernetes in vm's would be easier to upgrade I think then pure vm's with a pile of debs/rpms inside.<br>
<br>
Its tangentially related to the message bus stuff. If you solve the ddos attack issue with the message bus, you still have the upgrade problem. but depending on how you choose to solve the communications channel issues you can solve other issues such as upgrades
easier, harder, or not at all.<br>
<br>
Thanks,<br>
Kevin<br>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div id="divRpF607629" style="direction: ltr;"><font face="Tahoma" color="#000000" size="2"><b>From:</b> Darek Król [dkrol3@gmail.com]<br>
<b>Sent:</b> Tuesday, January 22, 2019 12:38 PM<br>
<b>To:</b> Fox, Kevin M<br>
<b>Cc:</b> Michael Richardson; openstack-discuss@lists.openstack.org<br>
<b>Subject:</b> Re: Subject: Re: [Trove] State of the Trove service tenant deployment model<br>
</font><br>
</div>
<div></div>
<div>
<div>
<div dir="auto">Is there any documentation written down from this discussion ? I would really like to read more about the problem and any ideas for possible solutions.</div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">Your recommendation abou k8s sounds interesting but I’m not sure if I understand it fully. Would you like to have a k8s cluster for all tenants on top of vms to handle trove instances ? And Is upgrade a different problem than ddos attack at
message bus ? </div>
<div dir="auto"><br>
</div>
<div dir="auto">Best,</div>
<div dir="auto">Darek</div>
<div><br>
<div class="gmail_quote">
<div dir="ltr">On Tue, 22 Jan 2019 at 21:25, Fox, Kevin M <<a href="mailto:Kevin.Fox@pnnl.gov" target="_blank" rel="noopener noreferrer">Kevin.Fox@pnnl.gov</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
We tried to solve it as a cross project issue for a while and then everyone gave up. lots of projects have the same problem. trove, sahara, magnum, etc.<br>
<br>
Other then just control messages, there is also the issue of version skew between guest agents and controllers and how to do rolling upgrades. Its messy today.<br>
<br>
I'd recommend at this point to maybe just run kubernetes across the vms and push the guest agents/workload to them. You can still drive it via an openstack api, but doing rolling upgrades of guest agents or mysql containers or whatever is way simpler for operators
to handle. We should embrace k8s as part of the solution rather then trying to reimplement it IMO.<br>
<br>
Thanks,<br>
Kevin<br>
________________________________________<br>
From: Darek Król [<a href="mailto:dkrol3@gmail.com" target="_blank" rel="noopener noreferrer">dkrol3@gmail.com</a>]<br>
Sent: Tuesday, January 22, 2019 12:09 PM<br>
To: Michael Richardson<br>
Cc: <a href="mailto:openstack-discuss@lists.openstack.org" target="_blank" rel="noopener noreferrer">
openstack-discuss@lists.openstack.org</a><br>
Subject: Subject: Re: [Trove] State of the Trove service tenant deployment model<br>
<br>
On Tue, Jan 22, 2019 at 07:29:25PM +1300, Zane Bitter wrote:<br>
> Last time I heard (which was probably mid-2017), the Trove team had<br>
> implemented encryption for messages on the RabbitMQ bus. IIUC each DB being<br>
> managed had its own encryption keys, so that would theoretically prevent<br>
> both snooping and spoofing of messages. That's the good news.<br>
><br>
> The bad news is that AFAIK it's still using a shared RabbitMQ bus, so<br>
> attacks like denial of service are still possible if you can extract the<br>
> shared credentials from the VM. Not sure about replay attacks; I haven't<br>
> actually investigated the implementation.<br>
><br>
> cheers,<br>
> Zane.<br>
<br>
> Excellent - many thanks for the confirmation.<br>
><br>
> Cheers,<br>
> Michael<br>
<br>
Hello Michael and Zane,<br>
<br>
sorry for the late reply.<br>
<br>
I believe Zane is referring to a video from 2017 [0].<br>
Yes, messages from trove instances are encrypted and the keys are kept<br>
in Trove DB. It is still a shared message bus, but it can be a message<br>
bus dedicated for Trove only and separated from message bus shared by<br>
other Openstack services.<br>
<br>
DDOS attacks are also mentioned in the video as a potential threat but<br>
there is very little details and possible solutions. Recently we had<br>
some internal discussion about this threat within Trove team. Maybe we<br>
could user Rabbitmq mechanisms for flow control mentioned in [1,2,3] ?<br>
<br>
Another point, I'm wondering if this is a problem only in Trove or is<br>
it something other services would be interesting in also ?<br>
<br>
Best,<br>
Darek<br>
<br>
[0] <a href="https://youtu.be/dzvcKlt3Lx8" rel="noreferrer" target="_blank">https://youtu.be/dzvcKlt3Lx8</a><br>
[1] <a href="https://www.rabbitmq.com/flow-control.html" rel="noreferrer" target="_blank">
https://www.rabbitmq.com/flow-control.html</a><br>
[2] <a href="http://www.rabbitmq.com/blog/2012/04/17/rabbitmq-performance-measurements-part-1/" rel="noreferrer" target="_blank">
http://www.rabbitmq.com/blog/2012/04/17/rabbitmq-performance-measurements-part-1/</a><br>
[3] <a href="https://tech.labs.oliverwyman.com/blog/2013/08/31/controlling-fast-producers-in-a-rabbit-as-a-service/" rel="noreferrer" target="_blank">
https://tech.labs.oliverwyman.com/blog/2013/08/31/controlling-fast-producers-in-a-rabbit-as-a-service/</a><br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</body>
</html>