I think, from an audit and security standpoint, whatever component is responsible for the placement of data needs to be solely responsible, and needs to fail in a clean and transactional way. If the ring is modified to guarantee that the *first* copy of an object is written into a specific zone (to support a hybrid local/remote cloud, for instance), then I could see such an extension passing muster. However, in any given failure scenario, we still need to ensure that the data is limited to the appropriate zones, and I don't think that working around the existing ring architecture is going to achieve that. <br>
<br><div>The goal of making the implementation pluggable was simply to take the existing methods, codify them somewhat, and then support alternate implementations of those methods. Not real-time pluggable or necessarily even exposed as a separate web service. </div>
<div><br></div><div>I think, if you start looking at the intersection of location compliance, a hybrid of local and remote zones (LAN/WAN), and multiple tiers of storage hardware (SATA, SSD, etc) - patching rings and zones together feels a bit limited. I'd like to support an arbitrarily intelligent/policy-driven component.</div>
<div><br></div><div>Of course, my use case is somewhat outside the "Service Provider Appropriate" mission statement of the core OpenStack project, hence the drive to support alternate ring components, rather than a proposal to make the existing one arbitrarily complex. As you point out, it *works*.</div>
<div><br></div><div>Josh</div><div><br><div class="gmail_quote">On Tue, May 31, 2011 at 6:37 AM, Rostyslav Slipetskyy <span dir="ltr"><<a href="mailto:rslipetskyy@yahoo.com">rslipetskyy@yahoo.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="font-family:'times new roman', 'new york', times, serif;font-size:12pt"><div><font face="'times new roman', 'new york', times, serif">The idea to make ring implementation pluggable seems nice. </font></div>
<div><font face="'times new roman', 'new york', times, serif"><br></font></div><div><font face="'times new roman', 'new york', times, serif">At the same time I am thinking that many developers might not will feel comfortable with modifying existing ring structure, since it *works* :) Probably, the other viable option for allowing data location compliance is to implement it <i>outside</i> of ring file structure (but maybe <i>inside</i> the future Ring component/service). </font></div>
<div><font face="'times new roman', 'new york', times,
serif"><br></font></div><div><font face="'times new roman', 'new york', times, serif">As an example I look at how replication works, "<a>If a replicator detects that a remote drive is has failed, it will use the ring's “get_more_nodes” interface to choose an alternate node to synchronize with." </a></font><font face="'times new roman', 'new york', times, serif"><a>It seems that "Ring#get_more_nodes" can be used in </a></font><span style="font-family:'times new roman', 'new york', times, serif"><a>a similar manner</a></span><span style="font-family:'times new roman', 'new york', times, serif"><a> to select alternative nodes in other zones for storing objects once some of the zones are banned for specific accounts.</a></span></div>
<div><a><font face="'times new roman', 'new york', times, serif"><br></font></a></div><div><font face="'times new roman', 'new york', times, serif">- Rostik</font></div><div><font face="'times new roman', 'new york', times, serif"><br>
</font><div><div class="hm"><font><hr size="1"><b style="font-size:12pt;font-family:'times new roman', 'new york', times, serif"><span style="font-weight:bold">From:</span></b><font face="'times new roman', 'new york', times, serif" style="font-size:12pt"> Joshua McKenty <josh@piston.cc></font><br>
<b style="font-size:12pt;font-family:'times new roman', 'new york', times, serif"><span style="font-weight:bold">To:</span></b><font face="'times new roman', 'new york', times, serif" style="font-size:12pt"> Rostyslav Slipetskyy <<a href="mailto:rslipetskyy@yahoo.com" target="_blank">rslipetskyy@yahoo.com</a>></font><br>
<b style="font-size:12pt;font-family:'times new roman', 'new york', times, serif"><span style="font-weight:bold">Cc:</span></b><font face="'times new roman', 'new york', times, serif" style="font-size:12pt"> OpenStack <<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>></font><br>
<b style="font-size:12pt;font-family:'times new roman', 'new york', times, serif"><span style="font-weight:bold">Sent:</span></b><font face="'times new roman', 'new york', times, serif" style="font-size:12pt"> Tue, May 31, 2011 1:45:01 AM</font><br>
<b style="font-size:12pt;font-family:'times new roman', 'new york', times, serif"><span style="font-weight:bold">Subject:</span></b><font face="'times new roman', 'new york', times, serif" style="font-size:12pt"> Re: [Openstack] suggestion for data location compliance in swift</font><br>
</font></div><div><div></div><div class="h5"><br>
<font face="'times new roman', 'new york', times, serif" style="font-size:12pt">This was one of the use cases that drove the design discussion on decoupling the swift ring implementation from the rest of swift (along with supporting multiple tiers of hardware). See </font><a rel="nofollow" href="https://blueprints.launchpad.net/swift/+spec/swift-pluggable-hashing-ring" style="font-size:12pt;font-family:'times new roman', 'new york', times, serif" target="_blank">https://blueprints.launchpad.net/swift/+spec/swift-pluggable-hashing-ring</a><font face="'times new roman', 'new york', times, serif" style="font-size:12pt"> for the basic proposal, however, and you'll note that we could definitely use some additional developers :)</font><div style="font-size:12pt;font-family:'times new roman', 'new york', times, serif">
<br></div><div style="font-size:12pt;font-family:'times new roman', 'new york', times, serif">Joshua<br><div><br></div><div><br><div>
Joshua McKenty<br>Piston Cloud Computing, Inc.<br><a href="tel:%28650%29%20283-6846" value="+16502836846" target="_blank">(650) 283-6846</a><br><a rel="nofollow" href="mailto:joshua@piston.cc" target="_blank">joshua@piston.cc</a><br>
<br><br>
</div>
<br><div><div>On 2011-05-30, at 4:18 PM, Rostyslav Slipetskyy wrote:</div><br><blockquote type="cite"><div>Some of the data stored in the cloud has legal requirements to be stored <br>physically within certain geographical boundary (for example within a country). <br>
Currently OpenStack does not allow to impose restrictions on data location.<br><br>It looks like zones can be very handy to achieve data location compliance <br>(according to the docs they can be used to group devices based on physical <br>
location). For example, suppose that a provider has servers in USA (zones 1-5) <br>and Canada (zones 6-10). Let's imagine that a customer has some legal <br>requirements to store its data on the servers in the USA. What I imagine doing <br>
is to restrict data for customer accounts to zones 1-5. <br><br>Most probably, ring modifications will be necessary in order to implement this.<br><br>Best
Regards,<br>Rostik<br><br>_______________________________________________<br>Mailing list: <a rel="nofollow" href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>Post to : <a rel="nofollow" href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a rel="nofollow" href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>More help : <a rel="nofollow" href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
</div></blockquote></div><br></div></div></div></div></div></div><div style="font-family:'times new roman', 'new york', times, serif;font-size:12pt"></div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>
<p><br></p>
<p>Joshua McKenty</p>
<p>Piston Cloud Computing, Inc.</p>
<p>(650) 283-6846</p>
<p>joshua@piston.cc</p><br>
</div>