<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 2/21/2013 5:26 PM, Bhandaru, Malini
K wrote:<br>
</div>
<blockquote
cite="mid:EE6FFF4F6C34C84C8C98DD2414EEA47E520859EE@FMSMSX105.amr.corp.intel.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hello
Caitlin!<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I
have been pondering your comment .. and some issues
pertaining to horizontal scaling for high availability.
Please let me know if I am<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">missing
something in your suggestion of storage nodes saving
encryption keys.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Say
on compute-node-1, we have a VM and a persistent volume
encrypted with key k1, user is done, volume detached etc.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Later
the VM may land on compute-node-2 and key K1 needs to be
accessed.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">A
single copy of the key would mean danger of key loss. Also
search for the key should the VM and volume host be
different from the prior run (in this case compute-node-2
instead of compute-node-1). Even snapshots may reside on
different storage media and whether we use the same
key-string-id or a copy of the key-string and new id.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If
we elect to store the encryption keys on the storage servers
themselves in secure lock boxes, we would need to replicate
the keys on other peer storage servers to ensure access.
Mirroring .. like Swift replicas.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">This
then becomes a swift like storage mechanism with the keys
encrypted using the storage server’s master key.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I
can see a single master key being distributed to the members
of the storage cluster, so that they can decrypt a key once
they retrieve it from the<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Key
manager (thinking of it as a single entity even if the data
is replicated in multiple places).<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">With
the service endpoint, aka storage node
(Cinder/Glance/Swift), being responsible for decrypting the
key-string, which happens locally, the communication between
key-manager and service-endpoint becomes a less valuable
target, the data flying by less vulnerable.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">TPM
could be used to verify if a storage-endpoint is legitimate
(identity and platform integrity) before it could access the
master key.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We
could have separate master keys for Cinder/Glance/Swift.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">To
get the protection of a double-key like a bank safe deposit
locker, the actual encryption key could be a combination of
the domain/account/user specific key and the per entity key.
That key too would reside in key-manager. Either through
delegation the service endpoint could access it (here I am
having trouble with which key to use to encrypt it ..
different services would be using the key). While stored it
could be encrypted with the key-manager’s master key or
keystone’s master key, but then it would have be passed
along to the service after decryption (vulnerable while in
transit).<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Caching
of keys, to reduce network traffic and latency possible,
with lifetime equal to token lifetime and usual cache space
reclamation policies.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
</blockquote>
<br>
In my evaluation there are only two types of storage encryption that
make any sense: true<br>
end-to-end encryption, and storage server encryption.<br>
<br>
There are a lot of merits to true end-to-end encryption. It provides
complete protection for<br>
the data that is fully under the user's control. The Service
Provider cannot be forced to divulge<br>
the content of user data for the simple reason that they literally
do not know how to decrypt<br>
any of the data.<br>
<br>
The challenge of true end-to-end encryption is that it is fully
under the user's control.<br>
If the user forgets their pass phrase to where they have stored
their keys, the data is lost.<br>
A solution is needed that reliably stores the keys independent of
the service provider, and<br>
does so in a way that is almost invisible to the user while still
being under their control.<br>
<br>
While I think end-to-end is a great solution, I think it can only be
enabled by the major<br>
OS vendors (Apple, Google,Linux, Microsoft). Openstack is not
sufficiently in the true "ends"<br>
of "end-to-end".<br>
<br>
The goals of storage server encryption are more modest -- to prevent
the theft of data drives<br>
or even servers leading to any data being exposed. The user is not
concerned with how many<br>
keys are used, or which keys are used, just that the data is secure.<br>
<br>
The specific concern I have relates to efficiency of replication,
local caching, taking snapshots<br>
and cloning from snapshots.<br>
<br>
Storage-server controlled lockboxes allow encryption/decryption to
be done efficiently by<br>
the storage servers (in hardware and/or native kernel code, rather
than from user-mode<br>
python code) and prevent encryption from slowing down basic
object/volume manipulation<br>
commands such as replication, snapshotting and cloning.<br>
<br>
This would be done by having the storage server that creates an
asset (such as a Cinder<br>
volume or a Swift Partition) to create a Key ID and a secret Key.
The secret key is not stored<br>
persistently anwhere other than in a secure lockbox, preferably on a
TPM but definitely on<br>
a different device than the content being referenced.<br>
<br>
When a derived object is created (such as a snapshot, a clone or
when the asset is replicated)<br>
the same Key ID is used for the new asset. If this asset is being
replicated, then a secure <br>
transfer of Secret Key must be arranged between the origin lock box
and the destination<br>
lock box.<br>
<br>
The role of openstack should be limited to orchestrating and
standardizing those transfers.<br>
<br>
First, the transferred keys should be signed by the origin lockbox
and encrypted specifically<br>
for the destination lockbox. This needs to be storage-grade
encryption, not the flimsy<br>
encryption used for ephemeral transfer protocols such as
TLS/SSL/HTTPS/IPSEC. Encryption<br>
for transport protocols do not have to worry about their code being
cracked in a month<br>
because they rekey more frequently than that. A stolen disk drive
that yields it's secrets<br>
after a month is a major problem. Using keystone as a central
repository for the keys does<br>
not make them safe, it provides a one stop convenience store for the
would be attacker.<br>
<br>
As for the number of replicas of the keys, the keys are stored in as
many locations as the<br>
asset itself is stored. The specific storage service (Swift or
Cinder) should already have<br>
a solution for ensuring the appropriate level of protection against
loss of a single server.<br>
<br>
One specific form of replication that this enables is partial
caching of assets near the compute<br>
node. Once the relevant secret key is shared between the local
storage server and the<br>
persistent central server. Updates are posted first to the local
storage server, which<br>
asynchronously forwards them to the central server while the local
storage server<br>
only keeps the most recently referenced subset of the asset(s). This
can provide<br>
effectively local storage (in terms of performance) with the
capacity of a central<br>
server. A typical compute node owns a lot more storage than it will
reference in<br>
the next hour. Using local SSD storage for data that will not be
referenced today<br>
is wasteful, providing a unified view of local and central storage
is something that<br>
storage vendors can provide.<br>
<br>
Self-encrypting drives are also a wonderful solution for offloading
work from the CPUs,<br>
but they need vendor-independent control over their keys in order to
be a scalable<br>
enterprise solution. Openstack could provide the guidance that would
enable self-<br>
encrypting drives to be an acceptable solution.<br>
<br>
<br>
</body>
</html>