<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body>
You are right in the sense that's the ideal scenario. <br>
<br>
(Impl-wise) However, even today we do not guarantee that behavior.
If someone were to propose a new driver or a change driver
capability or any thing of such order, images in status killed won't
be guaranteed to have removed the garbage data. The driver may not
choose to be resilient enough or would not take the responsibility
of data removal synchronously on failures.<br>
<br>
Taking that fact in account, I have thought of Brianna's patch to be
okay.<br>
<br>
<div class="moz-cite-prefix">On 9/11/15 4:42 AM, Flavio Percoco
wrote:<br>
</div>
<blockquote cite="mid:20150911084236.GR6373@redhat.com" type="cite">On
10/09/15 15:36 -0400, Nikhil Komawar wrote:
<br>
<blockquote type="cite">The solution to this problem is to improve
the scrubber to clean up the
<br>
garbage data left behind in the backend store during such failed
uploads.
<br>
<br>
Currently, scrubber cleans up images in pending_delete and
extending
<br>
that to images in killed status would avoid such a situation.
<br>
</blockquote>
<br>
While the above would certainly help, I think it's not the right
<br>
solution. Images in status "killed" should not have data to begin
<br>
with.
<br>
<br>
I'd rather find a way to clean that data as soon as the image is
<br>
moved to a "killed" state instead of extending the scrubber.
<br>
<br>
Cheers,
<br>
Flavio
<br>
<br>
<blockquote type="cite">On 9/10/15 3:28 PM, Poulos, Brianna L.
wrote:
<br>
<blockquote type="cite">Malini,
<br>
<br>
Thank you for bringing up the ³killed² state as it relates to
quota. We
<br>
opted to move the image to a killed state since that is what
occurs when
<br>
an upload fails, and the signature verification failure would
occur during
<br>
an upload. But we should keep in mind the potential to take
up space and
<br>
yet not take up quota when signature verification fails.
<br>
<br>
Regarding the MD5 hash, there is currently a glance spec [1]
to allow the
<br>
hash method used for the checksum to be configurable‹currently
it is
<br>
hardcoded in glance. After making it configurable, the
default would
<br>
transition from MD5 to something more secure (like SHA-256).
<br>
<br>
[1] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/191542/">https://review.openstack.org/#/c/191542/</a>
<br>
<br>
Thanks,
<br>
~Brianna
<br>
<br>
<br>
<br>
<br>
On 9/10/15, 5:10 , "Bhandaru, Malini K"
<a class="moz-txt-link-rfc2396E" href="mailto:malini.k.bhandaru@intel.com"><malini.k.bhandaru@intel.com></a>
<br>
wrote:
<br>
<br>
<blockquote type="cite">Brianna, I can imagine a denial of
service attack by uploading images
<br>
whose signature is invalid if we allow them to reside in
Glance
<br>
In a "killed" state. This would be less of an issue "killed"
images still
<br>
consume storage quota until actually deleted.
<br>
Also given MD-5 less secure, why not have the default hash
be SHA-1 or 2?
<br>
Regards
<br>
Malini
<br>
<br>
-----Original Message-----
<br>
From: Poulos, Brianna L. [<a class="moz-txt-link-freetext" href="mailto:Brianna.Poulos@jhuapl.edu">mailto:Brianna.Poulos@jhuapl.edu</a>]
<br>
Sent: Wednesday, September 09, 2015 9:54 AM
<br>
To: OpenStack Development Mailing List (not for usage
questions)
<br>
Cc: <a class="moz-txt-link-abbreviated" href="mailto:stuart.mclaren@hp.com">stuart.mclaren@hp.com</a>
<br>
Subject: Re: [openstack-dev] [glance] [nova] Verification of
glance
<br>
images before boot
<br>
<br>
Stuart is right about what will currently happen in Nova
when an image is
<br>
downloaded, which protects against unintentional
modifications to the
<br>
image data.
<br>
<br>
What is currently being worked on is adding the ability to
verify a
<br>
signature of the checksum. The flow of this is as follows:
<br>
1. The user creates a signature of the "checksum hash"
(currently MD5) of
<br>
the image data offline.
<br>
2. The user uploads a public key certificate, which can be
used to verify
<br>
the signature to a key manager (currently Barbican).
<br>
3. The user creates an image in glance, with signature
metadata
<br>
properties.
<br>
4. The user uploads the image data to glance.
<br>
5. If the signature metadata properties exist, glance
verifies the
<br>
signature of the "checksum hash", including retrieving the
certificate
<br>
</blockquote>
>from the key manager.
<br>
<blockquote type="cite">6. If the signature verification
fails, glance moves the image to a
<br>
killed state, and returns an error message to the user.
<br>
7. If the signature verification succeeds, a log message
indicates that
<br>
it succeeded, and the image upload finishes successfully.
<br>
<br>
8. Nova requests the image from glance, along with the image
properties,
<br>
in order to boot it.
<br>
9. Nova uses the signature metadata properties to verify the
signature
<br>
(if a configuration option is set).
<br>
10. If the signature verification fails, nova does not boot
the image,
<br>
but errors out.
<br>
11. If the signature verification succeeds, nova boots the
image, and a
<br>
log message notes that the verification succeeded.
<br>
<br>
Regarding what is currently in Liberty, the blueprint
mentioned [1] has
<br>
merged, and code [2] has also been merged in glance, which
handles steps
<br>
1-7 of the flow above.
<br>
<br>
For steps 7-11, there is currently a nova blueprint [3],
along with code
<br>
[4], which are proposed for Mitaka.
<br>
<br>
Note that we are in the process of adding official
documentation, with
<br>
examples of creating the signature as well as the properties
that need to
<br>
be added for the image before upload. In the meantime,
there's an
<br>
etherpad that describes how to test the signature
verification
<br>
functionality in Glance [5].
<br>
<br>
Also note that this is the initial approach, and there are
some
<br>
limitations. For example, ideally the signature would be
based on a
<br>
cryptographically secure (i.e. not MD5) hash of the image.
There is a
<br>
spec in glance to allow this hash to be configurable [6].
<br>
<br>
[1]
<br>
<a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/glance/+spec/image-signing-and-verificati">https://blueprints.launchpad.net/glance/+spec/image-signing-and-verificati</a>
<br>
o
<br>
n-support
<br>
[2]
<br>
<a class="moz-txt-link-freetext" href="https://github.com/openstack/glance/commit/484ef1b40b738c87adb203bba6107dd">https://github.com/openstack/glance/commit/484ef1b40b738c87adb203bba6107dd</a>
<br>
b
<br>
4b04ff6e
<br>
[3] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/188874/">https://review.openstack.org/#/c/188874/</a>
<br>
[4] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/189843/">https://review.openstack.org/#/c/189843/</a>
<br>
[5]
<br>
<a class="moz-txt-link-freetext" href="https://etherpad.openstack.org/p/liberty-glance-image-signing-instructions">https://etherpad.openstack.org/p/liberty-glance-image-signing-instructions</a>
<br>
[6] <a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/191542/">https://review.openstack.org/#/c/191542/</a>
<br>
<br>
<br>
Thanks,
<br>
~Brianna
<br>
<br>
<br>
<br>
<br>
On 9/9/15, 12:16 , "Nikhil Komawar"
<a class="moz-txt-link-rfc2396E" href="mailto:nik.komawar@gmail.com"><nik.komawar@gmail.com></a> wrote:
<br>
<br>
<blockquote type="cite">That's correct.
<br>
<br>
The size and the checksum are to be verified outside of
Glance, in this
<br>
case Nova. However, you may want to note that it's not
necessary that
<br>
all Nova virt drivers would use py-glanceclient so you
would want to
<br>
check the download specific code in the virt driver your
Nova
<br>
deployment is using.
<br>
<br>
Having said that, essentially the flow seems appropriate.
Error must be
<br>
raise on mismatch.
<br>
<br>
The signing BP was to help prevent the compromised Glance
from changing
<br>
the checksum and image blob at the same time. Using a
digital
<br>
signature, you can prevent download of compromised data.
However, the
<br>
feature has just been implemented in Glance; Glance users
may take time
<br>
to adopt.
<br>
<br>
<br>
<br>
On 9/9/15 11:15 AM, <a class="moz-txt-link-abbreviated" href="mailto:stuart.mclaren@hp.com">stuart.mclaren@hp.com</a> wrote:
<br>
<blockquote type="cite">The glance client (running
'inside' the Nova server) will
<br>
re-calculate the checksum as it downloads the image and
then compare
<br>
it against the expected value. If they don't match an
error will be
<br>
raised.
<br>
<br>
<blockquote type="cite">How can I know that the image
that a new instance is spawned from -
<br>
is actually the image that was originally registered
in glance - and
<br>
has not been maliciously tampered with in some way?
<br>
<br>
Is there some kind of verification that is performed
against the
<br>
md5sum of the registered image in glance before a new
instance is
<br>
spawned?
<br>
<br>
Is that done by Nova?
<br>
Glance?
<br>
Both? Neither?
<br>
<br>
The reason I ask is some 'paranoid' security (that is
their job I
<br>
suppose) people have raised these questions.
<br>
<br>
I know there is a glance BP already merged for L [1] -
but I would
<br>
like to understand the actual flow in a bit more
detail.
<br>
<br>
Thanks.
<br>
<br>
[1]
<br>
<br>
<a class="moz-txt-link-freetext" href="https://blueprints.launchpad.net/glance/+spec/image-signing-and-verif">https://blueprints.launchpad.net/glance/+spec/image-signing-and-verif</a>
<br>
ica
<br>
tion-support
<br>
<br>
<br>
--
<br>
Best Regards,
<br>
Maish Saidel-Keesing
<br>
<br>
<br>
<br>
------------------------------
<br>
<br>
_______________________________________________
<br>
OpenStack-dev mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
<br>
<br>
End of OpenStack-dev Digest, Vol 41, Issue 22
<br>
*********************************************
<br>
<br>
</blockquote>
<br>
______________________________________________________________________
<br>
___
<br>
_
<br>
<br>
OpenStack Development Mailing List (not for usage
questions)
<br>
Unsubscribe:
<br>
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
</blockquote>
--
<br>
<br>
Thanks,
<br>
Nikhil
<br>
<br>
<br>
_______________________________________________________________________
<br>
___ OpenStack Development Mailing List (not for usage
questions)
<br>
Unsubscribe:
<br>
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
</blockquote>
<br>
__________________________________________________________________________
<br>
OpenStack Development Mailing List (not for usage questions)
<br>
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
<br>
__________________________________________________________________________
<br>
OpenStack Development Mailing List (not for usage questions)
<br>
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
</blockquote>
<br>
__________________________________________________________________________
<br>
OpenStack Development Mailing List (not for usage questions)
<br>
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
</blockquote>
<br>
-- <br>
<br>
Thanks,
<br>
Nikhil
<br>
<br>
<br>
__________________________________________________________________________
<br>
OpenStack Development Mailing List (not for usage questions)
<br>
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Thanks,
Nikhil</pre>
</body>
</html>