<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Well some progress.<br>
I figured out the problem with the pools on xenserver/libvirt but
I am back to my same problem with a missing volume_id.<br>
<br>
It looks like the code relates to iscsi devices and I noticed a
warning in the log files that has me wondering.<br>
2013-11-26 22:19:26.017 WARNING nova.virt.xenapi.driver
[req-65e019f3-47d3-4e21-8717-8acb2b4b1ef5 admin demo] [instance:
26d231b2-4199-46b0-aacf-b5cb43de7993] Could not determine key:
'iscsi_iqn'<br>
<br>
Why would openstack be trying to use iscsi at this point since the
volume managment is through ceph?<br>
<br>
<br>
On 11/26/2013 02:11 AM, Bob Ball wrote:<br>
</div>
<blockquote
cite="mid:w8h26nnjm655msj099oyskvk.1385449888199@email.android.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<pre style="word-wrap:break-word; font-size:10.0pt; font-family:Tahoma; color:black">Hi,
I'll admit I really don't know the answer.
Can you send the question to xs-devel? That's where xenserver-core and the tech preview are typically developed and discussed.
Bob
Alvin Starr <a class="moz-txt-link-rfc2396E" href="mailto:alvin@netvel.net"><alvin@netvel.net></a> wrote:
</pre>
<div>
<div class="moz-cite-prefix">should I be able to get libvirt to
talk with ceph under the xenserver tech preivew?<br>
<br>
I start with a clean 6.4 then add.<br>
yum -y install xenserver-core<br>
I run xenserver-install-wizard<br>
yum -y install ceph ceph-radosgw rbd-fuse python-argparse<br>
rpm -ihv <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://xenbits.xen.org/djs/xenserver-tech-preview-release-0.3.0-0.x86_64.rpm">http://xenbits.xen.org/djs/xenserver-tech-preview-release-0.3.0-0.x86_64.rpm</a><br>
yum -y update<br>
reboot<br>
<br>
I can access my rbd pools using the ceph direct tools but
through libvirt I can see the volumes but I get the following
error when I try to use libvirt to do anything real with the
volumes.<br>
<br>
2013-11-26 04:24:16.207+0000: 7357: error :
virFDStreamOpenFileInternal:592 : Unable to open stream for
'volumes/volume-f066424b-1163-4e79-9f11-c5a12919d986': No such
file or directory<br>
<br>
am I missing something fundemental?<br>
<br>
On 11/22/2013 11:29 AM, Bob Ball wrote:<br>
</div>
<blockquote type="cite">
<style>
<!--
@font-face
{font-family:Calibri}
@font-face
{font-family:Tahoma}
@font-face
{font-family:Consolas}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
pre
{margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black}
span.HTMLPreformattedChar
{font-family:Consolas;
color:black}
p.msochpdefault, li.msochpdefault, div.msochpdefault
{margin-right:0cm;
margin-left:0cm;
font-size:10.0pt;
font-family:"Times New Roman","serif";
color:black}
span.htmlpreformattedchar0
{font-family:Consolas;
color:black}
span.emailstyle19
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle22
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.BalloonTextChar
{font-family:"Tahoma","sans-serif";
color:black}
.MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:72.0pt 72.0pt 72.0pt 72.0pt}
div.WordSection1
{}
ol
{margin-bottom:0cm}
ul
{margin-bottom:0cm}
-->
</style>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">The volume_id missing from the
connection_details is highly suspicious, yes. I’ve not
seen that before, and don’t know what could cause it.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Hopefully someone else on the list will
be able to assist. If not, I may be able to have
another look on Monday.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">I’m not sure why we didn’t use rdb-fuse –
that’s a question best asked on the xs-devel mailing
list.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Bob</span></p>
<p class="MsoNormal"><a moz-do-not-send="true"
name="_MailEndCompose"><span style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></a></p>
<div style="border:none; border-left:solid blue 1.5pt;
padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt;
padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;
font-family:"Tahoma","sans-serif";
color:windowtext" lang="EN-US">From:</span></b><span
style="font-size:10.0pt;
font-family:"Tahoma","sans-serif";
color:windowtext" lang="EN-US"> Alvin Starr [<a
moz-do-not-send="true"
class="moz-txt-link-freetext"
href="mailto:alvin@netvel.net">mailto:alvin@netvel.net</a>]
<br>
<b>Sent:</b> 22 November 2013 15:40<br>
<b>To:</b> Bob Ball; <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:openstack@lists.openstack.org">
openstack@lists.openstack.org</a><br>
<b>Subject:</b> Re: [Openstack] Openstack and xen
issues.</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">Doh. <br>
Now I feel stupid.<br>
<br>
It is getting much farther <br>
<br>
Now I am seeing the following but I expect it may be
because of all the other stuff I broke trying to
figure out my previous problem.<br>
<br>
<br>
Error: 'volume_id'<br>
Traceback (most recent call last):<br>
File "/opt/stack/nova/nova/compute/manager.py",
line 1030, in _build_instance<br>
set_access_ip=set_access_ip)<br>
File "/opt/stack/nova/nova/compute/manager.py",
line 1439, in _spawn<br>
LOG.exception(_('Instance failed to spawn'),
instance=instance)<br>
File "/opt/stack/nova/nova/compute/manager.py",
line 1436, in _spawn<br>
block_device_info)<br>
File "/opt/stack/nova/nova/virt/xenapi/driver.py",
line 219, in spawn<br>
admin_password, network_info, block_device_info)<br>
File "/opt/stack/nova/nova/virt/xenapi/vmops.py",
line 351, in spawn<br>
network_info, block_device_info, name_label,
rescue)<br>
File "/opt/stack/nova/nova/virt/xenapi/vmops.py",
line 499, in _spawn<br>
undo_mgr.rollback_and_reraise(msg=msg,
instance=instance)<br>
File "/opt/stack/nova/nova/utils.py", line 823, in
rollback_and_reraise<br>
self._rollback()<br>
File "/opt/stack/nova/nova/virt/xenapi/vmops.py",
line 477, in _spawn<br>
name_label)<br>
File "/opt/stack/nova/nova/virt/xenapi/vmops.py",
line 139, in inner<br>
rv = f(*args, **kwargs)<br>
File "/opt/stack/nova/nova/virt/xenapi/vmops.py",
line 339, in create_disks_step<br>
disk_image_type,
block_device_info=block_device_info)<br>
File
"/opt/stack/nova/nova/virt/xenapi/vm_utils.py", line
535, in get_vdis_for_instance<br>
vdi_uuid = get_vdi_uuid_for_volume(session,
connection_data)<br>
File
"/opt/stack/nova/nova/virt/xenapi/vm_utils.py", line
488, in get_vdi_uuid_for_volume<br>
sr_uuid, label, sr_params =
volume_utils.parse_sr_info(connection_data)<br>
File
"/opt/stack/nova/nova/virt/xenapi/volume_utils.py",
line 213, in parse_sr_info<br>
params = parse_volume_info(connection_data)<br>
File
"/opt/stack/nova/nova/virt/xenapi/volume_utils.py",
line 232, in parse_volume_info<br>
volume_id = connection_data['volume_id']<br>
KeyError: 'volume_id'<br>
<br>
<br>
A partly off topic question.<br>
Why not use rdb-fuse and mount the ceph blobs as files
instead of going through libvirt?<br>
<br>
On 11/22/2013 09:41 AM, Bob Ball wrote:</p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;
font-family:"Tahoma","sans-serif"">Could
you provide the full error log with nova crashing?
</span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;
font-family:"Tahoma","sans-serif""> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;
font-family:"Tahoma","sans-serif"">Thanks,</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;
font-family:"Tahoma","sans-serif""> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;
font-family:"Tahoma","sans-serif"">Bob</span></p>
<div>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr align="center" size="2" width="100%">
</div>
<div id="divRpF290828">
<p class="MsoNormal"
style="margin-bottom:12.0pt"><b><span
style="font-size:10.0pt;
font-family:"Tahoma","sans-serif"">From:</span></b><span
style="font-size:10.0pt;
font-family:"Tahoma","sans-serif"">
Alvin Starr [<a moz-do-not-send="true"
href="mailto:alvin@netvel.net">alvin@netvel.net</a>]<br>
<b>Sent:</b> 22 November 2013 14:31<br>
<b>To:</b> Bob Ball; <a
moz-do-not-send="true"
href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
<b>Subject:</b> Re: [Openstack] Openstack
and xen issues.</span></p>
</div>
<div>
<div>
<p class="MsoNormal">I have put Openstack on a
separate machine to try and separate and
isolate the various components I need to
work with in the interests of making my
debugging easier.<br>
This in retrospect may not have been the
best idea.<br>
<br>
I have had a very long history with xen and
that may be more of an impediment because I
think I know things about it that are no
longer true.<br>
<br>
I am using the default devstack scripts as
of a few weeks ago so it should be grabbing
the latest version of Openstack or at least
that is my belief.<br>
<br>
Here is my sr-param-list.<br>
<br>
uuid ( RO) :
7d56f548-174b-d42b-12f2-e0849588e503<br>
name-label ( RW): Ceph Storage<br>
name-description ( RW): <br>
host ( RO): localhost<br>
allowed-operations (SRO): unplug;
plug; PBD.create; PBD.destroy; VDI.clone;
scan; VDI.create; VDI.destroy<br>
current-operations (SRO): <br>
VDIs (SRO): <br>
PBDs (SRO):
40dd29a3-154a-e841-ce52-4547c817d856<br>
virtual-allocation ( RO): 348064577384<br>
physical-utilisation ( RO): 342363992064<br>
physical-size ( RO):
18986006446080<br>
type ( RO): libvirt<br>
content-type ( RO): <br>
shared ( RW): true<br>
introduced-by ( RO): <not in
database><br>
other-config (MRW): ceph_sr:
true<br>
sm-config (MRO): <br>
blobs ( RO): <br>
local-cache-enabled ( RO): false<br>
tags (SRW): <br>
<br>
<br>
I started tracing the xenapi transactions
over the network and could see the
pool.get_all and pool.get_default when the
sr_filter was not set but once I set it
nova would crash complaining about no
repository.<br>
I checked the TCP transactions and did not
see any SR.get_all while some debugging
prints assured me that the code was being
exercised.<br>
<br>
<br>
<br>
On 11/22/2013 04:40 AM, Bob Ball wrote:</p>
</div>
<blockquote style="margin-top:5.0pt;
margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Hi Alvin,</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Yes, we typically do
expect Nova to be running in a DomU.
It’s worth checking out
<a moz-do-not-send="true"
href="http://docs.openstack.org/trunk/openstack-compute/install/yum/content/introduction-to-xen.html"
target="_blank">
http://docs.openstack.org/trunk/openstack-compute/install/yum/content/introduction-to-xen.html</a>
just to make sure you’ve got everything
covered there.</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">I say typically because
in some configurations (notably using
xenserver-core) it may be possible to
run Nova in dom0 by setting the
connection URL to “unix://local”. This
is an experimental configuration and was
added near the end of Havana – see
<a moz-do-not-send="true"
href="https://blueprints.launchpad.net/nova/+spec/xenserver-core"
target="_blank">
https://blueprints.launchpad.net/nova/+spec/xenserver-core</a>.</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">In terms of
sr_matching_filter, check that you’re
setting it in the right group. If
you’re using the latest builds of
Icehouse then it should be in the
xenserver group. I’m also assuming that
the other-config for the SR does indeed
contain ceph-sr=true?</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Is the SR that is used
for VMs still the default-SR?</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Thanks,</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D">Bob</span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:#1F497D"> </span></p>
<div style="border:none; border-left:solid
blue 1.5pt; padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;
border-top:solid #B5C4DF 1.0pt;
padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;
font-family:"Tahoma","sans-serif";
color:windowtext" lang="EN-US">From:</span></b><span
style="font-size:10.0pt;
font-family:"Tahoma","sans-serif";
color:windowtext" lang="EN-US">
Alvin Starr [<a
moz-do-not-send="true"
href="mailto:alvin@netvel.net"
target="_blank">mailto:alvin@netvel.net</a>]
<br>
<b>Sent:</b> 22 November 2013
01:32<br>
<b>To:</b> <a
moz-do-not-send="true"
href="mailto:openstack@lists.openstack.org"
target="_blank">openstack@lists.openstack.org</a><br>
<b>Subject:</b> [Openstack]
Openstack and xen issues.</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"
style="margin-bottom:12.0pt"><br>
I am trying to use xen with Ceph and
openstack using the devstack package.<br>
I am slowly wacking my way through
things and have noticed a few issues.</p>
<ol start="1" style="margin-top:0cm"
type="1">
<li class="MsoNormal" style=""> openstack
expects to be running in a domU and
generates error messages even if
xenapi_check_host is false. I am not
sure if this causes other side
effects. The tests for the local dom0
should be completley bypassed if the
check is disabled. </li>
<li class="MsoNormal" style="">Open
stack tries to read the xen SRs and
checks the default one which ends up
being the xen local storage and not
any other SR. If I set the
sr_matching_filter =
other-config:ceph-sr=true there should
be a xapi SR.get_all request generated
but it looks like it is not generated
at all. I have tracked the http
traffic and no out put is generated
even though the approprate code is
being called.
</li>
</ol>
<p class="MsoNormal"
style="margin-bottom:12.0pt"> </p>
<pre>-- </pre>
<pre>Alvin Starr || voice: (905)513-7688</pre>
<pre>Netvel Inc. || Cell: (416)806-0133</pre>
<pre><a moz-do-not-send="true" href="mailto:alvin@netvel.net" target="_blank">alvin@netvel.net</a> ||</pre>
</div>
</div>
</blockquote>
<p class="MsoNormal"><br>
<br>
<br>
</p>
<pre>-- </pre>
<pre>Alvin Starr || voice: (905)513-7688</pre>
<pre>Netvel Inc. || Cell: (416)806-0133</pre>
<pre><a moz-do-not-send="true" href="mailto:alvin@netvel.net" target="_blank">alvin@netvel.net</a> ||</pre>
</div>
</div>
</div>
</div>
</blockquote>
<p class="MsoNormal"><br>
<br>
<br>
</p>
<pre>-- </pre>
<pre>Alvin Starr || voice: (905)513-7688</pre>
<pre>Netvel Inc. || Cell: (416)806-0133</pre>
<pre><a moz-do-not-send="true" href="mailto:alvin@netvel.net">alvin@netvel.net</a> ||</pre>
</div>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Alvin Starr || voice: (905)513-7688
Netvel Inc. || Cell: (416)806-0133
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:alvin@netvel.net">alvin@netvel.net</a> ||
</pre>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Alvin Starr || voice: (905)513-7688
Netvel Inc. || Cell: (416)806-0133
<a class="moz-txt-link-abbreviated" href="mailto:alvin@netvel.net">alvin@netvel.net</a> ||
</pre>
</body>
</html>