<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi Team,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am currently working on multiattach feature for HPE 3PAR cinder driver.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For this, while setting up devstack(on stable/queens) I made below change in the local.conf<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#0070C0">[[local|localrc]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">ENABLE_VOLUME_MULTIATTACH=True<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">ENABLE_UBUNTU_CLOUD_ARCHIVE=False<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">/etc/cinder/cinder.conf:<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#0070C0">[3pariscsi_1]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_api_url = https://192.168.1.7:8080/api/v1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_username = user<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_password = password<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">san_ip = 192.168.1.7<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">san_login = user<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">san_password = password<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">volume_backend_name = 3pariscsi_1<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_cpg = my_cpg<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_iscsi_ips = 192.168.11.2,192.168.11.3<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">volume_driver = cinder.volume.drivers.hpe.hpe_3par_iscsi.HPE3PARISCSIDriver<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_iscsi_chap_enabled = True<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">hpe3par_debug = True<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#0070C0">image_volume_cache_enabled = True<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">/etc/cinder/policy.json:<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#0070C0">'volume:multiattach': 'rule:admin_or_owner'<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Added <a href="https://review.opendev.org/#/c/560067/2/cinder/volume/drivers/hpe/hpe_3par_common.py">
https://review.opendev.org/#/c/560067/2/cinder/volume/drivers/hpe/hpe_3par_common.py</a> change in the code.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But I am getting below error in the nova log:<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [None req-2cda6e90-fd45-4bfe-960a-7fca9ba4abab demo admin] [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] Instance failed block
device setup: MultiattachNotSupportedByVirtDriver: Volume dc25f09a-6ae1-4b06-a814-73a8afaba62f has 'multiattach' set, which is not supported for this instance.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] Traceback (most recent call last):<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/compute/manager.py", line 1615, in _prep_block_device<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] wait_func=self._await_block_device_map_created)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 840, in attach_block_devices<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] _log_and_attach(device)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 837, in _log_and_attach<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] bdm.attach(*attach_args, **attach_kwargs)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 46, in wrapped<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] ret_val = method(obj, context, *args, **kwargs)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 620, in attach<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] virt_driver, do_driver_attach)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py",
line 274, in inner<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] return f(*args, **kwargs)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 617, in _do_locked_attach<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] self._do_attach(*args, **_kwargs)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 602, in _do_attach<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] do_driver_attach)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] File "/opt/stack/nova/nova/virt/block_device.py", line 509, in _volume_attach<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa] volume_id=volume_id)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa]
<span style="background:yellow;mso-highlight:yellow">MultiattachNotSupportedByVirtDriver: Volume dc25f09a-6ae1-4b06-a814-73a8afaba62f has 'multiattach' set, which is not supported for this instance.</span><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 04:23:04 CSSOSBE04-B09 nova-compute[31396]: ERROR nova.compute.manager [instance: fcaa5a47-fc48-489d-9827-6533bfd1a9fa]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 29 05:41:20 CSSOSBE04-B09 nova-compute[20455]: DEBUG nova.virt.libvirt.driver [-]
<span style="background:yellow;mso-highlight:yellow">Volume multiattach is not supported based on current versions of QEMU and libvirt. QEMU must be less than 2.10 or libvirt must be greater than or equal to 3.10</span>. {{(pid=20455) _set_multiattach_support
/opt/stack/nova/nova/virt/libvirt/driver.py:619}}<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">stack@CSSOSBE04-B09:/tmp$ virsh --version<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;background:yellow;mso-highlight:yellow">3.6.0</span><span style="font-size:10.0pt"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">stack@CSSOSBE04-B09:/tmp$ kvm --version<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">QEMU emulator version <span style="background:yellow;mso-highlight:yellow">
2.10.1</span>(Debian 1:2.10+dfsg-0ubuntu3.8~cloud1)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">openstack volume show -c multiattach -c status sneha1<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt">+-------------+-----------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">| Field | Value |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">+-------------+-----------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">| <span style="background:yellow;mso-highlight:yellow">
multiattach | True</span> |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">| status | available |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">+-------------+-----------+<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">cinder extra-specs-list<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt">+--------------------------------------+-------------+--------------------------------------------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">| ID | Name | extra_specs |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">+--------------------------------------+-------------+--------------------------------------------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">| bd077fde-51c3-4581-80d5-5855e8ab2f6b | 3pariscsi_1 | {'volume_backend_name': '3pariscsi_1',
<span style="background:yellow;mso-highlight:yellow">'multiattach': '<is> True'</span>}|<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">+--------------------------------------+-------------+--------------------------------------------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">echo $OS_COMPUTE_API_VERSION<o:p></o:p></p>
<p class="MsoNormal">2.60<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">pip list | grep python-novaclient<o:p></o:p></p>
<p class="MsoNormal">DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.<o:p></o:p></p>
<p class="MsoNormal"><span style="background:yellow;mso-highlight:yellow">python-novaclient 13.0.0</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">How do I fix this version issue on my setup to proceed? Please help.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal">Thanks & Regards,<o:p></o:p></p>
<p class="MsoNormal">Sneha Rai<o:p></o:p></p>
</div>
</body>
</html>