[victoria][cinder ?] Dell Unity + Iscsi
Hi everyone I've a small openstack configuration with 4 computes nodes, a Dell Unity 480F for the storage. I'm using cinder with iscsi. Everything work when I create a instance. But some instance after few time are not reponsive. When I check on the hypervisor I can see [888240.310461] sd 14:0:0:2: [sdb] tag#120 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.310493] sd 14:0:0:2: [sdb] tag#120 Sense Key : Illegal Request [current] [888240.310502] sd 14:0:0:2: [sdb] tag#120 Add. Sense: Logical unit not supported [888240.310510] sd 14:0:0:2: [sdb] tag#120 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.310519] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.311045] sd 14:0:0:2: [sdb] tag#121 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.311050] sd 14:0:0:2: [sdb] tag#121 Sense Key : Illegal Request [current] [888240.311065] sd 14:0:0:2: [sdb] tag#121 Add. Sense: Logical unit not supported [888240.311070] sd 14:0:0:2: [sdb] tag#121 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.311074] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.342482] sd 14:0:0:2: [sdb] tag#70 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.342490] sd 14:0:0:2: [sdb] tag#70 Sense Key : Illegal Request [current] [888240.342496] sd 14:0:0:2: [sdb] tag#70 Add. Sense: Logical unit not supported I check on the hypervisor, no error at all on the ethernet interface. I check on the switch, no error at all on the interface on the switch. No sure but it's seem the problem appear more often when the instance are doing nothing during some time. Every firmware, software on the Unity are uptodate. The 4 computes are exactly same, they run the same version of the nova-compute & OS & firmware on the hardware. Any clue ? Or place to search the problem ? Regards -- Albert SHIH Observatoire de Paris xmpp: jas@obspm.fr Heure local/Local time: Tue Jun 1 08:27:42 PM CEST 2021
Hi Albert, Did you configure multipath? Could you attach the output of `multipath -ll` and the content of ` /etc/multipath.conf`? Thanks Yong Huang -----Original Message----- From: Albert Shih <Albert.Shih@obspm.fr> Sent: Wednesday, June 2, 2021 2:45 AM To: openstack-discuss@lists.openstack.org Subject: [victoria][cinder ?] Dell Unity + Iscsi [EXTERNAL EMAIL] Hi everyone I've a small openstack configuration with 4 computes nodes, a Dell Unity 480F for the storage. I'm using cinder with iscsi. Everything work when I create a instance. But some instance after few time are not reponsive. When I check on the hypervisor I can see [888240.310461] sd 14:0:0:2: [sdb] tag#120 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.310493] sd 14:0:0:2: [sdb] tag#120 Sense Key : Illegal Request [current] [888240.310502] sd 14:0:0:2: [sdb] tag#120 Add. Sense: Logical unit not supported [888240.310510] sd 14:0:0:2: [sdb] tag#120 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.310519] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.311045] sd 14:0:0:2: [sdb] tag#121 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.311050] sd 14:0:0:2: [sdb] tag#121 Sense Key : Illegal Request [current] [888240.311065] sd 14:0:0:2: [sdb] tag#121 Add. Sense: Logical unit not supported [888240.311070] sd 14:0:0:2: [sdb] tag#121 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.311074] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.342482] sd 14:0:0:2: [sdb] tag#70 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.342490] sd 14:0:0:2: [sdb] tag#70 Sense Key : Illegal Request [current] [888240.342496] sd 14:0:0:2: [sdb] tag#70 Add. Sense: Logical unit not supported I check on the hypervisor, no error at all on the ethernet interface. I check on the switch, no error at all on the interface on the switch. No sure but it's seem the problem appear more often when the instance are doing nothing during some time. Every firmware, software on the Unity are uptodate. The 4 computes are exactly same, they run the same version of the nova-compute & OS & firmware on the hardware. Any clue ? Or place to search the problem ? Regards -- Albert SHIH Observatoire de Paris xmpp: jas@obspm.fr Heure local/Local time: Tue Jun 1 08:27:42 PM CEST 2021
On 01/06, Albert Shih wrote:
Hi everyone
I've a small openstack configuration with 4 computes nodes, a Dell Unity 480F for the storage.
I'm using cinder with iscsi.
Everything work when I create a instance. But some instance after few time are not reponsive. When I check on the hypervisor I can see
[888240.310461] sd 14:0:0:2: [sdb] tag#120 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.310493] sd 14:0:0:2: [sdb] tag#120 Sense Key : Illegal Request [current] [888240.310502] sd 14:0:0:2: [sdb] tag#120 Add. Sense: Logical unit not supported [888240.310510] sd 14:0:0:2: [sdb] tag#120 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.310519] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.311045] sd 14:0:0:2: [sdb] tag#121 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.311050] sd 14:0:0:2: [sdb] tag#121 Sense Key : Illegal Request [current] [888240.311065] sd 14:0:0:2: [sdb] tag#121 Add. Sense: Logical unit not supported [888240.311070] sd 14:0:0:2: [sdb] tag#121 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.311074] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.342482] sd 14:0:0:2: [sdb] tag#70 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.342490] sd 14:0:0:2: [sdb] tag#70 Sense Key : Illegal Request [current] [888240.342496] sd 14:0:0:2: [sdb] tag#70 Add. Sense: Logical unit not supported
I check on the hypervisor, no error at all on the ethernet interface.
I check on the switch, no error at all on the interface on the switch.
No sure but it's seem the problem appear more often when the instance are doing nothing during some time.
Hi, You should first check if the volume is still exported and mapped to the host in Unity's web console. If it is still properly mapped, you should configure mutlipathing to make it more resilient. If it isn't you probably should confirm that all nodes have different initiator name (/etc/iscsi/initiatorname.iscsi) and different hostname (if configured in nova's conf file under "host" or at the Linux level if not). In any case I would turn on debug logs on Nova and Cinder and try to follow what happened with that specific LUN. Cheers, Gorka.
Every firmware, software on the Unity are uptodate.
The 4 computes are exactly same, they run the same version of the nova-compute & OS & firmware on the hardware.
Any clue ? Or place to search the problem ?
Regards
-- Albert SHIH Observatoire de Paris xmpp: jas@obspm.fr Heure local/Local time: Tue Jun 1 08:27:42 PM CEST 2021
Le 07/06/2021 à 10:54:09+0200, Gorka Eguileor a écrit
Hi everyone
I've a small openstack configuration with 4 computes nodes, a Dell Unity 480F for the storage.
I'm using cinder with iscsi.
Everything work when I create a instance. But some instance after few time are not reponsive. When I check on the hypervisor I can see
[888240.310461] sd 14:0:0:2: [sdb] tag#120 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.310493] sd 14:0:0:2: [sdb] tag#120 Sense Key : Illegal Request [current] [888240.310502] sd 14:0:0:2: [sdb] tag#120 Add. Sense: Logical unit not supported [888240.310510] sd 14:0:0:2: [sdb] tag#120 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.310519] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.311045] sd 14:0:0:2: [sdb] tag#121 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.311050] sd 14:0:0:2: [sdb] tag#121 Sense Key : Illegal Request [current] [888240.311065] sd 14:0:0:2: [sdb] tag#121 Add. Sense: Logical unit not supported [888240.311070] sd 14:0:0:2: [sdb] tag#121 CDB: Read(10) 28 00 00 00 00 00 00 00 08 00 [888240.311074] blk_update_request: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 2 prio class 0 [888240.342482] sd 14:0:0:2: [sdb] tag#70 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [888240.342490] sd 14:0:0:2: [sdb] tag#70 Sense Key : Illegal Request [current] [888240.342496] sd 14:0:0:2: [sdb] tag#70 Add. Sense: Logical unit not supported
I check on the hypervisor, no error at all on the ethernet interface.
I check on the switch, no error at all on the interface on the switch.
No sure but it's seem the problem appear more often when the instance are doing nothing during some time.
Hi,
You should first check if the volume is still exported and mapped to the host in Unity's web console.
If it is still properly mapped, you should configure mutlipathing to make it more resilient.
If it isn't you probably should confirm that all nodes have different initiator name (/etc/iscsi/initiatorname.iscsi) and different hostname (if configured in nova's conf file under "host" or at the Linux level if not).
Yes...it's this f***ing install who put same initiatorname for two of my compute who make a big mess. Thanks a lot for your help Regards. -- Albert SHIH Observatoire de Paris xmpp: jas@obspm.fr Heure local/Local time: Thu Jun 17 08:15:39 PM CEST 2021
참여자 (3)
-
Albert Shih
-
Gorka Eguileor
-
Huang, Yong