ONTAP Selectでディスクに障害が発生したとマークされる
環境
- ONTAP Select
- VMware ESXi 6.5またはVMware ESXi 6.7を実行しているHPEサーバ。
問題
Disk NET-1.1
失敗しました。アグリゲートがオフラインであり、データにアクセスできません。同様のエラーがログに表示されます。
- ONTAP Select sysconfig -aログ:
slot 0: Virtual SAS Host Adapter 0b
Firmware rev: 4.2.0
2 : NETAPP PHA-DISK 0001 69.6GB 512B/sect (0-2sVvOXocGKsPNbJvg2)
3 : NETAPP PHA-DISK 0001 11534.3GB 512B/sect (0-2sVvOXMVRBTAOO/Qc3) (Failed)
- ONTAP Select EMSログ:
Tue Jun 01 01:34:32 +0200 [Node01: pha_main000: mlm.excessive.TPlatency:notice]: Average latency of 276855us on target port 53059d50444f5476, has exceeded 250000us
Tue Jun 01 01:34:32 +0200 [Node01: wafl_exempt01: wafl.cp.toolong:error]: Aggregate aggr0 experienced a long CP.
Tue Jun 01 01:34:34 +0200 [Node01: pha_main000: shm.threshold.agrsvIOCount:notice]: Disk 0b.3 has exceeded 12 IOs which have latencies greater than the threshold.
Tue Jun 01 01:34:34 +0200 [Node01: wafl_exempt02: wafl.cp.toolong:error]: Aggregate Aggr1 experienced a long CP.
Tue Jun 01 01:35:00 +0200 [Node01: intr: ems.engine.suppressed:debug]: Event 'dev.driver.throttling' suppressed 4 times in last 602 seconds.
Tue Jun 01 01:35:00 +0200 [Node01: intr: dev.driver.throttling:notice]: mpt driver unit 3 throttling I/O requests due to long latency.
Tue Jun 01 01:39:39 +0200 [Node01: cam: cam.timeout.retry:notice]: CAM device driver I/O timeout. Details: CAM command timeout (retrying), Device da3 (4 retries left). Device ID: 0-2sVvOXMVRBTAOO/Qc3. Command: WRITE(16). CDB: 8a 00 00 00 00 02 bf 93 ef 08 00 00 01 00 00 00 - outstanding for 60016006 milliseconds.
Tue Jun 01 01:39:39 +0200 [Node01: cam: cam.timeout.retry:notice]: CAM device driver I/O timeout. Details: CAM command timeout (retrying), Device da3 (4 retries left). Device ID: 0-2sVvOXMVRBTAOO/Qc3. Command: WRITE(16). CDB: 8a 00 00 00 00 02 bf 93 ee 08 00 00 01 00 00 00 - outstanding for 60017276 milliseconds.
Tue Jun 01 01:39:39 +0200 [Node01: cam: cam.timeout.retry:notice]: CAM device driver I/O timeout. Details: CAM command timeout (retrying), Device da3 (4 retries left). Device ID: 0-2sVvOXMVRBTAOO/Qc3. Command: WRITE(16). CDB: 8a 00 00 00 00 02 bf 93 ed 08 00 00 01 00 00 00 - outstanding for 60017444 milliseconds.
Tue Jun 01 01:39:39 +0200 [Node01: cam: cam.timeout.retry:notice]: CAM device driver I/O timeout. Details: CAM command timeout (retrying), Device da3 (4 retries left). Device ID: 0-2sVvOXMVRBTAOO/Qc3. Command: WRITE(16). CDB: 8a 00 00 00 00 02 bf 93 ec 08 00 00 01 00 00 00 - outstanding for 60017585 milliseconds.
- VMware VMkernelログ:
2021-05-31T22:53:22.005Z cpu29:7799666)Admission failure in path: host/vim/vmvisor/plugins/smx:sfcb-ProviderMa.7799660:uw.7799660
2021-05-31T23:02:09.083Z cpu3:7800957)Admission failure in path: host/vim/vmvisor/plugins/smx:sfcb-ProviderMa.7800955:uw.7800955