無応答と宣言されている容量階層で I/O を実行できませんでした
環境
- FabricPool
- ボリューム移動
- SnapMirror
- ONTAP 9
- Cloud Volumes ONTAP
- AWS S3
- BlueXP
問題
- ボリューム移動の実行に失敗しました
::> volume move show
Error Message:
--------------------------------
Vserver Name: vs0
Volume Name: volume2
Actual Completion Time: -
Bytes Remaining: -
Destination Aggregate: aggr1FP
Detailed Status: Internal error
Error: Failure to perform I/O with the capacity tier, which has been declared unresponsive
Estimated Time of Completion: -
Managing Node: node1
Percentage Complete: -
Move Phase: restarting
- SnapMirrorの更新が次のいずれかのエラーで失敗しました:
Transfer failed. (Filesystem error (Failure to perform I/O with the capacity tier, which has been declared unresponsive))
Transfer failed.(Filesystem error(Failure to perform I/O with the capacity tier. Retry the operation))
- EMSログ:クラウドからのレイテンシがしきい値を超えている(5000ミリ秒)
Tue Feb 20 01:21:17 +0800 [NetApp-02: OscLowPriThreadPool: wafl.ca.latency.threshold:notice]: Measured latency from cloud (99988 msec) more than threshold (5000 msec) (aggregate: "test", aggregate uuid: "287a4276-4694-7583-a3e2-1e83b7ebc51c", object store: "OSC-CT-443")
- Sktraceログ:接続が使用できず、操作がタイムアウトしました
2024-02-19T16:16:23Z 29244161221223132 [9:0] CLOUD_BIN_ERR: object_store_config_check_connection: HEAD failed for s3 handle 109 at try_count 1 with error 5 (Connection unavailable)
2024-02-19T16:16:34Z 29244184316598880 [10:0] CLOUD_BIN_ERR: object_store_config_check_connection: PUT failed for s3 handle 109 at try_count 2 with error 5 (Connection unavailable)
2024-02-20T01:24:16Z 29313030326095062 [10:0] OSC_ERR: logCmd:368 PUT CmdId: 1074 obj-name: 66b569f2-e91b-4a8c-b03e-0bdc9d685b95/34bcd168_000000000000444c_66b569f2-e91b-4a8c-b03e-0bdc9d685b95 failed with OSC error: Operation Timedout
2024-02-20T01:24:16Z 29313030326098162 [10:0] OSC_ERR: logCmd:378 Stats of PUT CmdId: 1074, wait 1us, start-send 20433us, 1stbyte 99975412us, 1st-lastbyte 0us
2024-02-20T01:24:16Z 29313031059319266 [10:0] OSC_ERR: logCmd:368 PUT CmdId: 951 obj-name: 66b569f2-e91b-4a8c-b03e-0bdc9d685b95/ce81223f_000000000000444d_66b569f2-e91b-4a8c-b03e-0bdc9d685b95 failed with OSC error: Operation Timedout
2024-02-20T01:24:16Z 29313031059321696 [10:0] OSC_ERR: logCmd:378 Stats of PUT CmdId: 951, wait 0us, start-send 20271us, 1stbyte 99975585us, 1st-lastbyte 0us