イレイジャーコーディングされたデータでStorageGRIDの運用停止プロセスが停止する
環境
NetApp StorageGRID 11.7以前
EC(イレイジャーコーディング)
問題
ストレージノードの運用停止が進行しておらず、次のアラートが表示されています。
"Unable to communicate with node
One or more services are unresponsive, or the node cannot be reached"
Cassandraシステムログに次のエラーメッセージが表示されます。
ランバージャックログ
/Node/Cassandra/system.log
ERROR [MutationStage-3] 2021-05-22 05:58:18,334 StorageProxy.java (line 1473) Failed to apply mutation locally : java.lang.IllegalArgumentException/Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
WARN [MutationStage-5] 2021-05-22 05:58:20,099 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-5,5,main]: Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
WARN [MutationStage-1] 2021-05-22 05:58:21,581 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-1,5,main]: Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
WARN [MutationStage-1] 2021-05-22 05:58:22,878 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-1,5,main]: Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
WARN [MutationStage-3] 2021-05-22 05:58:24,215 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-3,5,main]: Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
WARN [MutationStage-3] 2021-05-22 05:58:25,531 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-3,5,main]: Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
ERROR [MutationStage-2] 2021-05-22 05:58:26,467 StorageProxy.java (line 1473) Failed to apply mutation locally : java.lang.IllegalArgumentException/Mutation of 28.612MiB is too large for the maximum size of 16.000MiB
または
WARN [MutationStage-10] 2022-07-02 12:08:21,723 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-10,5,main]: Mutation (keyspace storagegrid, tables [b3f7ac20-fa33-11e9-bdaa-6313fe8d7848], partition key DecoratedKey(4090565182755706843, 333035393332363732)) with size 16.000MiB is too large for the maximum size of 16.000MiB
java.lang.IllegalArgumentException: Mutation (keyspace storagegrid, tables [b3f7ac20-fa33-11e9-bdaa-6313fe8d7848], partition key DecoratedKey(4090565182755706843, 333035393332363732)) with size 16.000MiB is too large for the maximum size of 16.000MiB
WARN [MutationStage-3] 2022-07-02 12:08:22,358 AbstractLocalAwareExecutorService.java (line 167) Uncaught exception on thread Thread[MutationStage-3,5,main]: Mutation (keyspace storagegrid, tables [b3f7ac20-fa33-11e9-bdaa-6313fe8d7848], partition key DecoratedKey(4090565182755706843, 333035393332363732)) with size 16.000MiB is too large for the maximum size of 16.000MiB
java.lang.IllegalArgumentException: Mutation (keyspace storagegrid, tables [b3f7ac20-fa33-11e9-bdaa-6313fe8d7848], partition key DecoratedKey(4090565182755706843, 333035393332363732)) with size 16.000MiB is too large for the maximum size of 16.000MiB