StorageGRID Appliance SG5600 Controller is rebooting unexpectedly
Applies to
- StorageGRID Webscale: 10.x and above
- StorageGRID Appliances (SGA): SG5612 and SG5660
Issue
The SG5600 Controller is randomly resetting. The impacts to the GRID in general appear to be minimal outside some alarms, although some operations may need to be retried from the client application. In very rare situations, two SGAs within the same Cassandra cluster could experience simultaneous resets causing more impacts and may prevent them from coming back online on their own. If multiple SGAs are impacted, please engage NetApp Support as soon as possible to check the health of Cassandra and take recovery steps.
When checking the lumberjack log collections, review of base-os-logs/var/log/storagegrid/nodes/<gridid>.log show the follow snippets in the file. This is indications that the controller has rebooted.
Example:
Feb 13 01:02:19 localhost logger: [2018-02-13T01:02:19+0000 INSG] Starting system logging: syslog-ng.
Feb 13 01:02:19 localhost logger: [2018-02-13T01:02:19+0000 INSG] Detected new container with persistent data. Initializing container and restoring data
Feb 13 01:02:19 localhost logger: [2018-02-13T01:02:19+0000 INSG] [root@SG:/root] >>> /usr/sbin/restore_container.sh 2>&1
......
Feb 13 01:02:45 localhost logger: [2018-02-13T01:02:45+0000 INSG] #######################################
Feb 13 01:02:45 localhost logger: [2018-02-13T01:02:45+0000 INSG] STARTING SERVICES
Feb 13 01:02:45 localhost logger: [2018-02-13T01:02:45+0000 INSG] #######################################