プロキシが原因でData Infrastructure Insights Workload Security AgentのステータスがNot Connectedになる
環境
- Data Infrastructure Insights(DII)(旧称Cloud Insights)ワークロードセキュリティ
- エージェントがプロキシサーバを介して設定されている
問題
- [Workload Security]>[Admin]>[Data Collectores]>[Agents]で、エージェントサーバのステータスが
Not Connected
なっています。
- 障害が発生したため、Cloud Secureエージェントデーモンサービスが開始されていません
# sudo systemctl status cloudsecure-agent.service
● cloudsecure-agent.service - Cloud Secure Agent Daemon Service
Loaded: loaded (/usr/lib/systemd/system/cloudsecure-agent.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2023-06-13 14:32:51 JST; 1h 36min ago
Process: 7898 ExecStart=/bin/bash /opt/netapp/cloudsecure/agent/bin/cloudsecure-agent (code=exited, status=1/FAILURE)
Main PID: 7898 (code=exited, status=1/FAILURE)
systemd[1]: cloudsecure-agent.service: Failed with result 'exit-code'.
systemd[1]: cloudsecure-agent.service: Service RestartSec=15s expired, scheduling restart.
systemd[1]: cloudsecure-agent.service: Scheduled restart job, restart counter is at 12.
systemd[1]: Stopped Cloud Secure Agent Daemon Service.
systemd[1]: cloudsecure-agent.service: Start request repeated too quickly.
systemd[1]: cloudsecure-agent.service: Failed with result 'exit-code'.
systemd[1]: Failed to start Cloud Secure Agent Daemon Service.
cloudsecure-agent-service.log
を示します。Failed to login: cannot find Authorization header
[CloudSecureAgent-akka.actor.default-dispatcher-33965] ERROR com.netapp.datafabric.metadatacatalog.agent.dslifecycle.datapoll.CommandProcessor - Failed to get commands with param isRestart=false from saas layer, reason: Failed to get agent commands from CloudSecure service. Reason - Failed to login: cannot find Authorization header
agent.log
プロキシがDIIサーバーへの接続を拒否することを示します。
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-LogStreamingActor] - Failed to stream 128 log messages to https://XXX.cloudinsights.netapp.com:443/rest/v1/log-receiver/logs with error: The HTTP(S) proxy rejected to open a connection to XXX.cloudinsights.netapp.com:443 with status code: 403 Forbidden
[INFO] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-DeadLetterActorRef] - Message [akka.io.TcpConnection$Unregistered$] without sender to Actor[akka://CloudSecureAgent/deadLetters] was not delivered. [47100] dead letters encountered. If this is not an expected behavior, then [Actor[akka://CloudSecureAgent/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-CommunicationManager] - Received status code 403 and status response [Forbidden] for the request GET:/rest/v1/agents/login/<TENANT_ID>
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-AgentJWTTokenSupplier] - Refreshing JWT token failed. Reason: Failed to login: cannot find Authorization header. Retry after 30 seconds.
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-LogStreamingActor] - Failed to stream[retry] 128 log messages to https://XXX.cloudinsights.netapp.com:443/rest/v1/log-receiver/logs with error: The HTTP(S) proxy rejected to open a connection to XXX.cloudinsights.netapp.com:443 with status code: 403 Forbidden
[INFO] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-DeadLetterActorRef] - Message [akka.io.TcpConnection$Unregistered$] without sender to Actor[akka://CloudSecureAgent/deadLetters] was not delivered. [47101] dead letters encountered. If this is not an expected behavior, then [Actor[akka://CloudSecureAgent/deadLetters]] may have terminated unexpectedly, This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-CommunicationManager] - Received status code 403 and status response [Forbidden] for the request GET:/rest/v1/agents/login/<TENANT_ID>
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-CommunicationManager] - Received status code 403 and status response [Forbidden] for the request GET:/rest/v1/agents/login/<TENANT_ID>
[ERROR] [prod] [<TENANT_ID>] [<AGENT_ID>] [agent-CommunicationManager] - Request failed for GET /rest/v1/agents/<AGENT_ID>/commands, reason: [Failed to login: cannot find Authorization header], retrying again.