ケーブルの不具合が原因でBrocadeスイッチポートがフラップする

環境

Brocadeスイッチ

問題

• switchshowでスイッチポートがonlineになっています

/fabos/bin/switchshow :

Index Slot Port Address Media  Speed     State   Proto
============================================================
129   4   33   708100   id   N32    Online    FC  F-Port  10:00:00:10:9b:e7:9c:fc

• portershowでは、複数の link failureloss sync エラーとエラー、 enc_out, crc err, crc g_eof, c3timeout Tx, pcs および uncorr エラーなどの他のメディアエラー が報告されます。

/fabos/cliexec/porterrshow :
      frames     enc    crc    crc    too    too    bad    enc   disc   link   loss   loss   frjt   fbsy    c3timeout    pcs    uncor
     tx     rx    in    err    g_eof   shrt   long   eof    out   c3    fail   sync   sig            tx    rx    err    err
100:   14.1k   5.8k   2.2k  964    932     0     0    32    3.9g  6    12    0     4.4k   0     0     0     0     0     0
/fabos/cliexec/porterrshow :
          frames        enc     crc     crc     too     too     bad     enc    disc    link    loss    loss    frjt    fbsy     c3timeout     pcs      uncor
        tx       rx      in     err     g_eof   shrt    long    eof     out    c3      fail    sync    sig                      tx      rx      err     err
129:   20.6k   20.7k    0       0       0       0       0       0       0       9       2.5k    0       7.2k    0       0       9       0       1.1k   26.0k

/fabos/cliexec/porterrshow:
      frames     enc    crc    crc    too    too    bad    enc   disc   link   loss   loss   frjt   fbsy    c3timeout    pcs    uncor
     tx     rx    in    err    g_eof   shrt   long   eof    out   c3    fail   sync   sig            tx    rx    err    err
  37:   2.3g   6.4g   0     0     0     0     0     0     0    53     2     0     4     0     0    53    0     0     0   
​​​​

• sfpshow Rx & Tx 値が最適な範囲にあることをレポートします。

RX Power:   1.4    dBm (1365.3uW) 
TX Power:   -0.4   dBm (910.2 uW) 

• sfpshow レポートのRx値が小さすぎます。

RX Power:    -15.1   dBm (30.7 uW)
TX Power:    -1.7    dBm (678.9 uW)

• portshow 出力は、ポートがOnline&in _Sync状態であり、 LR_inがOLS_OUTより大きいことを確認します。これは、問題がスイッチポートの外部であることを示します。

portshow 37
portDisableReason: None
[..]
portState: 1    Online  
Protocol: FC
portPhys:  6    In_Sync   portScn:   32    F_Port   
FC Fastwrite: OFF
Interrupts:     48   Link_failure: 2   Frjt:      0
Unknown:       6   Loss_of_sync: 0   Fbsy:      0
Lli:         48   Loss_of_sig:  4   
Proc_rqrd:      159  Protocol_err: 0   
Timed_out:      0   Invalid_word: 0   
Tx_unavail:     0   Invalid_crc:  0
Delim_err:      0   Address_err:  0
Lr_in:        6  Ols_in:     2
Lr_out:       2   Ols_out:    3
Cong_Prim_in:     0
 

• で Fabriclogは、これら2つのポートがフラッピングしていることがわかります。

Fabriclog:
Switch 0; Tue Oct 11 12:34:12 2022 IST (GMT+5:30)
12:34:12.020011 SCN Port Offline;rsn=0x2,g=0x530            D2,P0  D2,P0  15    NA
12:34:12.020017 *Removing all nodes from port               D2,P0  D2,P0  15    NA
12:34:12.112102 SCN Port Offline;rsn=0x0,g=0x532            D2,P0  D2,P0  127   NA
12:34:12.112108 *Removing all nodes from port               D2,P0  D2,P0  127   NA
12:36:40.840204 SCN LR_PORT(0);g=0x530                      D2,P0  D2,P0  15    NA
12:36:40.860941 SCN Port Online; g=0x530,isolated=0         D2,P0  D2,P1  15    NA
12:36:40.861044 Port Elp engaged                            D2,P1  D2,P0  15    NA
12:36:40.861057 *Removing all nodes from port               D2,P0  D2,P0  15    NA

• ルール defALL_32GSWL_SFPRXP_63 と defALL_OTHER_F_PORTSSTATE_CHG_5 トリガーは errdumploss signal、および link failuresframe timeout detected エラーとともにログの下で実行されます。

2022/10/11-12:41:00, [MAPS-1004], 46328, SLOT 1 | FID 128, INFO, XXX, SFP 3/15, Condition=ALL_32GSWL_SFP(RXP<=63), Current Value:[RXP, 0 uW], RuleName=defALL_32GSWL_SFPRXP_63, Dashboard Category=Port Health.
2022/10/11-12:41:00, [MAPS-1004], 46329, SLOT 1 | FID 128, INFO, XXX, SFP 12/15, Condition=ALL_32GSWL_SFP(RXP<=63), Current Value:[RXP, 0 uW], RuleName=defALL_32GSWL_SFPRXP_63, Dashboard Category=Port Health.
2022/10/11-12:43:00, [MAPS-1004], 46330, SLOT 1 | FID 128, INFO, XXX, SFP 3/15, Condition=ALL_32GSWL_SFP(RXP<=63), Current Value:[RXP, 0 uW], RuleName=defALL_32GSWL_SFPRXP_63, Dashboard Category=Port Health.
2022/10/11-12:43:00, [MAPS-1004], 46331, SLOT 1 | FID 128, INFO, XXX, SFP 12/15, Condition=ALL_32GSWL_SFP(RXP<=63), Current Value:[RXP, 0 uW], RuleName=defALL_32GSWL_SFPRXP_63, Dashboard Category=Port Health.

2023/02/20-16:42:20 (IST), [MAPS-1003], 9812, SLOT 2 | FID 128 | PORT 4/33, WARNING, switch, slot4 port33, F-Port 4/33, Condition=ALL_HOST_PORTS(STATE_CHG/min>5), Current Value:[STATE_CHG, 6], RuleName=defALL_HOST_PORTSSTATE_CHG_5, Dashboard Category=Port Health, Quiet Time=None.
2023/02/20-16:42:20 (IST), [MAPS-1003], 9813, SLOT 2 | FID 128 | PORT 4/33, WARNING, switch, slot4 port33, F-Port 4/33, Condition=ALL_OTHER_F_PORTS(STATE_CHG/min>5), Current Value:[STATE_CHG, 6], RuleName=defALL_OTHER_F_PORTSSTATE_CHG_5, Dashboard Category=Port Health, Quiet Time=None.

2022/06/26-03:53:42, [MAPS-1003], 42403, SLOT 2 | FID 128, WARNING, switch, slot6 port9, U-Port 6/9, Condition=ALL_PORTS(LOSS_SIGNAL/min>3), Current Value:[LOSS_SIGNAL, 386 LOS], RuleName=defALL_PORTSLOSS_SIGNAL_3, Dashboard Category=Port Health.
2022/06/26-03:54:42, [MAPS-1003], 42404, SLOT 2 | FID 128, WARNING, switch, slot6 port9, U-Port 6/9, Condition=ALL_PORTS(LOSS_SIGNAL/min>3), Current Value:[LOSS_SIGNAL, 378 LOS], RuleName=defALL_PORTSLOSS_SIGNAL_3, Dashboard Health.

2021/10/23-23:38:47, [MAPS-1003], 55690, SLOT 1 | FID 128, WARNING, Fabric1, slot10 port31, U-Port 10/31, Condition=ALL_PORTS(LF/min>3), Current Value:[LF, 4], RuleName=defALL_PORTSLF_3, Dashboard Category=Port Health.
2021/10/23-04:24:46, [PORT-1003], 53615, SLOT 1 | FID 128, WARNING, Fabric1, Port 223 Faulted because of many Link Failures.

2020/04/08-21:35:56, [C3-1014], 2556, CHASSIS, WARNING, Brocade6510,  Link Reset on Port S0,P8(12) vc_no=0 crd(s)lost=12 auto trigger.
2020/04/09-09:03:42, [C3-1014], 2557, CHASSIS, WARNING, Brocade6510,  Link Reset on Port S0,P8(12) vc_no=0 crd(s)lost=12 auto trigger.

2023/10/15-01:03:18, [AN-1014], 589, FID 128, INFO, SAN32G_FME1937T016_C2A1_F1, Frame timeout detected, tx port 37 rx port 15, sid 650900, did 632501, timestamp 2023-10-15 01:03:18 .
2023/10/15-01:03:18, [LOG-1000], 594, FID 128, INFO, SAN32G_FME1937T016_C2A1_F1, Previous message repeated 5 time(s).
 

• RuleName=defALL_32GSWL_SFPRXP_63 Rx電力が低下し始め、アップストリームの問題I、 SFPまたはケーブルに障害があることを示している場合にトリガーされます。