Analysis:
- We noticed lot of aborts being received at Qlogic hba as per below log snippets.
Apr 4 01:29:09 vmkernel: 82:03:10:13.242 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:10): Abort command issued -- 1 2d5e17be 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17bf 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:10): Abort command issued -- 1 2d5e17c2 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17c3 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:10): Abort command issued -- 1 2d5e17c7 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.244 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17c8 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.244 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17cc 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.244 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17ce 2002.
- We noticed SCSI resets SCSI code being instigated and received by the ESXi host.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x41043f9a8c40) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x41043f9a8640) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x41043fb7e640) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)<6>qla2xxx 0000:41:00.1: scsi(7:0:15): Abort command issued -- 1 2d535b63 2002.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x4103bfb26340) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Recommendations:
- Engage the server team to check the aborts on Qlogic HBA.
- Engage HP storage vendor to check SAN issues.
- Check with HP if Qlogic QMH2462 is on latest firmware and driver.
- We noticed lot of aborts being received at Qlogic hba as per below log snippets.
Apr 4 01:29:09 vmkernel: 82:03:10:13.242 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:10): Abort command issued -- 1 2d5e17be 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17bf 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:10): Abort command issued -- 1 2d5e17c2 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17c3 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.243 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:10): Abort command issued -- 1 2d5e17c7 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.244 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17c8 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.244 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17cc 2002.
Apr 4 01:29:09 vmkernel: 82:03:10:13.244 cpu39:4311)<6>qla2xxx 0000:41:00.0: scsi(6:2:6): Abort command issued -- 1 2d5e17ce 2002.
- We noticed SCSI resets SCSI code being instigated and received by the ESXi host.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x41043f9a8c40) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x41043f9a8640) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x41043fb7e640) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)<6>qla2xxx 0000:41:00.1: scsi(7:0:15): Abort command issued -- 1 2d535b63 2002.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)NMP: nmp_CompleteCommandForPath: Command 0x2a (0x4103bfb26340) to NMP device "naa.60060e8005493300000049330000509b" failed on physical path "vmhba4:C0:T0:L15" H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Apr 4 00:44:38 vmkernel: 82:02:25:42.557 cpu11:4159)ScsiDeviceIO: 1688: Command 0x2a to device "naa.60060e8005493300000049330000509b" failed H:0x8 D:0x0 P:0x0 Possible sense data: 0x0 0x0 0x0.
Recommendations:
- Engage the server team to check the aborts on Qlogic HBA.
- Engage HP storage vendor to check SAN issues.
- Check with HP if Qlogic QMH2462 is on latest firmware and driver.