Kernel 6.8.0-76060800daily20240311 causes xHCI host controller not responding, assume dead

[DASPRiD]

Distribution (run cat /etc/os-release):

NAME="Pop!_OS"
VERSION="22.04 LTS"

Related Application and/or Package Version (run apt policy $PACKAGE NAME):

linux-headers-6.8.0-76060800daily20240311:
  Installed: 6.8.0-76060800daily20240311.202403110203~1710198088~22.04~1a3dbc7

Issue/Bug Description:
After the upgrade yesterday from linux-headers-6.6.10-76060610 to linux-headers-6.8.0-76060800daily20240311 my system fails to initialize one of the USB root hubs with the following messages in dmesg:

[  591.951911] xhci_hcd 0000:06:00.0: WARNING: Host System Error
[  617.365628] xhci_hcd 0000:06:00.0: Abort failed to stop command ring: -110
[  617.365638] xhci_hcd 0000:06:00.0: xHCI host controller not responding, assume dead
[  617.365641] xhci_hcd 0000:06:00.0: HC died; cleaning up
[  617.365730] xhci_hcd 0000:06:00.0: Error while assigning device slot ID: Command Aborted
[  617.365740] xhci_hcd 0000:06:00.0: Max number of devices this xHCI host supports is 32.
[  617.365742] usb usb9-port1: couldn't allocate usb_device
[  617.365874] usb usb10-port1: couldn't allocate usb_device

This occurs consistently after every boot. Booting into the old kernel resolves the issue.

Steps to reproduce (if you know):

Since I don't know if this is related to some specific USB controller hardware, I can't tell. My mainboard is a ROG STRIX Z790-E GAMING WIFI.

Expected behavior:

Initialization error should not occur.

Other Notes:

lsusb output (on old kernel):

Bus 012 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 011 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 009 Device 007: ID 0424:2740 Microchip Technology, Inc. (formerly SMSC) Hub Controller
Bus 009 Device 006: ID 28de:2102 Valve Software Valve VR Radio & HMD Mic
Bus 009 Device 005: ID 28de:2102 Valve Software Valve VR Radio
Bus 009 Device 004: ID 28de:2300 Valve Software Index HMD
Bus 009 Device 003: ID 0424:2744 Microchip Technology, Inc. (formerly SMSC) Hub
Bus 009 Device 002: ID 28de:2613 Valve Software 
Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 010 Device 003: ID 28de:2400 Valve Software 3D Camera
Bus 010 Device 002: ID 0424:5744 Microchip Technology, Inc. (formerly SMSC) Hub
Bus 010 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 009: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 008: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 007: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 006: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 005: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 004: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 003: ID 28de:2102 Valve Software Dongle McDongleface
Bus 005 Device 002: ID 0424:2517 Microchip Technology, Inc. (formerly SMSC) Hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 005: ID 05e3:0612 Genesys Logic, Inc. Hub
Bus 002 Device 004: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
Bus 002 Device 003: ID 0bda:0420 Realtek Semiconductor Corp. 4-Port USB 3.0 Hub
Bus 002 Device 002: ID 174c:3074 ASMedia Technology Inc. ASM1074 SuperSpeed hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 016: ID 046d:c068 Logitech, Inc. G500 Laser Mouse
Bus 001 Device 013: ID 1050:0407 Yubico.com Yubikey 4/5 OTP+U2F+CCID
Bus 001 Device 014: ID b58e:9e84 Blue Microphones Yeti Stereo Microphone
Bus 001 Device 010: ID 05e3:0610 Genesys Logic, Inc. Hub
Bus 001 Device 008: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
Bus 001 Device 004: ID 0b05:1a52 ASUSTek Computer, Inc. USB Audio
Bus 001 Device 007: ID 046d:082d Logitech, Inc. HD Pro Webcam C920
Bus 001 Device 006: ID 046a:0023 Cherry GmbH Keyboard
Bus 001 Device 005: ID 046d:c900 Logitech, Inc. Litra Glow
Bus 001 Device 003: ID 0bda:5420 Realtek Semiconductor Corp. 4-Port USB 2.0 Hub
Bus 001 Device 019: ID 8087:0032 Intel Corp. AX210 Bluetooth
Bus 001 Device 017: ID 0b05:19af ASUSTek Computer, Inc. AURA LED Controller
Bus 001 Device 020: ID 1e71:2007 NZXT NZXT USB Device
Bus 001 Device 018: ID 1b1c:1c06 Corsair                       
Bus 001 Device 015: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 011: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 012: ID 1e71:2001 NZXT NZXT USB Device
Bus 001 Device 009: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 002: ID 174c:2074 ASMedia Technology Inc. ASM1074 High-Speed hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

[lanthth]

I'm having this problem, and I'm also having another problem. #308

[DASPRiD]

Still an issue on linux-image-6.9.3-76060903-generic.

[DASPRiD]

Some additional information: This is not affecting the USB controllers on the motherboard, but specifically an expansion card, namely "StarTech.com 4 Port USB 3.0 PCIe".

[numberer6]

Same problem on the latest PopOS Linux Kernel after reboot, but different USB3 hardware manufacturer:

[ 2235.250278] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.09
[ 2235.250286] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2235.250289] usb usb3: Product: xHCI Host Controller
[ 2235.250292] usb usb3: Manufacturer: Linux 6.9.3-76060903-generic xhci-hcd
[ 2235.250294] usb usb3: SerialNumber: 0000:03:00.0
[ 2236.056078] usb 4-1.3: Product: Best USB Device
[ 2236.056082] usb 4-1.3: Manufacturer: ULT-Best
[ 2236.056086] usb 4-1.3: SerialNumber: 235678C218CA

I did do this to reset the device, and so far it is "more stable":

echo -n "0000:03:00.0" > /sys/bus/pci/drivers/xhci_hcd/unbind
sleep 5
echo -n "0000:03:00.0" > /sys/bus/pci/drivers/xhci_hcd/bind

To reproduce this error I did the following:

1. Reboot the system
2. Mount the USB-3 external disk
3. Ran md5sum /mnt/disk/data/large-file.dat
4. After 20min or so, the "not responding, assume dead" error appeared and the disk was no longer accessible at all
   [ 1256.888663] xhci_hcd 0000:03:00.0: xHCI host controller not responding, assume dead

The file being read is a 500GB dd disk image stored on the external USB-3 disk formatted as ext4. I am stress testing both the mounted 4TB disk and the new USB-3 PCIe adapter card.

After the unbind/bind cycle, I was able to complete one md5sum pass over the 500GB file. I am continuing testing to see if "I can break" this device again.

After more disk read/write access, the "assume dead" error occurred again.