AttributeError: 'NoneType' object has no attribute 'groups'

[njzjz]

>>> nvgpu.gpu_info()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jz748/anaconda3/lib/python3.7/site-packages/nvgpu/__init__.py", line 8, in gpu_info
    gpu_infos = [re.match('GPU ([0-9]+): ([^(]+) \(UUID: ([^)]+)\)', gpu).groups() for gpu in gpus]
  File "/home/jz748/anaconda3/lib/python3.7/site-packages/nvgpu/__init__.py", line 8, in <listcomp>
    gpu_infos = [re.match('GPU ([0-9]+): ([^(]+) \(UUID: ([^)]+)\)', gpu).groups() for gpu in gpus]
AttributeError: 'NoneType' object has no attribute 'groups'

nvidia-smi output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.14       Driver Version: 430.14       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX TIT...  Off  | 00000000:01:00.0 Off |                  N/A |
| 27%   45C    P0    81W / 250W |      0MiB /  6083MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  TITAN X (Pascal)    Off  | 00000000:0B:00.0 Off |                  N/A |
| 27%   45C    P0    57W / 250W |      0MiB / 12196MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX TIT...  Off  | 00000000:0D:00.0 Off |                  N/A |
| 27%   45C    P0    81W / 250W |      0MiB /  6083MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  GeForce GTX 980 Ti  Off  | 00000000:0E:00.0 Off |                  N/A |
| 17%   45C    P0    66W / 250W |      0MiB /  6083MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  GeForce GTX TIT...  Off  | 00000000:0F:00.0 Off |                  N/A |
| 28%   45C    P0    82W / 250W |      0MiB /  6083MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   5  GeForce GTX 1080    Off  | 00000000:10:00.0 Off |                  N/A |
| 29%   39C    P0    37W / 180W |      0MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   6  GeForce GTX TIT...  Off  | 00000000:11:00.0 Off |                  N/A |
| 27%   44C    P0    77W / 250W |      0MiB /  6083MiB |      2%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[x45dev]

This problem also exists for Nvidia MIG GPU devices. The output from nvidia-smi (v535.86.10 in my case) is different and completely breaks nvgpu version 0.10.0. :(

[x45dev]

@bzamecnik Please have a look at this - MIG device support in nvidia-smi will need to be supported in nvgpu as it becomes more widely available.

[bzamecnik]

@njzjz & @x45dev What is the output of nvidia-smi -L? Without it I can't fix the regex.

[njzjz]

Sorry, I no longer have access to this machine where I got the error in 2020.

[x45dev]

@njzjz & @x45dev What is the output of nvidia-smi -L? Without it I can't fix the regex.

Thank you for the quick response. I have just left the office but will get the info to you tomorrow. Thank you!

[x45dev]

Hello again @bzamecnik

The following thread has examples of nvidia-smi -L and the full output of nvidia-smi when MIG devices are available: pytorch/serve#1237

Note the following:

1. You can use nvidia-smi to verify whether MIG is active or not with the command nvidia-smi --query-gpu=mig.mode.current --format=csv,noheader with expected string output equals "Enabled" or "Disabled".
2. Whenever MIG devices are available, the 'parent' (hardware) GPU is still listed, but is not considered usable. Ie, in your code this would mean gpus = gpus[1:].
3. The nvidia-smi -L output in the example includes a MIG UUID that starts with "MIG-GPU-". They don't always start with "MIG-GPU" - in my case the UUID starts with "MIG-", so I suggest that you just lazy regex match against whatever is present after the "UUID: " string.
4. The nvidia-smi output provides the MIG-related stats many lines lower down. In your code, that means line_distance = 2 and selected_lines = lines[20:20 + line_distance * gpu_count].

[bzamecnik]

@x45dev Thank you! I'll try to look at that, but I can't guarantee when it will be ready. Any PR fixing this would be welcome.

[x45dev]

@x45dev Thank you! I'll try to look at that, but I can't guarantee when it will be ready. Any PR fixing this would be welcome.

PR submitted. :)
#32