Problem encountered in the training process

[Pepper-FlavoredChewingGum]

Thank you for your great work of open source. When I was training the building scene, I encountered the following error. Could you give me some advice?
My configuration is pytorch 2.0.0 and cuda 11.8. The instruction for training is as follows:
bash script/train/train-building-8blocks.sh 4 0,1,2,3 ./tmp

Sincerely look forward to your reply!

[Pepper-FlavoredChewingGum]

The problem I encountered seems to be similar to issue 3 , but the error occurred in different training stages.

[Jixuan-Fan]

Hi @Pepper-FlavoredChewingGum,
This error also occurs during the camera loading process and might be caused by memory exhaustion. You can open a new terminal while running the program and use the command watch -n 1 free -h to monitor whether the memory usage keeps increasing until it is fully consumed, resulting in the error.

[Pepper-FlavoredChewingGum]

Hi @Jixuan-Fan
I have encountered a new problem. I hope to find the coordinates of the point cloud, which is self._xyz in the initial code. But I didn't find this variable in the code of Momentum-GS. I found a variable named all_xyz, but its shape is [n, n_offsets, 3], which is not what I want to find the coordinates of the gaussian center. Can you tell me where I can find this variable?

[Jixuan-Fan]

Hi @Jixuan-Fan I have encountered a new problem. I hope to find the coordinates of the point cloud, which is self._xyz in the initial code. But I didn't find this variable in the code of Momentum-GS. I found a variable named all_xyz, but its shape is [n, n_offsets, 3], which is not what I want to find the coordinates of the gaussian center. Can you tell me where I can find this variable?

Is this what you were looking for?

Momentum-GS/gaussian_renderer/__init__.py

Line 233 in f787805

xyz, color, opacity, scaling, rot, neural_opacity, mask = generate_neural_gaussians(viewpoint_camera, pc, visible_mask, is_training=is_training)