Strange results from ZJU-Mocap 315 sequence

[shanemankiw]

Hi, thank you for sharing your work!

I tried to train your model on ZJU-Mocap 315 sequence following your configuration, the only difference is that I used only one RTX 3090. The training took about 2 days yet the evaluation result seems strange.

This is val_ind result:

This is ood result:

The metrics seem to be acceptable though.
This is ind metrics log:
step=10000, test_render_every=5253, psnr=27.6131, ssim=0.9523
step=20000, test_render_every=5253, psnr=28.0429, ssim=0.9556
step=30000, test_render_every=5253, psnr=28.2928, ssim=0.9566
step=40000, test_render_every=5253, psnr=27.9075, ssim=0.9584
step=50000, test_render_every=5253, psnr=28.7775, ssim=0.9605
step=60000, test_render_every=5253, psnr=28.2880, ssim=0.9582
step=70000, test_render_every=5253, psnr=28.5239, ssim=0.9606
step=80000, test_render_every=5253, psnr=28.9092, ssim=0.9616
step=90000, test_render_every=5253, psnr=28.6294, ssim=0.9594
step=100000, test_render_every=5253, psnr=28.6554, ssim=0.9586
step=110000, test_render_every=5253, psnr=28.7274, ssim=0.9584
step=120000, test_render_every=5253, psnr=28.6370, ssim=0.9589
step=130000, test_render_every=5253, psnr=28.4819, ssim=0.9584
step=140000, test_render_every=5253, psnr=28.7621, ssim=0.9583
step=150000, test_render_every=5253, psnr=28.6087, ssim=0.9575
step=160000, test_render_every=5253, psnr=28.4336, ssim=0.9555
step=170000, test_render_every=5253, psnr=28.6071, ssim=0.9572
step=180000, test_render_every=5253, psnr=28.5575, ssim=0.9559
step=190000, test_render_every=5253, psnr=28.5406, ssim=0.9569
step=200000, test_render_every=5253, psnr=28.5062, ssim=0.9558
step=210000, test_render_every=5253, psnr=28.2677, ssim=0.9553
step=220000, test_render_every=5253, psnr=28.3719, ssim=0.9560
step=230000, test_render_every=5253, psnr=28.3678, ssim=0.9565
step=240000, test_render_every=5253, psnr=28.3076, ssim=0.9561
step=250000, test_render_every=5253, psnr=28.3422, ssim=0.9564
step=260000, test_render_every=5253, psnr=28.3457, ssim=0.9562
step=270000, test_render_every=5253, psnr=28.2343, ssim=0.9560
step=280000, test_render_every=5253, psnr=28.2339, ssim=0.9557
step=290000, test_render_every=5253, psnr=28.1508, ssim=0.9561
step=300000, test_render_every=5253, psnr=28.2380, ssim=0.9562

This is ood metrics log:
step=10000, test_render_every=876, psnr=24.9693, ssim=0.9287
step=20000, test_render_every=876, psnr=25.3594, ssim=0.9310
step=30000, test_render_every=876, psnr=25.3456, ssim=0.9363
step=40000, test_render_every=876, psnr=25.7967, ssim=0.9402
step=50000, test_render_every=876, psnr=25.8560, ssim=0.9385
step=60000, test_render_every=876, psnr=25.7005, ssim=0.9362
step=70000, test_render_every=876, psnr=25.9159, ssim=0.9374
step=80000, test_render_every=876, psnr=25.9287, ssim=0.9380
step=90000, test_render_every=876, psnr=25.9482, ssim=0.9385
step=100000, test_render_every=876, psnr=25.7621, ssim=0.9363
step=110000, test_render_every=876, psnr=25.6561, ssim=0.9364
step=120000, test_render_every=876, psnr=25.6878, ssim=0.9379
step=130000, test_render_every=876, psnr=25.6890, ssim=0.9383
step=140000, test_render_every=876, psnr=25.8649, ssim=0.9381
step=150000, test_render_every=876, psnr=25.8306, ssim=0.9383
step=160000, test_render_every=876, psnr=25.7063, ssim=0.9366
step=170000, test_render_every=876, psnr=25.5456, ssim=0.9353
step=180000, test_render_every=876, psnr=25.5683, ssim=0.9353
step=190000, test_render_every=876, psnr=25.5363, ssim=0.9356
step=200000, test_render_every=876, psnr=25.5620, ssim=0.9333
step=210000, test_render_every=876, psnr=25.4459, ssim=0.9329
step=220000, test_render_every=876, psnr=25.3567, ssim=0.9333
step=230000, test_render_every=876, psnr=25.3890, ssim=0.9334
step=240000, test_render_every=876, psnr=25.4219, ssim=0.9330
step=250000, test_render_every=876, psnr=25.3964, ssim=0.9336
step=260000, test_render_every=876, psnr=25.3680, ssim=0.9336
step=270000, test_render_every=876, psnr=25.3631, ssim=0.9335
step=280000, test_render_every=876, psnr=25.3580, ssim=0.9333
step=290000, test_render_every=876, psnr=25.3318, ssim=0.9334
step=300000, test_render_every=876, psnr=25.3467, ssim=0.9337

[liruilong940607]

Hey sorry for the late reply. I didn't keep track of the issues here.

This is very odd. We did verified the code can reproduce the results in the paper. Have you already figured out something? Maybe a data setup issue?