Loading SMPL with Texture as Pytorch3D mesh

[pramishp]

Hello everyone, I am facing an issue while trying to load texture from PeopleSnapshot dataset into SMPL mesh.

In order to get verts_uv, I am using the official smpl_uv.obj file provided by https://smpl.is.tue.mpg.de/ as shown:

obj_path = 'path to smpl_uv.obj'
_, _, aux = load_obj(obj_path)
verts_uv = aux[1][:, :]

To load the texture map, I am using the image provided officially.

# texture_image_path = 'path to smpl_uv_20200910.png'
tex_image  = load_texture_from_image(texture_image_path)

And, I am passing in the body params to get output verts as shown:

model_folder = 'path to smpl models'
model_type = smpl_params['model_type']
gender = smpl_params['gender']

body_model_params = {
        'betas': torch.from_numpy(smpl_params['betas'][None, ...]).float(),
        'global_orient': torch.from_numpy(smpl_params['global_orient'][None, ...]).float(),
        'transl': torch.from_numpy(smpl_params['transl'][None, ...]).float(),
        'v_personal': torch.from_numpy(smpl_params['v_personal'][None, ...]).float()
    }
body_model_params.update({
    'body_pose': torch.from_numpy(smpl_params['body_pose'][None, ...]).float(),
})

model = create(model_folder, model_type=model_type, gender=gender)
body_model_out = model(**body_model_params, return_verts=True)

Finally, I am creating Pytorch3D mesh as shown:

vertices = torch.tensor(body_model_out['vertices'][0], device=device)

faces = torch.tensor(model.faces_tensor[...], dtype=torch.int64, device=device)
# Texture
verts_uv = verts_uv.to(device)
tex_image = tex_image.to(device)
texture = TexturesUV(maps=tex_image, faces_uvs=faces[None],
                     verts_uvs=verts_uv[None],
                    )

mesh = Meshes(
    verts=[vertices],
    faces=[faces],
    textures=texture
#     textures=textures_vertex
)

show_mesh_info(mesh)

Output:

mesh.verts_padded().shape = torch.Size([1, 6890, 3])
mesh.faces_padded().shape = torch.Size([1, 13776, 3])
mesh.textures.maps_padded().shape = torch.Size([1, 1000, 1000, 3])
--------
VERTS UVS:
mesh.textures.verts_uvs_padded().shape = torch.Size([1, 7576, 2])
tensor([[0.3716, 0.8811],
        [0.3595, 0.8661],
        [0.3792, 0.8631],
        [0.3916, 0.8723],
        [0.3697, 0.8514]], device='cuda:0')
FACES UVS:
mesh.textures.faces_uvs_padded().shape = torch.Size([1, 13776, 3])
tensor([[1, 2, 0],
        [0, 2, 3],
        [2, 1, 4],
        [4, 1, 5],
        [2, 6, 3]], device='cuda:0')

Plotting the UV map seems perfect.

But, when I render it or preview this, there is distortion.

Fig 1: Rendering Result

Fig 2: Preview Result

How can I fix it ?

[BukuBukuChagma]

Did you figure this out?

[JunukCha]

I've implemented it. Please refer to this link.
https://github.com/JunukCha/textured_smpl.git