1506.02178

IJCV 2016

Dimitrios Tzionas, Luca Ballan, Abhilash Srikantha, Pablo Aponte, Marc Pollefeys, Juergen Gall

Track hands that interact with each other and objects (rigid & articulated)

single objective function that aligns the model with the observed data, with collision and physical model that tests for stability

generative model + discriminative classifier trained on salient points (fingertips)

physics simulation: test configuration by taking into account friction, gravity, mass and verify object displacement as proxy for stability

collision detection : avoid inter-penetration

Objective function almost everywhere differentiable ==> can be optimized with standard optimization techniques

37 DoF for the hand

Takes into account :

coherence between mesh and depth data
- depth from model, and 3d Chamer distance (point to point or point to plane using normal info)
- from data to model only with depth discontinuities (otherwise expensive to evaluate because recomputed for each new model)
consistency of the estimates of the hands and the manipulated object
- fingertip detector
- manual fingertip annotation
- Hough forest on patches
- high threshold to only recover high confidence fingertip detections
penalization of collisions (intersections)
- bounding volume hierarchy (BVH) to detect colisions BVH: tree-like structure that encloses objects in larger volumes that contain an increasing number of objects, if parent volumes do not intersect, children volumes do not either so speed of collision detection is increased.
physics
- penalize distance of object to identified contact point
anatomical joint limits
- exponential around stability postion
- (Albrecht and al 2003)
regularization by penalizing difference from previously estimated joint angles

test first if estimate is 'physically correct' : if center did not move significantly (less then 3mm in 35 iterations)
if not, find distance between finger parts (phallanges) and the object. If distance <10mm : finger part is candidate for contact
combination between 2 and 4 candidates are tested
select combination with lowest displacement of object
introduce an energy that forces contact for those points