Abstract

3D human pose estimation is estimating the position of the main body joints in the 3D space from 2D images. It remains a challenging problem despite being well studied in computer vision domain. This stems from the ambiguity caused by capturing 2D imagery from 3D objects and thus the loss of depth information. 3D human pose estimation is especially challenging when not all the human body is present (visible) in the input 2D image. This work proposes solutions to reconstruct the 3D human pose from a 2D image under partial body presence. Partial body presence includes all the cases in which some of the body's main joints do not fall inside the image. We propose two different deep learning based approaches to address partial body presence: 1) 3D pose estimation from 2D poses estimated from the 2D input image and 2) 3D pose estimation directly from the 2D input image. In both approaches, we use Convolutional Neural Networks (CNN) for regression. These networks are designed and trained to work under partial body presence but output the full 3D human pose (i.e., including not visible joints). In addition, we propose a detection CNN network to detect those joints present in the input image. We then propose to integrate both regression and detection networks so to estimate the partial 3D human pose, in addition to the full 3D human pose estimated by the regression network. Experimental results comparing the performance of the state-of-the-art demonstrate the effectiveness of our approaches under partial body presence. Experimental results also show that the direct regression of the 3D human pose from 2D images yields more accurate estimation compared to having 2D pose estimation as an intermediate stage.