Abstract

Point clouds are typically captured from surfaces of real-world objects using scanning devices. This scanning process invariably results in the loss of sharp edges as well as other geometric features of the original surface, which we collectively refer to as "fine features". This thesis explores leveraging recent advances in deep learning in order to recover fine features of surfaces which were lost when acquiring the point cloud. We first focus on reconstructing sharp edges of the original surface. We define a new concept – a sharpness field – over the underlying surface of a point cloud, whose ridges give the locations of sharp edges even at points not originally sampled from the surface. We then demonstrate that with appropriate training data, deep neural networks can be trained to compute the sharpness field for a point cloud. We evaluate several different local neighborhood representations and deep learning models to improve the accuracy of sharpness field computation across different neighborhood scales. Some applications of the sharpness field are then described. The most important such application is feature-aware smoothing: using the computed sharpness field to preserve sharp edges while removing noise from point clouds. Our novel smoothing algorithm shows superior performance in reconstructing sharp edges and corners compared to the state-of-the-art RIMLS algorithm, while also yielding points lying on sharp edges. Other applications of the sharpness field are also presented: generating a graphical representation of sharp edges and segementing a point cloud into smooth surface patches. We then expand the scope of the problem from sharp edges to undersampled fine features in general. We tackle this by developing a unique deep learning approach to point cloud super-resolution using an innovative application of generative adversarial networks (GANs). The novelty of our super-resolution method lies in framing point cloud super-resolution as a domain translation task between heightmaps obtained from point clouds and heightmaps obtained from triangular meshes. By using recent developments in domain translation using GANs, we obtain results qualitatively and quantitatively superior to state-of-the-art point cloud super-resolution methods, all while using a radically different deep learning approach which is also more computationally efficient.

The main contributions of this thesis are:
1. Establishing sharpness field computation as a novel method for localizing sharp edges in 3D point clouds.
2. Several new data-driven methods to compute the sharpness field for a point cloud using different local neighborhood representations and machine learning models.
3. A novel feature-aware smoothing algorithm for denoising point clouds while preserving sharp edges, using the aforementioned sharpness field. This method produces denoising results superior to the state-of-the-art RIMLS smoothing method.
4. An innovative application of GAN-based domain translation, in order to transform sparse heightmaps obtained from point cloud neighborhoods into dense heightmaps obtained from triangular meshes.
5. A unique method for reconstrucing fine features in point clouds, by using heightmap domain translation to perform point cloud super-resolution. The reconstructed surfaces are qualitatively and quantitatively superior to those produced by state-of-the-art point cloud super-resolution methods.

Other contributions include:
1. An algorithm for extracting an explicit graphical representation of the sharp edges of a point cloud, using the sharpness field.
2. An algorithm for segmenting a point cloud into smooth patches, using the sharpness field.
3. Feature-aware smoothing algorithms which incorporate the aforementioned edge graph and patch segmentation.