Abstract

Deep learning-based 3D semantic segmentation requires a large volume of training data to achieve robust generalization and prevent overfitting. However, acquiring extensive 3D datasets remains a significant challenge. This study addresses data scarcity in 3D semantic segmentation by leveraging a 2D projection-based approach, utilizing advancements in 2D segmentation techniques to enhance 3D segmentation performance. The key insight is that while 3D scene availability may be limited, an unlimited number of 2D projections can be generated, which can then be reprojected back into 3D space. This approach enables models to benefit from state-of-the-art 2D segmentation techniques while mitigating information loss during dimensional transformation. This study evaluates widely used 2D projection techniques, including spherical projection, bird’s-eye view, and perspective projection, to convert 3D data into the 2D domain. 2D CNN-based segmentation models, such as U-Net and SegFormer, are then applied to generate predictions, which are subsequently reprojected into the 3D domain for final segmentation. Experimental results demonstrate that using data from only a single area in the S3DIS dataset, our 2D projection-based method achieves a 3D IoU score of 47.21%. Moreover, when using a 1:1 train-test ratio on similar rooms, 3D IoU of 56.23% is achieved, highlighting that even with limited data, projection-based approaches offer a viable and effective solution for 3D point cloud semantic segmentation.