Abstract

Multi-Object Tracking (MOT) remains a challenging problem, particularly in crowded scenes with occlusion, appearance ambiguity, and non-linear motion. Conventional MOT frameworks often rely on appearance-based Re-Identification (Re-ID) and Intersection-over-Union (IoU) of object bounding boxes for object association. However, these cues become unreliable when objects are visually similar or overlapping, and computing pixel-level IoU for segmentation masks can be computationally expensive.

In this thesis, we propose two complementary MOT frameworks that incorporate monocular depth and segmentation cues to improve robustness in association. The first zero-shot depth-aware framework is training-free and introduces a Hierarchical Alignment Score (HAS), a novel metric that combines coarse bounding box IoU with fine-grained mask-level IoU using promptable segmentation. This hierarchical formulation improves matching precision in cluttered or occluded scenes.

The second framework avoids computing segmentation IoU altogether. Instead, it leverages a self-supervised encoder to fuse and refine depth-segmentation features into temporally stable embeddings, which are then used as an additional similarity signal in the association process. This reduces computational overhead while improving robustness to noise and appearance variation.

Both approaches operate under the efficient Tracking-by-Detection (TBD) paradigm and extend conventional 2D association strategies with spatially expressive cues. Evaluations on DanceTrack and SportsMOT benchmarks with non-linear motion demonstrate competitive performance, highlighting the utility of depth and segmentation as underutilized, yet powerful, cues for robust non-linear MOT.