Abstract

Fashion landmarks are the functional key-points on the apparels that can be used for a more discriminative visual analysis of the apparel images. Such a framework can facilitate apparel alignment in displaying apparel images on the websites or help build a system to ensure dress code in a particular environment. However, challenges such as background clutter, human poses, scales and apparel variation can render such a task difficult. We present a conceptually simple, flexible, and general framework for apparels’ landmark detection that can be simultaneously used for apparel classification and localization. In addition to the position of the landmarks in the apparels, we also classify the landmarks as visible or occluded in the same framework. The fashion landmark detection task is similar to joint localization and detection problems like human pose estimation, hence our approach extends stacked hourglass architecture, originally proposed to solve human pose estimation. We perform all these tasks in parallel using multi-task learning. Our proposed convolutional neural network is end-to-end differentiable and simple to train, since all these tasks are performed on the same architecture without any additional parameters to learn. Over the past few years, many modifications have been proposed to improve this architecture. We also compare the performances of some of these different variations of stacked hourglass architectures. These architectures leverage both global and local features captured by the deep convolutional neural networks to better localize the apparel in the image as well as the landmarks in those apparels. We test and analyze our results on DeepFashion dataset. We also weigh the trade-offs of the detecting the landmarks in a category-aware environment, i.e., pre-classified apparels and category-agnostic environment, i.e., unclassified apparels.