Abstract

The ability to correctly recognize natural terrain is regarded as a critical actor in autonomous path planning success. Following recent promise shown by deep learning algorithms, much research has directed focus towards conveying these successes to visual terrain classification, however the scarcity of work directing informed neural network design has proved limiting to these efforts. This work presents an algorithm that can be used to quantify the difficulty of specific image classification tasks and to investigate the characteristics of particular difficulties and trends in a way that is interpretable by humans. An accompanying analytical procedure characterizes such image classification difficulties; identifying what makes some images easily distinguishable exemplars of their class and what makes others readily confused with other classes. Case studies are presented of insights identified through selected example analyses of terrain image classification datasets: discussing relative intensities of terrain classes from images taken by Mars rovers, and the impact of color gradients in separating sand from bedrock in color images of terrain, and its implications for remote sensing hardware used to supply classifier input. Additional investigations cover more general image datasets: the MNIST hand written digits dataset, and key background color features in CIFAR-10. We validated the technique’s potential to architecture design through comparisons to various neural networks, discovering characteristics mutual between predicted difficulty and classification error. The results presented in this paper provide a jumping off point for the analysis of terrain classification difficulty, and can inform designers of autonomous vehicles how challenging it can be to distinguish classes of terrain and provide insight into the risk associated with making traverse decisions.