Abstract

Information Geometry is a relatively young branch of Mathematics, which roots back to studies of invariant geometrical structure involved in statistical inference. It de- fines a Riemannian metric together with dually coupled affine connections in a mani- fold of probability distributions. These structures provide tools not only for studying statistical inference but also for research in wider areas of information sciences, such as machine learning, signal processing, optimization, and even neuroscience, not to mention mathematics and physics. The aim of this thesis is to give a brief intro- duction to Information Geometry with focus on the exponential family. In Chapter 1, we first introduce the notion and basic properties of statistical models. We then define some common notions in information geometry such as Fisher information, Christoffel symbols, connections, Skewness tensor, geodesic and Jeffreys Prior. We also introduce the geometry of entropy, including entropy, Kullback-Leibler diver- gence (or relative entropy) and information energy on statistical models. Chapter 2 focuses on the geometry of the exponential family of probability distributions. Ex- amples and properties of exponential families are firstly discussed in this chapter. Fisher metric and geodesics are worked out explicitly for common exponential fam- ilies. Chapter 3 contains important examples of exponential families for which the entropy, Kullback-Leibler relative entropy and information energy are worked out explicitly. This chapter deals also with the problem of finding the density of maximum entropy subject to the first N moment constraints, with unique solutions for the cases N ≤ 2.