Differential geometry of smooth families of probability distributions

A smoothly parametrized family of probability distributions forms a manifold. Its differential-geometrical structures are elucidated by introducing a Riemannian metric and one-parameter families of affine connections ( $$\alpha $$ -connections). There exists duality between $$\alpha $$ - and $$-\alpha $$ -connections, so that an $$\alpha $$ -flat manifold is automatically $$-\alpha $$ -flat. In an $$\alpha $$ -flat manifold, a natural quasi-distance, called the $$\alpha $$ -divergence can naturally be introduced from the intrinsic dualistic structure. When $$\alpha =-1$$ , this reduces to the Kullback divergence, and when $$\alpha =0$$ it is the Hellinger distance (which in this case is related to the Riemannian distance). The geometry of $$\alpha $$ -divergence is connected with the $$\alpha $$ - and $$-\alpha $$ -geodesics due to the $$\alpha $$ - and $$-\alpha $$ -connections. It is important in many statistical problems to approximate a distribution by one belonging to a prescribed family of distributions that is closest to the distribution in the sense of the $$\alpha $$ -divergence. This problem of $$\alpha $$ -approximation is solved with the help of the $$\alpha $$ -geodesic and $$-\alpha $$ -geodesic. The geometrical structures of the function space of distributions are also touched upon.