Convergence in distribution

This lecture discusses convergence in distribution, first for sequences of random variables and then for sequences of random vectors.

Table of contents

1. The intuition
2. How to define convergence
3. Definition for sequences of random variables
4. Example - Maximum of uniform random variables
5. Proper distribution functions
6. Definition for sequences of random vectors
7. Relation between univariate and multivariate convergence
8. Solved exercises
   1. Exercise 1
   2. Exercise 2
   3. Exercise 3

   The intuition

   We have previously explained that different concepts of convergence are based on different ways of measuring the distance between two random variables (how "close to each other" two random variables are).

   The concept of convergence in distribution is based on the following intuition: two random variables are "close to each other" if their distribution functions are "close to each other".

   How to define convergence

   Let us consider a sequence of random variables and a generic random variable belonging to the sequence.

   Denote by the distribution function of . is a function .

   Once we fix , the following facts are true:

   • the value associated to the point is a real number;
   • the sequence is a sequence of real numbers;
   • we can easily assess whether the sequence is convergent by using the standard definition of convergence of a sequence of real numbers.

   If the sequence is convergent, its limit depends on the specific we have fixed.

   A sequence of random variables is said to be convergent in distribution if and only if the sequence is convergent for any choice of , except possibly for some "special values" of where is not continuous in .

   Definition for sequences of random variables

   The following definition summarizes what we have said above.

   Definition Let be a sequence of random variables. Denote by the distribution function of . We say that is convergent in distribution (or convergent in law) if and only if there exists a distribution function such that the sequence converges to for all points where is continuous.

   If a random variable has distribution function , then is called the limit in distribution (or limit in law) of the sequence and convergence is indicated by

   Note that convergence in distribution only involves the distribution functions of the random variables belonging to the sequence and that these random variables need not be defined on the same sample space.

   On the contrary, the modes of convergence we have discussed in previous lectures (pointwise convergence, almost sure convergence, convergence in probability, mean-square convergence) require that all the variables in the sequence be defined on the same sample space.

   Example - Maximum of uniform random variables

   Let us make an example

   Let be a sequence of IID random variables all having a uniform distribution on the interval .

   In other words, the distribution function of is

   

   The distribution function of is

   

   Thus,

   

   Since we have where is the distribution function of an exponential random variable.

   Therefore, the sequence converges in law to an exponential distribution.

   Proper distribution functions

   Let be a sequence of random variables and denote by the distribution function of .

   Suppose that we find a function such that for all where is continuous.

   How do we check that is a proper distribution function, so that we can say that the sequence converges in distribution?

   As explained in the glossary entry on distribution functions, we need to check that satisfies the four properties characterizing proper distribution functions, that is, must be increasing, right-continuous and its limits at minus and plus infinity must be and

   Definition for sequences of random vectors

   The definition of convergence in distribution of a sequence of random vectors is almost identical to that given for random variables.

   We just need to replace distribution functions in the above definition with joint distribution functions.

   Definition Let be a sequence of random vectors. Denote by the joint distribution function of . We say that is convergent in distribution (or convergent in law) if and only if there exists a joint distribution function such that the sequence converges to for all points where is continuous.

   If a random vector has joint distribution function , then is called the limit in distribution (or limit in law) of the sequence and convergence is indicated by

   Relation between univariate and multivariate convergence

   It is important to note that for other notions of stochastic convergence (in probability, almost sure and in mean-square), the convergence of each single entry of the random vector is necessary and sufficient for their joint convergence, that is, for the convergence of the vector as a whole. Instead, for convergence in distribution, the individual convergence of the entries of the vector is necessary but not sufficient for their joint convergence.

   Solved exercises

   Below you can find some exercises with explained solutions.

   Exercise 1

   

   Let be a sequence of random variables having distribution functions

   Find the limit in distribution (if it exists) of the sequence .

   If , then If , then We now need to verify that the function is a proper distribution function. The function is increasing, continuous, its limit at minus infinity is and its limit at plus infinity is , hence it satisfies the four properties that a proper distribution function needs to satisfy. This implies that converges in distribution to a random variable having distribution function .

   Exercise 2

   

   Let be a sequence of random variables having distribution functions

   Find the limit in distribution (if it exists) of the sequence .

   

   If , then If , then Therefore, the distribution functions converge to the function which is not a proper distribution function, because it is not right-continuous at the point . However, note that the function is a proper distribution function and it is equal to at all points except at the point . But this is a point of discontinuity of . As a consequence, the sequence converges in distribution to a random variable having distribution function .

   Exercise 3

   Let be a sequence of random variables having distribution functions

   Find the limit in distribution (if it exists) of the sequence .

   

   The distribution functions converge to the function This is the same limiting function found in the previous exercise. As a consequence, the sequence converges in distribution to a random variable having distribution function

   How to cite

   Taboga, Marco (2021). "Convergence in distribution", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/convergence-in-distribution.

   Most of the learning materials found on this website are now available in a traditional textbook format.

   Featured pages
   • Wald test
   • Convergence in distribution
   • Chi-square distribution
   • Gamma distribution
   • Student t distribution
   • Wishart distribution
   • Bayes rule
   • Multinomial distribution
   • Independent events
   Main sections
   • Mathematical tools
   • Fundamentals of probability
   • Probability distributions
   • Asymptotic theory
   • Fundamentals of statistics
   • Glossary
   • About Statlect
   • Contacts
   • Cookies, privacy and terms of use
   Glossary entries
   • Null hypothesis
   • Continuous mapping theorem
   • Loss function
   • Alternative hypothesis
   • Convolutions
   • Almost sure
   • To enhance your privacy,
   • we removed the social buttons,
   • but don't forget to share.