Statement Let X 1 , X 2 , … , X n {\displaystyle \ X_{1},X_{2},\ldots ,X_{n}\ } be an n -sized sample of independent and identically-distributed random variables , each of whose cumulative distribution function is F . {\displaystyle \ F~.} Suppose that there exist two sequences of real numbers a n > 0 {\displaystyle \ a_{n}>0\ } and b n ∈ R {\displaystyle \ b_{n}\in \mathbb {R} \ } such that the following limits converge to a non-degenerate distribution function:
lim n → ∞ P { max { X 1 , … , X n } − b n a n ≤ x } = G ( x ) , {\displaystyle \lim _{n\to \infty }{\boldsymbol {\mathcal {P}}}\left\{{\frac {\ \max\{X_{1},\dots ,X_{n}\}-b_{n}\ }{a_{n}}}\leq x\ \right\}=G(x)\ ,} or equivalently:
lim n → ∞ ( F ( a n x + b n ) ) n = G ( x ) . {\displaystyle \lim _{n\to \infty }{\Bigl (}\ F\left(\ a_{n}\ x+b_{n}\ \right){\Bigr )}^{n}=G(x)~.} In such circumstances, the limiting distribution G {\displaystyle \ G\ } belongs to either the Gumbel , the Fréchet , or the Weibull distribution family .[6]
In other words, if the limit above converges, then up to a linear change of coordinates G ( x ) {\displaystyle G(x)} will assume either the form:[7]
G γ ( x ) = exp ( − ( 1 + γ x ) ( − 1 γ ) ) {\displaystyle G_{\gamma }(x)=\exp \left(-{\Bigl (}1+\gamma \ x{\Bigr )}^{\left({\tfrac {-1\;}{\gamma }}\right)}\right)\quad } for γ ≠ 0 , {\displaystyle \quad \gamma \neq 0\ ,} with the non-zero parameter γ {\displaystyle \ \gamma \ } also satisfying 1 + γ x > 0 {\displaystyle \ 1+\gamma \ x>0\ } for every x {\displaystyle \ x\ } value supported by F {\displaystyle \ F\ } (for all values x {\displaystyle \ x\ } for which F ( x ) ≠ 0 {\displaystyle \ F(x)\neq 0\ } ). Otherwise it has the form:
G 0 ( x ) = exp ( − exp ( − x ) ) {\displaystyle G_{0}(x)=\exp {\bigl (}\ -\exp(-x)\ {\bigr )}\quad } for γ = 0 . {\displaystyle \quad \gamma =0~.} This is the cumulative distribution function of the generalized extreme value distribution (GEV) with extreme value index γ . {\displaystyle \ \gamma ~.\ } The GEV distribution groups the Gumbel, Fréchet, and Weibull distributions into a single composite form.
Conditions of convergence The Fisher–Tippett–Gnedenko theorem is a statement about the convergence of the limiting distribution G ( x ) , {\displaystyle \ G(x)\ ,} above. The study of conditions for convergence of G {\displaystyle \ G\ } to particular cases of the generalized extreme value distribution began with Mises (1936)[3] [5] [4] and was further developed by Gnedenko (1943).[5]
Let F {\displaystyle \ F\ } be the distribution function of X , {\displaystyle \ X\ ,} and X 1 , … , X n {\displaystyle \ X_{1},\dots ,X_{n}\ } be some i.i.d. sample thereof. Also let x m a x {\displaystyle \ x_{\mathsf {max}}\ } be the population maximum: x m a x ≡ sup { x ∣ F ( x ) < 1 } . {\displaystyle \ x_{\mathsf {max}}\equiv \sup \ \{\ x\ \mid \ F(x)<1\ \}~.\ } The limiting distribution of the normalized sample maximum, given by G {\displaystyle G} above, will then be:[7]
Fréchet distribution ( γ > 0 ) {\displaystyle \ \left(\ \gamma >0\ \right)} For strictly positive γ > 0 , {\displaystyle \ \gamma >0\ ,} the limiting distribution converges if and only if x m a x = ∞ {\displaystyle \ x_{\mathsf {max}}=\infty \ } and lim t → ∞ 1 − F ( u t ) 1 − F ( t ) = u ( − 1 γ ) {\displaystyle \ \lim _{t\rightarrow \infty }{\frac {\ 1-F(u\ t)\ }{1-F(t)}}=u^{\left({\tfrac {-1~}{\gamma }}\right)}\ } for all u > 0 . {\displaystyle \ u>0~.} In this case, possible sequences that will satisfy the theorem conditions are b n = 0 {\displaystyle b_{n}=0} and a n = F − 1 ( 1 − 1 n ) . {\displaystyle \ a_{n}={F^{-1}}\!\!\left(1-{\tfrac {1}{\ n\ }}\right)~.} Strictly positive γ {\displaystyle \ \gamma \ } corresponds to what is called a heavy tailed distribution.
Gumbel distribution ( γ = 0 ) {\displaystyle \ \left(\ \gamma =0\ \right)} For trivial γ = 0 , {\displaystyle \ \gamma =0\ ,} and with x m a x {\displaystyle \ x_{\mathsf {max}}\ } either finite or infinite, the limiting distribution converges if and only if lim t → x m a x 1 − F ( t + u g ~ ( t ) ) 1 − F ( t ) = e − u {\displaystyle \ \lim _{t\rightarrow x_{\mathsf {max}}}{\frac {\ 1-F{\bigl (}\ t+u\ {\tilde {g}}(t)\ {\bigr )}\ }{1-F(t)}}=e^{-u}\ } for all u > 0 {\displaystyle \ u>0\ } with g ~ ( t ) ≡ ∫ t x m a x ( 1 − F ( s ) ) d s 1 − F ( t ) . {\displaystyle \ {\tilde {g}}(t)\equiv {\frac {\ \int _{t}^{x_{\mathsf {max}}}{\Bigl (}\ 1-F(s)\ {\Bigr )}\ \mathrm {d} \ s\ }{1-F(t)}}~.} Possible sequences here are b n = F − 1 ( 1 − 1 n ) {\displaystyle \ b_{n}={F^{-1}}\!\!\left(\ 1-{\tfrac {1}{\ n\ }}\ \right)\ } and a n = g ~ ( F − 1 ( 1 − 1 n ) ) . {\displaystyle \ a_{n}={\tilde {g}}{\Bigl (}\;{F^{-1}}\!\!\left(\ 1-{\tfrac {1}{\ n\ }}\ \right)\;{\Bigr )}~.}
Weibull distribution ( γ < 0 ) {\displaystyle \ \left(\ \gamma <0\ \right)} For strictly negative γ < 0 {\displaystyle \ \gamma <0\ } the limiting distribution converges if and only if x m a x < ∞ {\displaystyle \ x_{\mathsf {max}}\ <\infty \quad } (is finite) and lim t → 0 + 1 − F ( x m a x − u t ) 1 − F ( x m a x − t ) = u ( − 1 γ ) {\displaystyle \ \lim _{t\rightarrow 0^{+}}{\frac {\ 1-F\!\left(\ x_{\mathsf {max}}-u\ t\ \right)\ }{1-F(\ x_{\mathsf {max}}-t\ )}}=u^{\left({\tfrac {-1~}{\ \gamma \ }}\right)}\ } for all u > 0 . {\displaystyle \ u>0~.} Note that for this case the exponential term − 1 γ {\displaystyle \ {\tfrac {-1~}{\ \gamma \ }}\ } is strictly positive, since γ {\displaystyle \ \gamma \ } is strictly negative. Possible sequences here are b n = x m a x {\displaystyle \ b_{n}=x_{\mathsf {max}}\ } and a n = x m a x − F − 1 ( 1 − 1 n ) . {\displaystyle \ a_{n}=x_{\mathsf {max}}-{F^{-1}}\!\!\left(\ 1-{\frac {1}{\ n\ }}\ \right)~.} Note that the second formula (the Gumbel distribution) is the limit of the first (the Fréchet distribution) as γ {\displaystyle \ \gamma \ } goes to zero.
Examples Fréchet distribution The Cauchy distribution 's density function is:
f ( x ) = 1 π 2 + x 2 , {\displaystyle f(x)={\frac {1}{\ \pi ^{2}+x^{2}\ }}\ ,} and its cumulative distribution function is:
F ( x ) = 1 2 + 1 π arctan ( x π ) . {\displaystyle F(x)={\frac {\ 1\ }{2}}+{\frac {1}{\ \pi \ }}\arctan \left({\frac {x}{\ \pi \ }}\right)~.} A little bit of calculus show that the right tail's cumulative distribution 1 − F ( x ) {\displaystyle \ 1-F(x)\ } is asymptotic to 1 x , {\displaystyle \ {\frac {1}{\ x\ }}\ ,} or
ln F ( x ) → − 1 x a s x → ∞ , {\displaystyle \ln F(x)\rightarrow {\frac {-1~}{\ x\ }}\quad {\mathsf {~as~}}\quad x\rightarrow \infty \ ,} so we have
ln ( F ( x ) n ) = n ln F ( x ) ∼ − − n x . {\displaystyle \ln \left(\ F(x)^{n}\ \right)=n\ \ln F(x)\sim -{\frac {-n~}{\ x\ }}~.} Thus we have
F ( x ) n ≈ exp ( − n x ) {\displaystyle F(x)^{n}\approx \exp \left({\frac {-n~}{\ x\ }}\right)} and letting u ≡ x n − 1 {\displaystyle \ u\equiv {\frac {x}{\ n\ }}-1\ } (and skipping some explanation)
lim n → ∞ ( F ( n u + n ) n ) = exp ( − 1 1 + u ) = G 1 ( u ) {\displaystyle \lim _{n\to \infty }{\Bigl (}\ F(n\ u+n)^{n}\ {\Bigr )}=\exp \left({\tfrac {-1~}{\ 1+u\ }}\right)=G_{1}(u)\ } for any u . {\displaystyle \ u~.} The expected maximum value therefore goes up linearly with n .
Gumbel distribution Let us take the normal distribution with cumulative distribution function
F ( x ) = 1 2 erfc ( − x 2 ) . {\displaystyle F(x)={\frac {1}{2}}\operatorname {erfc} \left({\frac {-x~}{\ {\sqrt {2\ }}\ }}\right)~.} We have
ln F ( x ) → − exp ( − 1 2 x 2 ) 2 π x a s x → ∞ {\displaystyle \ln F(x)\rightarrow -{\frac {\ \exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\quad {\mathsf {~as~}}\quad x\rightarrow \infty } and thus
ln ( F ( x ) n ) = n ln F ( x ) → − n exp ( − 1 2 x 2 ) 2 π x a s x → ∞ . {\displaystyle \ln \left(\ F(x)^{n}\ \right)=n\ln F(x)\rightarrow -{\frac {\ n\exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\quad {\mathsf {~as~}}\quad x\rightarrow \infty ~.} Hence we have
F ( x ) n ≈ exp ( − n exp ( − 1 2 x 2 ) 2 π x ) . {\displaystyle F(x)^{n}\approx \exp \left(-\ {\frac {\ n\ \exp \left(-{\tfrac {1}{2}}x^{2}\right)\ }{\ {\sqrt {2\pi \ }}\ x\ }}\right)~.} If we define c n {\displaystyle \ c_{n}\ } as the value that exactly satisfies
n exp ( − 1 2 c n 2 ) 2 π c n = 1 , {\displaystyle {\frac {\ n\exp \left(-\ {\tfrac {1}{2}}c_{n}^{2}\right)\ }{\ {\sqrt {2\pi \ }}\ c_{n}\ }}=1\ ,} then around x = c n {\displaystyle \ x=c_{n}\ }
n exp ( − 1 2 x 2 ) 2 π x ≈ exp ( c n ( c n − x ) ) . {\displaystyle {\frac {\ n\ \exp \left(-\ {\tfrac {1}{2}}x^{2}\right)\ }{{\sqrt {2\pi \ }}\ x}}\approx \exp \left(\ c_{n}\ (c_{n}-x)\ \right)~.} As n {\displaystyle \ n\ } increases, this becomes a good approximation for a wider and wider range of c n ( c n − x ) {\displaystyle \ c_{n}\ (c_{n}-x)\ } so letting u ≡ c n ( c n − x ) {\displaystyle \ u\equiv c_{n}\ (c_{n}-x)\ } we find that
lim n → ∞ ( F ( u c n + c n ) n ) = exp ( − exp ( − u ) ) = G 0 ( u ) . {\displaystyle \lim _{n\to \infty }{\biggl (}\ F\left({\tfrac {u}{~c_{n}\ }}+c_{n}\right)^{n}\ {\biggr )}=\exp \!{\Bigl (}-\exp(-u){\Bigr )}=G_{0}(u)~.} Equivalently,
lim n → ∞ P ( max { X 1 , … , X n } − c n ( u c n ) ≤ u ) = exp ( − exp ( − u ) ) = G 0 ( u ) . {\displaystyle \lim _{n\to \infty }{\boldsymbol {\mathcal {P}}}\ {\Biggl (}{\frac {\ \max\{X_{1},\ \ldots ,\ X_{n}\}-c_{n}\ }{\left({\frac {u}{~c_{n}\ }}\right)}}\leq u{\Biggr )}=\exp \!{\Bigl (}-\exp(-u){\Bigr )}=G_{0}(u)~.} With this result, we see retrospectively that we need ln c n ≈ ln ln n 2 {\displaystyle \ \ln c_{n}\approx {\frac {\ \ln \ln n\ }{2}}\ } and then
c n ≈ 2 ln n , {\displaystyle c_{n}\approx {\sqrt {2\ln n\ }}\ ,} so the maximum is expected to climb toward infinity ever more slowly.
Weibull distribution We may take the simplest example, a uniform distribution between 0 and 1 , with cumulative distribution function
F ( x ) = x {\displaystyle F(x)=x\ } for any x value from 0 to 1 .For values of x → 1 {\displaystyle \ x\ \rightarrow \ 1\ } we have
ln ( F ( x ) n ) = n ln F ( x ) → n ( 1 − x ) . {\displaystyle \ln {\Bigl (}\ F(x)^{n}\ {\Bigr )}=n\ \ln F(x)\ \rightarrow \ n\ (\ 1-x\ )~.} So for x ≈ 1 {\displaystyle \ x\approx 1\ } we have
F ( x ) n ≈ exp ( n − n x ) . {\displaystyle \ F(x)^{n}\approx \exp(\ n-n\ x\ )~.} Let u ≡ 1 + n ( 1 − x ) {\displaystyle \ u\equiv 1+n\ (\ 1-x\ )\ } and get
lim n → ∞ ( F ( u n + 1 − 1 n ) ) n = exp ( − ( 1 − u ) ) = G − 1 ( u ) . {\displaystyle \lim _{n\to \infty }{\Bigl (}\ F\!\left({\tfrac {\ u\ }{n}}+1-{\tfrac {\ 1\ }{n}}\right)\ {\Bigr )}^{n}=\exp \!{\bigl (}\ -(1-u)\ {\bigr )}=G_{-1}(u)~.} Close examination of that limit shows that the expected maximum approaches 1 in inverse proportion to n .
See also References Further reading