# Modes of Convergence Continuous Mapping Theorems Delta Method

Electronics - Devices

Oct 8, 2013 (4 years and 9 months ago)

131 views

ModesofConvergence
Deﬁnition1(convergencealmostsurely).Thematrix-valuedsequence
ofrandomvariablesZn
issaidtoconvergetoarandommatrixZalmost
surely(orwithprobabilityone),writtenasZn
as
→Z,if
Pr
￿
lim
n→∞Zn
=Z
￿
=1,
i.e.almosteverytrajectoryconvergestoZ.
Deﬁnition2(convergenceinprobability).Thematrix-valuedsequence
ofrandomvariablesZn
issaidtoconvergetoarandommatrixZinproba-
bility,writtenasZn
p
→ZorplimZn
=Z,if
∀ε>0lim
n→∞
Pr{Zn
−Z>ε}=0,
i.e.theprobabilityoflargedeviationsconvergesto0.
Result.Zn
as
→Z⇒Zn
p
→Z.
Deﬁnition3(convergenceinmeansquare).Thematrix-valuedse-
quenceofrandomvariablesZn
issaidtoconvergetoarandommatrixZin
meansquare,writtenasZn
ms
→Z,if
lim
n→∞E
￿
Zn
−Z2
￿
=0,
i.e.themeansquarederrorconvergesto0.
Result.Zn
ms
→Z⇒Zn
p
→Z.
Deﬁnition4(convergenceindistribution).Thevector-valuedsequence
ofrandomvariablesZn
issaidtoconvergetoarandomvectorZindistri-
bution,writtenasZn
d
→ZorZn
d
→D
Z
,whereDZ
isthedistributionofZ,
if
lim
n→∞
Pr{Zn
≤z}=Pr{Z≤z}
forallcontinuitypointszofPr{Z≤z}.
Result.Zn
p
→Z⇒Zn
d
→Z.IfZisconstant,Zn
p
→Z⇔Zn
d
→Z.
ContinuousMappingTheorems
Theorem(Mann–Wald).Supposethatg(z)isacontinuousRk1
×k2

R1×2
function.
•IfZn
as
→Zasn→∞,theng(Zn)
as
→g(Z).
•IfZn
p
→Zasn→∞,theng(Zn)
p
→g(Z).
•IfZn
ms
→Zasn→∞andgislinear,theng(Zn)
ms
→g(Z).
•IfZn
d
→Zasn→∞,theng(Zn)
d
→g(Z).
Theorem(Slutsky).IfUn
p
→U=constandVn
d
→Vasn→∞,then
•Un
+Vn
d
→U+V.
•UnVn
d
→UV,V
n
Un
d
→VU.
•U
−1
n
Vn
d
→U
−1V,V
n
U
−1
n
d
→VU
−1
ifPr{det(Un)=0}=0.
DeltaMethod
Theorem(DeltaMethod).Letthesequenceofk×1randomvectorsZn
satisfy

n(Zn
−Z)
d
→N(0,Σ),
asn→∞,whereZ=plimZn
isconstant,andtheRk
→R
functiong(z)
becontinuouslydiﬀerentiableatZ.Then

n(g(Zn)−g(Z))
d
→N(0,GΣG),
whereG=
∂g(z)
∂z
￿
￿
￿
￿z=Z
.
LawsofLargeNumbers
TheoremA(Kolmogorov,independentidenticalobservations).Let
{Zi
}∞
i=1
beindependentandidenticallydistributed(IID),andletE[|Zi|]exist.
Then
1
n
n
￿
i=1
Zi
as
→E[Zi
]
asn→∞.
TheoremB(Kolmogorov,independentheterogeneousobservations).
Let{Zi
}∞
i=1
beindependentwithﬁnitevarianceσ2
i
.If

￿
i=1
σ
2
i
i2
<∞,
then
1
n
n
￿
i=1
Zi
−E
￿
1
n
n
￿
i=1
Zi
￿
as
→0
asn→∞.
TheoremC(Chebyshev,uncorrelatedobservations).Let{Zi
}∞
i=1
be
uncorrelated,i.e.C[Zi
,Z
j
]=0ifi
=j.If
1
n2
n
￿
i=1
σ
2
i

n→∞
0
then
1
n
n
￿
i=1
Zi
−E
￿
1
n
n
￿
i=1
Zi
￿
p
→0
asn→∞.
TheoremD(Birkhoﬀ-Khinchin,dependentobservations,”Ergodic
Theorem”).Let{Zt
}∞
t=1
beastationaryandergodicsequenceofrandom
variables,andletE[|Zt
|]<∞.Then
1
T
T
￿
t=1
Zt
as
→E[Zt]
asT→∞.
CentralLimitTheorems
TheoremE(Lindberg-Levy,independentidenticalobservations).
Let{Zi
}∞
i=1
beindependentandidenticallydistributed(IID)withE[Zi
]=µ
andV[Zi]=σ
2.Then

n
￿
1
n
n
￿
i=1
Zi
−µ
￿
d
→N(0,σ
2)
asn→∞.
TheoremF(Lyapunov,independentheterogeneousobservations).
Let{Zi
}∞
i=1
beindependentwithE[Zi
]=µi,V[Zi]=σ2
i
andE[|Zi−µi|3
]=ν
i.
If
(
￿n
i=1
ν
i
)
1
3
(
￿n
i=1
σ2
i
)
1
2

n→∞
0,
then
￿n
i=1
(Zi
−µi
)
(
￿n
i=1
σ
2
i
)
1
2
d
→N(0,1)
asn→∞.
TheoremG(Billingsley,martingalediﬀerencesequences).Let{Zt}+∞
t=−∞
beastationaryandergodicmartingalediﬀerencesequencerelativetoitsown
past,withσ
2
=E[Z
2
t
]<∞.Then
1

T
T
￿
t=1
Zt
d
→N(0,σ
2
)
asT→∞.
TheoremH(generaldependentobservations).Let{Zt
}+∞
t=−∞
bea
stationaryandergodicsequenceofrandomvariableswith
vz
=
+∞
￿
j=−∞
C[Zt
,Z
t−j
]<∞.
Thenundersuitableconditions,

T
￿
1
T
T
￿
t=1
Zt
−E[Zt]
￿
d
→N(0,v
z
)
asT→∞.