Reading for Today's Lecture: Chapter 2
Goals of Today's Lecture:
Today's notes
Last time defined expectation and stated Monotone Convergence Theorem, Dominated Convergence Theorem and Fatou's Lemma. Reviewed elementary definitions of expected value and basic properties of E.
Def'n: The
moment (about the origin) of a real
random variable X is
(provided it exists).
We generally use
for E(X). The
central moment is
Def'n: For an Rp valued random vector X we define
to be the vector whose
entry is E(Xi)(provided all entries exist).
Def'n: The (
)
variance covariance matrix of X is
Moments and probabilities of rare events are closely connected as will
be seen in a number of important probability theorems. Here is one
version of Markov's inequality (one case is Chebyshev's inequality):
The intuition is that if moments are small then large deviations from
average are unlikely.
Example moments: If Z is standard normal then
and (integrating by parts)
so that
If now
,
that is,
,
then
and
Proof: Suppose each Xi is simple:
where the xij are the possible values of Xi. Then
For general Xi>0 we create a sequence of simple approximations
by rounding Xi down to the nearest multiple of 2-n (to
a maximum of n) and applying the case just done and the monotone
convergence theorem. The general case uses the fact that
we can write each Xi as the difference of its positive and negative
parts:
Def'n: The moment generating function of a real valued X is
Def'n: The moment generating function of
is
The mgf has the following formal connection to moments:
It is thus sometimes possible to find the power series expansion of
MX and read off the moments of X from the coefficients of the
powers tk/k!.
The proof, and many other facts about mgfs, rely on techniques of complex variables.
If
are independent and
then
the moment generating function of Y is the product of those
of the individual Xi:
However this formula makes the power series expansion of MY not
a particularly nice function of the expansions of the individual MXi.
In fact this is related to the following observation. The first 3 moments
(meaning ,
and
)
of Y are just the sums of those
of the Xi but this doesn't work for the fourth or higher moment.
but
It is possible, however, to replace the moments by other objects called
cumulants which do add up properly. The way to define them relies
on the observation that the log of the mgf of Y is the sum of the logs
of the mgfs of the Xi. We define the cumulant generating function of a
variable X by
To see the relation between cumulants and moments proceed as follows:
the cumulant generating function is
To compute the power series expansion we think of the quantity in
as x and expand
Check the book by Kendall and Stuart (or the new version called Kendall's Theory of Statistics by Stuart and Ord) for formulas for larger orders r.
Example: If
are independent and Xi has a
distribution
then
This makes the cumulant generating function
Example: I am having you derive the moment and cumulant generating function
and all the moments of a Gamma rv. Suppose that
are independent
N(0,1) rvs. Then we have defined
to have a
distribution. It is easy to check S1=Z12 has density
Example: The Cauchy density is
This observation has led to the development of a substitute for the mgf which is defined for every distribution, namely, the characteristic function.
Definition: The characteristic function of a real rv X is
Aside on complex arithmetic.
The complex numbers are the things you get if you add
to the
real numbers and require that all the usual rules of algebra work. In particular
if i and any real numbers a and b are to be complex numbers then so must
be a+bi. If we multiply a complex number a+bi with a and b real by
another such number, say c+di then the usual rules of arithmetic (associative,
commutative and distributive laws) require
so this is precisely how we define multiplication.
Addition is simply
(again by following the usual rules)
Now look at transcendental functions. For real x we know
so
our insistence on the usual rules working means
Now every point in the plane can be written in polar co-ordinates as
and comparing this with our formula for the
exponential we see we can write
We will need from time to time a couple of other definitions:
Definition: The modulus of the complex number x+iy is
Definition: The complex conjugate of x+iy is
.
Notes on calculus with complex variables. Essentially the usual rules apply so,
for example,
End of Aside
Since