Week 9: Fourier Series, Stepan Malkov

We now turn our attention to Fourier series, which, not unlike power series, allows us to represent periodic functions in terms of an infinite series.

Definition:

Let $V$ be a vector space over the complex numbers. Then, an inner product (sometimes called a Hermitian product) on $V$ is a map $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{R}^+$ such that

(Positive definitieness) $\langle v, v\rangle > 0$ if and only if $v \not = 0.$
$\langle a_1 v_1 + a_2 v_2, w \rangle = a_1 \langle v_1,w \rangle + a_2 \langle v_2,w \rangle$ for all $v_1,v_2,w \in V, a_1, a_2 \in \mathbb{C}.$
$\langle v,a_1 w_1 + a_2 w_2 \rangle = \overline{a_1}\langle v,w_1 \rangle + \overline{a_2} \langle v,w_2 \rangle$ for all $v_1,v_2,w \in V, a_1, a_2 \in \mathbb{C}.$

In other words, the inner product is linear in the first argument and conjugate-linear in the second argument. If $V$ admits an inner product, $V$ is called an inner product space.

Remark:

Note that an inner product on $V$ defines a norm on $V$ by $\|v\| = \langle v, v\rangle^{\frac{1}{2}}.$

Definition:

A complete inner product space is called a Hilbert space.

Proposition (Cauchy-Schwarz):

If $\langle \cdot, \cdot \rangle$ is an inner product on $V$ and $\|\cdot\|$ is a norm on $V$ induced by the inner product, then $\langle v,w \rangle \leq \|v\|\|w\|$ for all $v,w \in V.$

Proof:

The proof of Cauchy-Schwarz was part of homework 1.

Definition:

Let $B=\{e_n\}$ be a linearly independent set in a Hilbert space $V.$ Then, $\{e_n\}$ is called an orthonormal basis if every $v \in V$ can be written in the form

$v = \sum_B a_n e_n$

for some coefficients $a_n \in \mathbb{C}$ (i.e. $B$ spans $V$ ), and $\langle e_i, e_j \rangle = 1$ when $i=j$ and $0$ otherwise.

Remark:

Note that this implies that $\|e_j\| = \sqrt{\langle e_j,e_j\rangle} = 1$ for all $j,$ i.e. $e_j$ are unit vectors.

Remark:

Note that this is different from a regular basis for a vector space, as infinitely many terms are allowed in the sum. This is where the assumption of completeness is necessary, also creating an issue regarding the question of whether the sum $\sum_B a_n e_n$ is well-defined (since it can have infinitely many terms), but it turns out that if an orthonormal basis exists, then there are at most countably many terms in the sum, which lets it take the form

$\sum_{j=1}^\infty a_n e_j$

after rearranging the basis elements $e_j,$ which can be well-defined in terms of convergence of partial sums.

Example:

$\mathbb{C}^n$ with the Hermitian product $\langle x,y \rangle = x \cdot \overline{y}$ is a Hilbert space with orthonormal basis $(1,0,0,...),(0,1,0,...),...,(0,0,...,0,1).$

Proposition (Pythagorean Theorem):

If $v = \sum_n a_n e_n,$ then $\|v\|^2= \sum_n |a_n|^2.$

Proof:

Calculate $\langle v, v\rangle.$

Proposition:

If $\{e_n\}$ is an orthonormal basis for $V,$ then $v$ can be written uniquely as

$v = \sum_{B} \langle v,e_n \rangle e_n.$

Proof:

By the Pythagorean theorem, if $v=\sum_{B} a_n e_n = \sum_{B} b_n e_n,$ then

$\left\|\sum_B (a_n-b_n) e_n \right\| = \sum_B (a_n-b_n)^2 = 0,$

i.e. $a_n=b_n$ for all $n,$ so the decomposition is unique. It is left to verify that this sums equals $v.$ Indeed, if

$v= \sum_n a_n e_n,$

since the $\{e_j\}$ are orthonormal, $a_j = \langle v,e_j\rangle.$ Thus,

$v = \sum_{B} \langle v, e_n \rangle e_n,$

which yields the desired equality.

Corollary (Parseval’s Identity):

$\sum_{n} |\langle v, e_n \rangle|^2 = \|v\|^2.$

Proof:

Follows immediately from the Pythagorean theorem.

Now, consider the vector space of complex-valued periodic functions with period 1, denoted $C(\mathbb{R}/\mathbb{Z}),$ with the inner product given by

$\langle f, g \rangle = \int_0^1 f(x) \overline{g(x)} dx.$

One easily checks that this is indeed an inner product, which induces the $L^2$ norm on $C(\mathbb{R}/\mathbb{Z})$ given by

$\|f\|_{L^2} = \left(\int f(x)^2 dx\right)^{\frac{1}{2}},$

which in turn induces the $L^2$ metric. One says $f_n \to f$ in $L^2$ if $\|f_n-f\|_{L^2} \to 0,$ or equivalently,

$\left(\int_0^1 (f_n(x)-f(x))^2 dx\right)^{\frac{1}{2}} \to 0.$

Definition:

Define $e_n \in C(\mathbb{R}/\mathbb{Z})$ to be $e_n(x) = e^{2\pi i n x} = \cos (2\pi i nx) + i\sin(2 \pi nx)$ for $n \in \mathbb{Z}.$ A sum of the form

$\sum_{k=-n}^n a_n e_n,a_n \in \mathbb{C},$

is called a trigonometric polynomial. We now ask a question that comes from linear algebra: does there exist an orthonormal basis for $L^2$ ? To answer the following question, one requires the following important theorem:

Theorem (Weierstrass Approximation Theorem):

Trigonometric polynomials are dense in $C(\mathbb{R}/\mathbb{Z}),$ or equivalently, for any $\epsilon>0$ and $f \in C(\mathbb{R}/\mathbb{Z}),$ there exists a trigonometric polynomial $P$ such that

$\|f-P\|_{L^2} < \epsilon.$

Proof:

Unfortunately, we do not have time to prove this theorem, but it relies on an important concept known as an approximation to the identity. We are now able to prove our main result:

Theorem:

Then, $\{e_n\}_{n \in \mathbb{Z}}$ is an orthonormal basis for $C(\mathbb{R}/\mathbb{Z}).$

Proof:

Indeed,

$\int_0^1 e^{2\pi i nx} \overline{e^{2\pi i mx}}dx = \int_0^1 e^{2\pi i (n-m) x} dx = 1 \text{ when } n=m, 0 \text{ otherwise}.$

Finally, one needs to show that $\{e_n\}$ spans $C(\mathbb{R}/\mathbb{Z}),$ i.e. every periodic function with period 1 can be represented as a (possibly infinite) sum of sines and cosines. We claim that for any $f \in C(\mathbb{R}/\mathbb{Z}),$

$f=\sum_{n=-\infty}^\infty \langle f,e_n \rangle e_n = \sum_{n=-\infty}^\infty \left(\int_0^1 f(y) e^{-2\pi i ny} dy\right) e^{2\pi i nx}$

Since $C(\mathbb{R}/\mathbb{Z})$ has the $L^2$ norm, we need to show that the partial sums converge

$f_k = \sum_{n=-k}^k \langle f,e_n\rangle e_n \to f$

in $L^2$ norm as $k \to \infty,$ or equivalently,

$\int_0^1 (f(x)-f_k(x))^2 dx \to 0.$

By the Weierstrass Approximation Theorem, pick a trigonometric polynomial $P = \sum_n a_n e_n$ such that $\|f-P\|_{L^2}<\epsilon.$ Now, note that for any $e_m,$

$\left\langle f-\sum_{n=-k}^k \langle f, e_n \rangle, e_m\right\rangle = 0$

for sufficiently large $k,$ so by linearity of the inner product,

$\left\langle f-f_k, P-f_k\right\rangle=0.$

Then, by the Cauchy-Schwarz inequality, we get

$\|f-f_k\|_{L^2}^2=\langle f-f_k, f-f_k\rangle= \langle f-f_k,f-P\rangle + \langle f-f_k, P-f_k\rangle \leq \|f-f_k\|_{L^2}\epsilon,$

so $\|f-f_k\|_{L^2}<\epsilon$ for any $\epsilon >0,$ completing the proof.

Definition:

The coefficients of the basis elements $\langle f,e_n\rangle = \widehat{f}(n)$ are called the Fourier coefficients of $f,$ and the function $\widehat{f}:\mathbb{Z} \to \mathbb{C}$ is called the Fourier transform of $f.$

Corollary (Plancherel’s Theorem):

$\sum_n |\widehat{f}(n)|^2 = \int_0^1 f(x)^2dx \Longleftrightarrow \|\widehat{f}\|_{l^2}=\|f\|_{L^2}.$