Inner Product

Resources

Axler, Sheldon. 2020. Measure, Integration & Real Analysis. Vol. 282. Graduate Texts in Mathematics. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-33143-6.

Hilbert Space

Main Idea

For a Vector Space $V$ and a Field $F$ , the inner product is a function mapping $V \times V \to F$ , and satisfying some further properties. Letting $f, g, h \in V$ , $α \in F$ , and denoting the inner product as $⟨ \cdot, \cdot ⟩$ , the inner product must satisfy

Positivity: $⟨ f, f ⟩ \geq 0$ ,
Definiteness: $⟨ f, f ⟩ = 0 ⟺ f = 0$ (note that here $f = 0$ is a vector in $V$ , not $F$ ),
First Argument Linearity: $⟨ f + g, h ⟩ = ⟨ f, h ⟩ + ⟨ g, h ⟩$ and $⟨ α f, g ⟩ = α ⟨ f, g ⟩$ .
Conjugate Symmetry: $⟨ f, g ⟩ = \overset{―}{⟨ g, f ⟩}$

Further Properties

$⟨ 0, f ⟩ = ⟨ f, 0 ⟩ = 0$ (use linearity in first argument, then symmetry)
$⟨ f, g + h ⟩ = ⟨ f, g ⟩ + ⟨ f, h ⟩$ (use conjugate symmetry, linearity, then conjugate symmetry)
$⟨ f, α g ⟩ = \overset{―}{α} ⟨ f, g ⟩$ (same approach as above)

Thus, if $F = R$ , then the inner product is linear in both arguments!

Define the norm as

| | f | | = \sqrt{⟨ f, f ⟩} .

To show that an inner product is a Normed Vector Space with this induced norm, we would need to also show:

Absolute Homogeneity: $| | α f | | = | α | | | f | |$
Begin with $| | α f | |^{2} = ⟨ α f, α f ⟩$ . Use linearity of first argument and conjugate linearity of second argument to give $| | α f | |^{2} = α \overset{―}{α} ⟨ f, f ⟩ = | α |^{2} | | f | |^{2}$ . Square root both sides.

Triangle Inequality: $| | f + g | | \leq | | f | | + | | g | |$
Using some of the simplifications from the Pythagorean Theorem proof

\begin{aligned} | | f + g | |^{2} & = | | f | |^{2} + | | g | |^{2} + ⟨ f, g ⟩ + ⟨ g, f ⟩, \\ = | | f | |^{2} + | | g | |^{2} + ⟨ f, g ⟩ + \overset{―}{⟨ f, g ⟩}, \\ = | | f | |^{2} + | | g | |^{2} + 2 Re (⟨ f, g ⟩), \\ \leq | | f | |^{2} + | | g | |^{2} + 2 | ⟨ f, g ⟩ | . \end{aligned}

Next, applying the Cauchy-Schwartz Inequality,

| | f + g | |^{2} \leq | | f | |^{2} + | | g | |^{2} + 2 | | f | | | | g | | = (| | f | | + | | g | |)^{2} .

Finally, taking the square root, we prove the original statement. $◻$

Pythagorean Theorem

If $f$ and $g$ are orthogonal, then $| | f + g | |^{2} = | | f | |^{2} + | | g | |^{2}$ .
Proof: Use the definition of the norm and then linearity in the two arguments. Then use the definition of the norm again and apply the definition of orthogonality to remove the "cross terms".

\begin{aligned} | | f + g | |^{2} & = ⟨ f + g, f + g ⟩ \\ = ⟨ f, f + g ⟩ + ⟨ g, f + g ⟩ \\ = ⟨ f, f ⟩ + ⟨ f, g ⟩ + ⟨ g, f ⟩ + ⟨ g, g ⟩ \\ = | | f | |^{2} + ⟨ f, g ⟩ + ⟨ g, f ⟩ + | | g | |^{2}, \\ = | | f | |^{2} + | | g | |^{2} \\ ◻ . \end{aligned}

The orthogonal decomposition is quickly summarized for a simplified case. If $g \neq 0$ , then there exists some $h$ (within the Orthogonal Complement of ${g}$ ) such that

⟨ h, g ⟩ = 0 and f = \frac{⟨ f, g ⟩}{⟨ g, g ⟩} g + h .

Cauchy-Schwartz Inequality

For any vectors $f$ and $g$ of the inner product space,

| ⟨ f, g ⟩ | \leq | | f | | | | g | |,

where the equality is realized if and only if $f$ and $g$ are scalar multiples of one another.
Proof: If $g = 0$ , both sides of the inequality are $0$ , thus, safely assume $g \neq 0$ . Begin with the orthogonal decomposition of $f$ and then the Pythagorean Theorem gives

| | f | |^{2} = {| | \frac{⟨ f, g ⟩}{⟨ g, g ⟩} g | |}^{2} + | | h | |^{2} .

Which simplifies

\begin{aligned} | | f | |^{2} & = {| \frac{⟨ f, g ⟩}{⟨ g, g ⟩} |}^{2} | | g | |^{2} + | | h | |^{2} \\ = \frac{{| ⟨ f, g ⟩ |}^{2}}{(| | g | |^{2})^{2}} | | g | |^{2} + | | h | |^{2} \\ = \frac{{| ⟨ f, g ⟩ |}^{2}}{| | g | |^{2}} + | | h | |^{2} \\ \geq \frac{{| ⟨ f, g ⟩ |}^{2}}{| | g | |^{2}} . \end{aligned}

Then, rearranging gives

| ⟨ f, g ⟩ |^{2} \leq | | f | |^{2} | | g | |^{2} .

Only if $h = 0$ would the inequality reduce to the equality, and this would mean that there is no component of $f$ orthogonal to $g$ , which means that $f$ is a scalar multiple of $g,$ concluding the proof. $◻$

Parallelogram Equality

For any elements $f$ and $g$ of an inner product space,

| | f + g | |^{2} + | | f - g | |^{2} = 2 | | f | |^{2} + 2 | | g | |^{2} .

Proof: We rewrite the LHS norms as inner products, use some steps from the Pythagorean theorem proof (twice), and use linearity / conjugate linearity to factor out $- 1$ .

\begin{aligned} | | f + g | |^{2} + | | f - g | |^{2} & = ⟨ f + g, f + g ⟩ + ⟨ f - g, f - g ⟩ \\ = | | f | |^{2} + | | g | |^{2} + ⟨ f, g ⟩ + ⟨ g, f ⟩ + | | f | |^{2} + | | - g | |^{2} + ⟨ f, - g ⟩ + ⟨ - g, f ⟩ \\ = | | f | |^{2} + | | g | |^{2} + ⟨ f, g ⟩ + ⟨ g f ⟩ + | | f | |^{2} + | | g | |^{2} - ⟨ f, g ⟩ - ⟨ g f ⟩ \\ = 2 | | f | |^{2} + 2 | | g | |^{2} \\ ◻ . \end{aligned}