N dimensional complex space

In this section we review important features of N dimensional complex vector space \(\CC^N\).

Definition

Let \(\CC\) denote the field of complex numbers. For any positive integer \(N\), the set of all \(N\)-tuples of complex numbers forms an \(N\)-dimensional vector space over \(\CC\) which is denoted as \(\CC^N\) and sometimes called complex vector space.

An element \(x\) in \(\CC^N\) is written as

\[x = (x_1, x_2, \ldots, x_N),\]

where each \(x_i\) is a complex number.

Vector space operations on \(\CC^N\) are defined by:

\[\begin{split}&x + y = (x_1 + y_1, x_2 + y_2, \dots, x_N + y_N), \quad \forall x, y \in \CC^N.\\ & \alpha x = (\alpha x_1, \alpha x_2, \dots, \alpha x_N) \quad \forall x \in \CC^N, \alpha \in \CC .\end{split}\]

\(\CC^N\) comes with the standard ordered basis \(B = \{e_1, e_2, \dots, e_N\}\):

(1)\[\begin{split}\begin{aligned} & e_1 = (1,0,\dots, 0),\\ & e_2 = (0,1,\dots, 0),\\ &\vdots\\ & e_N = (0,0,\dots, 1) \end{aligned}\end{split}\]

We note that the basis is same as the basis for \(N\) dimensional real vector space (the Euclidean space).

An arbitrary vector \(x\in\CC^N\) can be written as

\[x = \sum_{i=1}^{N}x_i e_i\]

Inner product

Standard inner product is defined as:

\[\langle x, y \rangle = \sum_{i=1}^{N} x_i \overline{y_i} = x_1 \overline{y_1} + x_2 \overline{y_2} + \dots + x_N \overline{y_N} \quad \forall x, y \in \CC^N.\]

where \(\overline{y_i}\) denotes the complex conjugate.

This makes \(\CC^N\) an inner product space.

This satisfies the inner product rule:

\[\langle x, y \rangle = \overline{\langle y, x \rangle}\]

Norm

The length of the vector (a.k.a. \(\ell_2\) norm) is defined as:

\[\| x \| = \sqrt{\langle x, x \rangle} = \sqrt{\sum_{i=1}^{N} x_i \overline{x_i} } = \sqrt{\sum_{i=1}^{N} |x_i|^2 } \quad \forall x \in \CC^N.\]

This makes \(\CC^N\) a normed linear space.

Distance

Distance between two vectors is defined as:

\[d(x,y) = \| x - y \| = \sqrt{\sum_{i=1}^{N} |x_i - y_i|^2}\]

This makes \(\CC^N\) a metric space.

\(\ell_p\) norms

In addition to standard Euclidean norm, we define a family of norms indexed by \(p \in [1, \infty]\) known as \(\ell_p\) norms over \(\CC^N\).

Definition

\(\ell_p\) norm is defined as:

(2)\[\begin{split} \| x \|_p = \begin{cases} \left ( \sum_{i=1}^{N} | x |_i^p \right ) ^ {\frac{1}{p}} & p \in [1, \infty)\\ \underset{1 \leq i \leq N}{\max} |x_i| & p = \infty \end{cases}\end{split}\]

So we have:

\[\| x \| = \| x \|_2\]

\(\ell_1\) norm

From above definition we have

\[\|x\|_1 = \sum_{i=1}^N |x_i|= |x_1| + |x_2| + \dots + | x_N|\]

We use norms as a measure of strength of a signal or size of an error. Different norms signify different aspects of the signal.

Quasi-norms

In some cases it is useful to extend the notion of \(\ell_p\) norms to the case where \(0 < p < 1\).

In such cases norm as defined in (2) doesn’t satisfy triangle inequality, hence it is not a proper norm function. We call such functions as quasi-norms.

\(\ell_0\) “norm”

Of specific mention is \(\ell_0\) “norm”. It isn’t even a quasi-norm. Note the use of quotes around the word norm to distinguish \(\ell_0\) “norm” from usual norms.

Definition

\(\ell_0\) “norm” is defined as:

(3)\[ \| x \|_0 = | \supp(x) |\]

where \(\supp(x) = \{ i : x_i \neq 0\}\) denotes the support of \(x\).

Note that \(\| x \|_0\) defined above doesn’t follow the definition in (2).

Yet we can show that:

\[\lim_{p\to 0} \| x \|_p^p = | \supp(x) |\]

which justifies the notation.