Orthogonal Sinusoids

On the page 'Orthogonal, Orthonormal', I tried to illustrate the importance of perpendicular vectors in a transform matrix. Without that, a satisfying reversal of the transform would not be possible. On this page we will check which sinusoids are perpendicular and which are not.

Below, the product function sin(2*pi*x)*cos(2*pi*x) is illustrated. The inner product could be computed; it is the sum of the products over the interval of one period. From the figure it can be visually verified, that the sum will be zero, because the negative parts will exactly cancel the positive parts. Vectors and functions with inner product zero are orthogonal or perpendicular. Thus we see that sin(2*pi*x) and cos(2*pi*x) are orthogonal.

From here on I will have to use different scales on the x and y axis, otherwise the curves would be squeezed to the point of being indistinguishable. Below is exactly the same representation once again but with the x axis stretched.

Now I have space to double the periodicity of sine and cosine. As can be seen from the product curve, this does not affect the orthogonality. Sine and cosine of equal periodicity can be used as a pair of correlation vectors, which together will cover all phases of real signals with that periodicity.

But it turns out that the pair cos(2*pi*x)+sin(2*pi*x) is also orthogonal to the pair cos(2*pi*x)-sin(2*pi*x). These pairs are centered around zero.

Strictly, if you want a complete orthogonal matrix with sine/cosine pairs, or complex vectors as they say, you have to include the vectors with negative sine as well. These are called the complex conjugates. You then have pairs of pairs, which are called complex conjugate pairs.

At first sight, this may seem to complicate matters. But that is not quite the case. With the complex conjugates included, matrices become symmetrical and invertible. Formulas become simpler. Moreover, such complete complex matrices open up the way to matrix factorization and very effective algorithms, like the Fast Fourier Transform. So let us give the conjugates a place in our heart.

Till here I have illustrated integrals of sinusoids with equal periodicity. Now it is time to check the inner products of sinusoids with different periods. cos(2*pi*x) is one period, and cos(4*pi*x) is two periods.

The product cos(2*pi*x)*cos(4*pi*x) over the interval [0,1] sums to zero. This means that these functions are orthogonal.

Below is another one, with cos(4*pi*x)*cos(6*pi*x). Notice that the integrated area is consistently defined over the interval [0,1]. That is the fundamental period, 2*pi*x, describing one full circle on the complex plane.

4*pi*x and 6*pi*x are integer multiples, or harmonics, of this fundamental period. They describe two and three full circles respectively. And seem to be orthogonal to each other as well.

One more example with harmonics, three and four cycles respectively:

Oops. At first sight the product of cos(6*pi*x)*cos(10*pi*x) looks like being evenly spread around the x-axis, like the preceding figures. But there are no symmetric parts identifiable. From this curve, orthogonality cannot be verified by the eye.

I made another plot where the corresponding sine parts are included. Because sine and cosine of equal periodicity are orthogonal, I can safely do that. Look, now there is symmetry round the x-axis again:

The product of harmonic sines and cosines has it's own periodicity. In the above figure we see that two periods of the product fit in the interval [0,1]. In earlier figures it was one period. It can also be more than two periods. The pattern is, that one or more periods of the product of two harmonics will always fit nicely in the interval [0,1]. Therefore, in the harmonic case the integral over [0,1] is well defined.

Let us now check what will happen to sinuoids with periods that can not be expressed as a ratio of integers. Say, cos(2*pi*x) and cos(6*x). I have plotted these two functions and their product. Here, the product curve does not show periodicity over the interval [0,1]:

When the curves are inspected on a wider interval, they seem to show their period. But that is illusory. The period of a sum of two cosines can only be expressed as the product of two integers. Compare with the beats of polyrithmic music. And try to imagine how big integer numbers you would need to express the ratio of 2*pi and 6. It will never be exact, but for a better approximation you will need ever bigger numbers. That is how the fundamental period of this sum of cosines extends into infinity. And you would never stop summing products, although the sum would be fluctuating around the zero line, rather than growing spectacularly.... That is what they call an infinite sum. The inner product of such non-harmonic sinusoids is by itself undefined.

It seems we have the following cases for the inner products of two sines or two cosines:

- they have inner product 0.5 if their period is equal
- they have inner product 0.0 if their period is a ratio of (non-equal) integers
- they have an indefinite inner product if their combined period cannot be expressed as a ratio of integers

These conclusions may not sound very promising. On the other hand, they lead to a straightforward hint: fill your correlation matrix with harmonic sines and cosines. They are orthogonal. That is one condition fulfilled, for a start. Oh, and do not forget about the complex conjugates.

With these simple statements made, the step to classic Fourier Transform is not a big step. But the story of orthogonal sinusoids is not yet complete. I want to outline an orthogonal sinusoids set that is a bit more obscure, because they can not be expressed as complex numbers or complex exponentials. It is cosines only.

Below, cosine functions with period 0.5 and period 1 are plotted, and the product of these:

These functions are not harmonics, but 'half-harmonics'. Still they are clearly orthogonal over the interval [0,1]. Can we also go down to a 0.25 and 0.5 period cosine pair?

No, a quarter period does not work. The product is still periodic but not within the interval [0,1]. It would go beyond the analysis window. So half a period is the smallest period and thus the smallest interval. Let us check more 'half-harmonics', does the orthogonality hold for other cases? For example, cos(pi*x)*cos(3*pi*x), which are 0.5 and 1.5 period respectively:

Seems to work but can not be verified by the eye strictly. And cos(2*pi*x)*cos(3*pi*x), one period and one-and-a-half period respectively:

Still works okay. Let's get bold and try 3 and 3.5 periods:

Hmmm. Works like a charm. Actually, I am doing such half-period-product plots for the first time now. It does not appear so obscure at all. There is something uncomfortable in it though. Dsp without complex vectors, feels like sailing without a compass.

Cosines with half-period intervals are used in Discrete Cosine Transform. Not surprisingly, there is also Discrete Sine Transform. Interesting stuff. But I will stop plotting now, because this page is growing too long.