Creating Partially Correlated Random Variables: Part I

Note: This blog is split into two parts because edaboard limits the number of pictures and equations per blog. The second part can be found here: [link].

Introduction

It is often useful to be able to generate (real or complex-valued) signals with specific cross-correlations. In the simplest case, this means we want to produce two signals with a specified cross-correlation (and specified powers). Or, in the general case, it means we want to specify an (N x N) covariance matrix and generate N signals which will produce this covariance matrix.

As a particularly useful example of a type of "signal", we will consider Gaussian (a.k.a. Normally distributed) random variables. However, the ideas presented here may be applied for different types of signals.

In this blog, I will explain in detail how to generate partially correlated signals using common mathematical operations. I will show that discrete correlated signals are extremely easy to generate in software, using the Cholesky Decomposition (for which free source code is provided in 37 programming languages at this Rosetta Code page). I will show that in Matlab (or GNU Octave) we basically only need these two lines of code:




where Rxx is the (N x N) desired covariance matrix, N is the number of signals and Nsamps is the number of discrete samples to generate.

I will provide full Matlab source codes for the ideas in this blog. I will also describe the ideas in detail, so you can implement them in any way you want.

If you're not interested in the introductory discussion and derivations for the simple 2-signal case, you can skip straight to Part II of this blog, which contains the main results.

The Simple 2-Signal Case

It is easiest to start by discussing the simple 2-signal case, before generalising to N signals. Let us denote the signals we are designing as x₁(t) and x₂(t) and assume them to be complex-valued (since it is trivial to then convert the result for real-valued signals if required).To keep the discussion simple, we will assume we are interested in zero-mean signals:


where curly E{...} denotes the expectation operator. So, our task is to design x₁(t) and x₂(t) to have the following second order statistics:


where * denotes the complex conjugate. NOTE: the above expression for ρ is simplified because we are designing zero-mean signals. If the means are non-zero, they must be subtracted as shown here: [link].

Now, although it is not always easy to obtain partially correlated signals, there are many ways of producing uncorrelated signals (i.e. where the cross-correlation is zero). Therefore, let's start by considering two zero-mean, unit-power, Normally distributed complex random variables, s₁(t) and s₂(t), which are uncorrelated such that:


Or, in Matlab (GNU Octave) code, we simply write:



A useful property of Normally distributed random variables is that when we add them together, the result is also Normally distributed. Or, as more correctly described in this article [link]:

the sum of two independent normally distributed random variables is normal, with its mean being the sum of the two means, and its variance being the sum of the two variances

Click to expand...

Therefore, let's choose x₁(t) to be:

since this satisfies Equation (3), because:


Then, we just have to design an x₂(t) which will satisfy Equations (4) and (5). Recalling (from the quote above) that linear combinations of Normally distributed random variables are also Normally distributed, let us consider constructing x₂(t) as follows:


Then, we need to find (complex) values for A₁ and A₂ which satisfy our remaining requirements (Equations 4 and 5):


An initial observation is that these conditions are independent of the complex phase of A₂. Therefore, we can assume A₂ to be purely real (so that A₂=|A₂|). So, solving these simultaneous equations (Equations 12 and 13) for real-valued A₂ therefore yields:


Finally, we rearrange Equation 13 to get:


Putting these results together, it follows that we can construct our two partially correlated signals, x₁(t) and x₂(t), as:


This gives us a neat result for 2 signals, but to generalise to N signals, we will need a more general approach. This will be described in Part II of this blog.

To round off this part, here is a simple Matlab (GNU Octave) example:

Matlab Example



The above code will produce an output similar to the following:

Note that we never expect the sample covariance matrix to exactly match the theoretical covariance matrix. Sample statistics are themselves random variables, having their own probability distributions.