Adjustment of window length with sliding/moving RMS method for frequency drifts

[johnnysmith911]

I'm using a sliding RMS mechanism to compute RMS of a signal, i.e. with a window size of N, I add the squared value of the new sample x to the running total while deducting the squared value of the x(n - N) sample and then performing a square root to get the RMS value.

This method works fine when a cycle's worth data in the signal under question lines up exactly with the length of the window. For ex,

Let the window length N = 512, frequency of the signal = 100 Hz & sampling frequency = 51200 Hz.

With the above settings, every set of 512 samples would have a cycle's worth data of the signal & the computation works fine.

Now, the issue begins when the frequency of the signal deviates from 100Hz giving more or less number of samples in the same 512 samples window which throws off the accuracy of the sliding RMS algorithm.

This happens because the division by 512 to get the RMS, is no longer valid due to the fact that this particular cycle of the signal is getting complete before the end of the 512 samples window. However, the running squared total would already have the contributions from the previous samples which were divided by 512 while the samples of this cycle would require a different window length.

Is there a way to compensate for the addition/lack of samples due to the frequency drifts while using this technique?

 

[barry]

Since you're saving N samples in an array, you could look for the zero-crossings and compute the RMS value between three zero crossings.

 

[KlausST]

Hi,

A sliding window of a fixed length combined with a varying input frequency causes problems.
If you are using lower output frequency than input frequency... it acts like a decimation filter.
Then it's causing similar artefacts than undersampling... it is causing alias frequencies.

Do you know the range of the input frequency?

Klaus

 

[johnnysmith911]

Since you're saving N samples in an array, you could look for the zero-crossings and compute the RMS value between three zero crossings.

Hi Barry,

The RMS needs to be computed for every sample & not every cycle, i.e. every time a new sample comes in, it is squared, divided by N & added to the running total. Then, the squared & divided by N value of (n - N)th sample is subtracted. So, I think computing RMS every 3 zero crossings wouldn't work in this case.

However, I do have the provision to accumulate samples till I get a full cycle, get the new window length for that cycle & then compute the RMS. Even so, the computation has to be done for each sample in the newly acquired cycle of data. The key problem here is the deletion of the squared & divided by N value of (n - N)th sample which no longer works as expected due to the fact that N is no longer the same as the previous cycle.

- - - Updated - - -

Hi,

A sliding window of a fixed length combined with a varying input frequency causes problems.
If you are using lower output frequency than input frequency... it acts like a decimation filter.
Then it's causing similar artefacts than undersampling... it is causing alias frequencies.

Do you know the range of the input frequency?

Klaus

Hi,

Thanks for the response.

The input frequency is well below the sampling frequency. As mentioned in the example, the sampling frequency is 51200 Hz while the input signal is 100 Hz. Let's say the max deviation is +/- 100 Hz giving an input signal whose frequency varies between 0-200 Hz & is normally 100 Hz. Even at 200 Hz, the sampling frequency is still 256 times greater than the input signal frequency. So, I guess aliasing won't be an issue here?

However, having a sliding window with a variable frequency is certainly an issue as you mentioned. The problem here is that although the sampling frequency is several times greater than the input frequency, this does not compensate for the error introduced into the sliding RMS calculation due to any frequency change.

As an example, lets say the 1st 3 cycles of the input signal has a steady frequency of 100 Hz. This means, all three 512 sample windows have a full cycle. Nothing more, nothing less. So, if we were to move the sliding RMS window one sample at a time for the three windows while adding the contribution of the new sample & dropping that of the (n - N)th one, the RMS would be a stable value.
Now, in the 4th 512 sample window, lets say the input channel frequency rises to 200 Hz. In this case, there'd be 2 cycles of the signal instead of 1. This is where the problem starts. Unlike in earlier cases wherein the "squared & divided by N" value of (n - N)th sample was being deducted, we now have samples for a different window length(256 in this case) instead of 512. So, the deduction & addition operations don't work correctly any more.

I do have the provision to accumulate samples till I get a full cycle, get the new window length for that cycle & then compute the RMS instead of computing right when a new sample comes in. Even so, the computation still has to be done for each sample in the newly acquired cycle of data & the deduction & addition poses a problem.

 

[barry]

If your signal can arbitrarily change frequency from sample to sample, your RMS calculation becomes kind of meaningless, since you say you want to compute the value over a complete cycle. If you're halfway through a 100 hz signal and it jumps to 200 hz, what do you use for your window size? In other words, is your window 1/100 or 1/200?

 

[KlausST]

Hi,

It's getting clearer now. But not easier.

The problem is the input frequency range.
0...200Hz. The dynamic is 200Hz / 0Hz ... which is infinite. Not good.

0Hz is more the problem than 200Hz.
Imagine you have to wait for a full cycle at 0Hz. That's really a long time....

If you could limit the lower frequency to 0.1Hz .... this is 10s of cycle time..and gives a frequency dynamic of 2000.
This means you need 2000 times the sliding window width than with 200Hz.

One approach could be not to store the squared values in the ring buffer, but the [squared_and_accumulated] values.
But with this method you need
* to restart the measurement from time to time
* and you need a larger bit width of the buffer contents.
Then you could decide integration_time eg integration_count

The calculation is: sqrt((actual_value - buffer_value)/ integration_count)
Where the buffer_value is the value integration_count before the actual value.

With this method you could adjust window size dynamically at any time without much processing power.

*****
0..200Hz somehow reminds me on an inverter motor application..three phase

If so, then it eases the problem drastically.

Klaus

 

[johnnysmith911]

If your signal can arbitrarily change frequency from sample to sample, your RMS calculation becomes kind of meaningless, since you say you want to compute the value over a complete cycle. If you're halfway through a 100 hz signal and it jumps to 200 hz, what do you use for your window size? In other words, is your window 1/100 or 1/200?

Umm, where & when did I mention that the frequency shifts from sample to sample?:???:
I said, the frequency can change from cycle to cycle. Say you have 1 cycle of 100Hz, the next cycle maybe 110Hz. Something like that.

- - - Updated - - -

Hi,

It's getting clearer now. But not easier.

The problem is the input frequency range.
0...200Hz. The dynamic is 200Hz / 0Hz ... which is infinite. Not good.

0Hz is more the problem than 200Hz.
Imagine you have to wait for a full cycle at 0Hz. That's really a long time....

Klaus

Agreed. I think I made our(mine tbh) lives quite a bit more complicated by going for 0 Hz. Let me fix that by limiting the lower frequency to 1 Hz instead.

If you could limit the lower frequency to 0.1Hz .... this is 10s of cycle time..and gives a frequency dynamic of 2000.
This means you need 2000 times the sliding window width than with 200Hz.

One approach could be not to store the squared values in the ring buffer, but the [squared_and_accumulated] values.
But with this method you need
* to restart the measurement from time to time
* and you need a larger bit width of the buffer contents.
Then you could decide integration_time eg integration_count

The calculation is: sqrt((actual_value - buffer_value)/ integration_count)
Where the buffer_value is the value integration_count before the actual value.

With this method you could adjust window size dynamically at any time without much processing power.

*****
0..200Hz somehow reminds me on an inverter motor application..three phase

If so, then it eases the problem drastically.

Klaus

Read this several times now but I'm just not able to get my head around it. Currently, this is my algorithm.

1. Keep acquiring samples until one full cycle is complete (using zero crossings to detect the completion).

2. Once a full cycle is obtained, get the window length.

3. Go through each sample in the window by adding the contribution of the current sample x * x / N to the running total while subtracting the contribution of the oldest sample(assume there was one full cycle before the start of this cycle) in the window x(n-N) * x(n-N) / N and obtaining a square root.

This works fine if the window N has the same length in both the present & previous cycles. As this isn't the case & the lengths differ between cycles during frequency drifts, the process of addition & subtraction introduces errors.

How does your change work with this? Or is it a complete overhaul with respect to what I'm doing now? Could you please explain in a more detailed manner? Maybe with a simple example or in an algorithm like fashion?

Many thanks in advance for all your help!

 

[KlausST]

Hi,

example:

input
-2 / 1 / 0 / 1 / 2 / 2 / 1 / 0 / -1 / -2 / -2 / -1 / 0 / 1 / 2 / 1 / 0 / -1 / -2 / -1 / 0 
squared                                                                             
4  / 1 / [COLOR="#00FF00"]0[/COLOR] / 1 / 4 / 4 / 1 / 0 / 1  / 4  / 4  /  1 / [COLOR="#0000FF"]0[/COLOR] / 1 / 4 / 1 / 0 / 1  / 4  / 1  / [COLOR="#FF0000"]0[/COLOR] 
in buffer:
4  / 5 / [COLOR="#00FF00"]5[/COLOR] / 6 /10 /14 /15 /15 / 16 / 20 / 24 / 25 /[COLOR="#0000FF"]25[/COLOR] /26 /30 /31 /31 / 32 / 35 / 36 / [COLOR="#FF0000"]36[/COLOR]

for the first full wave: green to blue:
= sqrt (25 -5)

for the second full wave blue to red
= sqrt (36 -25)


Klaus

 

[johnnysmith911]

Hi,

example:

input
-2 / 1 / 0 / 1 / 2 / 2 / 1 / 0 / -1 / -2 / -2 / -1 / 0 / 1 / 2 / 1 / 0 / -1 / -2 / -1 / 0 
squared                                                                             
4  / 1 / [COLOR="#00FF00"]0[/COLOR] / 1 / 4 / 4 / 1 / 0 / 1  / 4  / 4  /  1 / [COLOR="#0000FF"]0[/COLOR] / 1 / 4 / 1 / 0 / 1  / 4  / 1  / [COLOR="#FF0000"]0[/COLOR] 
in buffer:
4  / 5 / [COLOR="#00FF00"]5[/COLOR] / 6 /10 /14 /15 /15 / 16 / 20 / 24 / 25 /[COLOR="#0000FF"]25[/COLOR] /26 /30 /31 /31 / 32 / 35 / 36 / [COLOR="#FF0000"]36[/COLOR]

for the first full wave: green to blue:
= sqrt (25 -5)

for the second full wave blue to red
= sqrt (36 -25)


Klaus

Hi Klaus,

I've been working on implementing this method and a few doubts popped up.

1. What's the integration_count/integration_time that you've mentioned?

The calculation is: sqrt((actual_value - buffer_value)/ integration_count)
Where the buffer_value is the value integration_count before the actual value.
Klaus

I tried guessing what this might be but just couldn't come up with an answer. For some reason, the results I see are all over the place & I have a feeling that this might be one of the causes for it.
2. At some point, the data type assigned to the elements of the buffer will overflow. Is this the reason that you mentioned the need to restart the measurement?

But with this method you need
* to restart the measurement from time to time
Klaus

If so, what'd be the proper/efficient way to do so considering the additional processing load this presents and also the effect this restart would have on the accuracy?

Again, thanks for your help.

- - - Updated - - -

Hi Klaus,

I've been working on implementing this method and a few doubts popped up.

1. What's the integration_count/integration_time that you've mentioned?

I tried guessing what this might be but just couldn't come up with an answer. For some reason, the results I see are all over the place & I have a feeling that this might be one of the causes for it.
2. At some point, the data type assigned to the elements of the buffer will overflow. Is this the reason that you mentioned the need to restart the measurement?

If so, what'd be the proper/efficient way to do so considering the additional processing load this presents and also the effect this restart would have on the accuracy?

Again, thanks for your help.

Ah, just realised that you're referring to the effective window length when you said integration count(I hope?). After adding the division, I'm observing a strange thing.

There's always a massive spike during the transition from one frequency to another which vanishes after a short interval(equal to the ratio between the 2 frequencies I think) as can be seen in the above diagram. The magnitude of the spike seems to depend on how big the deviation is.

 

[KlausST]

Hi,

The spike comes because of window length mismatch. I think there is no way to avoid this...as long as you can't see into the future, how frequency changes.

Klaus

 

[johnnysmith911]

Hi,

The spike comes because of window length mismatch. I think there is no way to avoid this...as long as you can't see into the future, how frequency changes.

Klaus

Actually, you could say that in a way I am able to "see" the future frequency. The method I'm following is to keep accumulating new samples into a buffer until I get a full cycle(based on zero crossings). Once a full cycle is obtained, I process the all the samples in that cycle as a "batch". This means, I already know the frequency before working on the samples of that cycle.

Would this help in filtering out that spike?

 

[KlausST]

No,

the problem is that before you know the new_frequency .. you still process the incoming datastream with the old_frequency. .. but the frequency already changes.

Klaus

 

[johnnysmith911]

No,

the problem is that before you know the new_frequency .. you still process the incoming datastream with the old_frequency. .. but the frequency already changes.

Klaus

Hi Klaus,

I've been trying out a few things to handle the spike. I added an additional short length moving average filter on top of the RMS & this seems to get rid of the spike with minimal response delay(during a change in magnitude).

The other question I had was regarding what you mentioned about restarting the calculations time to time. Is this because at some point, the data type assigned to the elements of the buffer will overflow?

If so, what do you feel would be the right way to handle this efficiently?

John

 

[KlausST]

Hi,

Yes, the value will overflow sooner or later.

When to reset the process... I can´t say, because it depends on your application.

If your application allowes a short time of no valid data, then

If you need continous input and output stream, then it will become a bit more difficult.
Maybe interrupt driven, once every xx seconds. Just reduce all values inside the buffer by the value of the oldest value.

(oldest value) 234, 267, 298, 312, 334
-->Subtract oldest value = 234 from all buffer contents (all within an atomic process)
(oldest value) 0, 33, 64, 78, 100

Klaus