7VoltCrayon 7VoltCrayon - 4 months ago 30
C# Question

Filtering continuous data, how to get rid of transients?

I'm writing a C# application which gets accelerometer data continuously at a rate of ~100Hz from a Wii remote. This data is stored in a list (one list for each axis) as it arrives.
I have a Timer, which fires every one second,(so by the time it fires the list contains~100 elements), it then applies a low pass filter to the list (using the Signal Processing Math.NET Neodym library), and writes the data to a file, and clears all the list for the next batch of data.

Now the problem is, the outputted filter data has large swings at the start, this happens EACH time the filter is applied, so every one second later, I have some false values in the data, which render it completely useless.

How do I go about fixing this so that the filter is STILL applied only every one second, but the transients can be avoided. I have a suspicion this can be done using overlapping windows, but am not quite sure how?

Here is the code for my filter, this code executes each second:

listXLow = MathNet.SignalProcessing.Filter.OnlineFilter.CreateLowpass(MathNet.SignalProcessing.Filter.ImpulseResponse.Finite, 100, 1, 30).ProcessSamples(listX.ToArray()).ToList();

In the arguments, right after the ImpulseResponse.Finite, 100 is the sampling rate, 1 is the cutoff frequency, and 30 is the filter order.

Here is screenshot showing what my I/O looks like. the input data is the first graph, and the second one shows the filter output:
Graphs showing input and output data


Ok, problem solved, leaving this here for anyone else facing similar trouble, as hotpaw2 mentioned, this could have been done by overlapping windows, but I found that I was simply not using the Filter in the Math.NET Neodym library the right way.

The problem was that a new filter was being created each time the timer was being fired, I resolved this by creating a global filter object, and calling it's ProcessSamples method each time within the Timer tick event.

So now , a filter object is created at the start of the program:

  MathNet.SignalProcessing.Filter.OnlineFilter XlowFilter = MathNet.SignalProcessing.Filter.FIR.OnlineFirFilter.CreateLowpass(MathNet.SignalProcessing.Filter.ImpulseResponse.Finite, 100, 1.5, 30);

and only a method of the filter is called in the Timer Tick event:

 listXLow = XlowFilter.ProcessSamples(listX.ToArray()).ToList();