user6634447 - 11 months ago 68

C# Question

I have list of numbers

`var nums = new List<double> {1,2,2.4,2.6,1.5,3,1.9};`

In the about series I want to filter out 3 which is an anomaly in range, It should be dynamic series values may change so can't hard code any value in a condition check.

Answer Source

If the distribution of `nums`

items is assumed to be *normal* one, we can treat a value being anomaly if it's beyond `[mean - k * sigma..mean + k * sigma]`

range (`sigma`

stands for the standard deviation), where `k`

is typically `2`

(95%), `3`

(99.76%), sometimes even `5`

.
If it's your case, you can implement

```
public static IEnumerable<T> Anomaly<T>(IEnumerable<T> source,
Func<T, double> map,
double maxSigma = 3.0) {
if (null == source)
throw new ArgumentNullException("source");
else if (null == map)
throw new ArgumentNullException("map");
T[] data = source.ToArray();
if (data.Length <= 1)
yield break;
double s = 0.0;
double s2 = 0.0;
foreach (var item in data) {
double x = map(item);
s += x;
s2 += x * x;
}
double mean = s / data.Length;
double sigma = Math.Sqrt(s2 / data.Length - (s / data.Length) * (s / data.Length));
double leftMargin = mean - maxSigma * sigma;
double rightMargin = mean + maxSigma * sigma;
foreach (var item in data) {
double x = map(item);
if (x < leftMargin || x > rightMargin)
yield return item;
}
}
```

And so you can find anomalies for different `k`

:

```
var nums = new List<double> { 1, 2, 2.4, 2.6, 1.5, 3, 1.9 };
// k = 3 (typical criterium of 3 sigma deviation - 99.76%) - empty output
Console.Write(string.Join(", ", Anomaly(nums, x => x, 3)));
// k = 1 (unusual criterium of 1 sigma deviation - 67%) the output is "1, 3"
Console.Write(string.Join(", ", Anomaly(nums, x => x, 1)));
```