user6634447 - 1 year ago 100
C# Question

# Find anomaly in a series of numbers

I have list of numbers

``````var nums = new List<double> {1,2,2.4,2.6,1.5,3,1.9};
``````

In the about series I want to filter out 3 which is an anomaly in range, It should be dynamic series values may change so can't hard code any value in a condition check.

If the distribution of `nums` items is assumed to be normal one, we can treat a value being anomaly if it's beyond `[mean - k * sigma..mean + k * sigma]` range (`sigma` stands for the standard deviation), where `k` is typically `2` (95%), `3` (99.76%), sometimes even `5`. If it's your case, you can implement

``````public static IEnumerable<T> Anomaly<T>(IEnumerable<T> source,
Func<T, double> map,
double maxSigma = 3.0) {
if (null == source)
throw new ArgumentNullException("source");
else if (null == map)
throw new ArgumentNullException("map");

T[] data = source.ToArray();

if (data.Length <= 1)
yield break;

double s = 0.0;
double s2 = 0.0;

foreach (var item in data) {
double x = map(item);

s += x;
s2 += x * x;
}

double mean = s / data.Length;
double sigma = Math.Sqrt(s2 / data.Length - (s / data.Length) * (s / data.Length));
double leftMargin = mean - maxSigma * sigma;
double rightMargin = mean + maxSigma * sigma;

foreach (var item in data) {
double x = map(item);

if (x < leftMargin || x > rightMargin)
yield return item;
}
}
``````

And so you can find anomalies for different `k`:

``````var nums = new List<double> { 1, 2, 2.4, 2.6, 1.5, 3, 1.9 };

// k = 3 (typical criterium of 3 sigma deviation - 99.76%) -  empty output
Console.Write(string.Join(", ", Anomaly(nums, x => x, 3)));

// k = 1 (unusual criterium of 1 sigma deviation - 67%) the output is "1, 3"
Console.Write(string.Join(", ", Anomaly(nums, x => x, 1)));
``````
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download