David Wolever David Wolever - 7 months ago 44
Python Question

Why is numpy.polyfit is off by a large margin?

I'm trying to to use

to fit a fairly simple dataset, but it's off by a fairly large margin:

badly fit data

And the code:

import numpy as np
import matplotlib as plt

fit = np.polyfit(xvals, yvals, 1)
f = np.poly1d(fit)
plt.scatter(xvals, yvals, color="blue", label="input")
plt.scatter(xvals, f(yvals), color="red", label="fit")
plt.legend()


What am I doing wrong? How can I improve the fit?

The original data:

xvals = array([ 0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 14,
15, 16, 17, 18, 20, 21, 22, 23, 24, 25, 27, 28, 29,
30, 31, 32, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44,
45, 47, 48, 49, 50, 51, 52, 54, 55, 56, 57, 58, 60,
61, 62, 63, 64, 65, 67, 68, 69, 70, 71, 72, 74, 75,
76, 77, 78, 80, 81, 82, 83, 84, 85, 87, 88, 89, 90,
91, 92, 94, 95, 96, 97, 98, 100])
yvals = array([ 0, 3, 5, 8, 10, 12, 15, 17, 19, 21, 23, 25, 27,
28, 30, 32, 33, 35, 36, 37, 39, 40, 41, 43, 44, 45,
46, 47, 48, 49, 50, 51, 52, 53, 54, 54, 55, 56, 57,
58, 58, 59, 60, 61, 61, 62, 63, 63, 64, 65, 66, 66,
67, 67, 68, 69, 70, 70, 71, 72, 73, 73, 74, 75, 76,
77, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 89,
90, 91, 92, 94, 95, 97, 98, 100])

Answer

You need f(xvals) not f(yvals). But of course you can do much better for this data we a higher order polynomial. E.g.,

import numpy as np
import matplotlib.pyplot as plt

xvals = np.array([  0,   1,   2,   3,   4,   5,   7,   8,   9,  10,  11,  12,  14,
                15,  16,  17,  18,  20,  21,  22,  23,  24,  25,  27,  28,  29,
                30,  31,  32,  34,  35,  36,  37,  38,  40,  41,  42,  43,  44,
                45,  47,  48,  49,  50,  51,  52,  54,  55,  56,  57,  58,  60,
                61,  62,  63,  64,  65,  67,  68,  69,  70,  71,  72,  74,  75,
                76,  77,  78,  80,  81,  82,  83,  84,  85,  87,  88,  89,  90,
                91,  92,  94,  95,  96,  97,  98, 100])
yvals = np.array([  0,   3,   5,   8,  10,  12,  15,  17,  19,  21,  23,  25,  27,
                28,  30,  32,  33,  35,  36,  37,  39,  40,  41,  43,  44,  45,
                46,  47,  48,  49,  50,  51,  52,  53,  54,  54,  55,  56,  57,
                58,  58,  59,  60,  61,  61,  62,  63,  63,  64,  65,  66,  66,
                67,  67,  68,  69,  70,  70,  71,  72,  73,  73,  74,  75,  76,
                77,  77,  78,  79,  80,  81,  82,  83,  84,  85,  86,  87,  89,
                90,  91,  92,  94,  95,  97,  98, 100])

fit = np.polyfit(xvals, yvals, 3)
f = np.poly1d(fit)
#print f
fig, ax = plt.subplots(1,1,figsize=(6,4),dpi=400)
ax.scatter(xvals, yvals, color="blue", label="input")
ax.scatter(xvals, f(xvals), color="red", label="fit")
ax.legend()
plt.show()

plot