Lin K - 1 year ago 89
Python Question

# Python linear least squares function not working

Ok, so I'm writing a function for linear least squares in python and it's pretty much just one equation. Yet for some reason, I'm getting a ValueError. My best guess is it has something to do with the

`.reshape`
function, since in this question I had a very similar problem and reshaping was the solution. I've read up on it and from what I gather, w in my function is in format (n,) and the result would be in (n,1) as in my previously mentioned question. I tried reshaping
`x_train`
and
`y_train`
but I only got an error that I can't change the size of the array. I guess my parameters were set wrong. Right now I'm lost, and I have many more functions like these to go through -- I wish I could understand what am I missing in my code. The equation seems to be in order, so I suppose there's something I should be adding everytime - possibly the
`reshape`
function cause I'm still using the same models as in the last situation. I hope it's the right place to post this question, I don't know what else to do but I really want to understand so I won't have these problems in the future, thank you.

Code (np. stands for numpy):

``````def least_squares(x_train, y_train, M):
'''
:param x_train: training input vector  Nx1
:param y_train: training output vector Nx1
:param M: polynomial degree
:return: tuple (w,err), where w are model parameters and err mean squared error of fitted polynomial
'''
w = np.linalg.inv(design_matrix(x_train, M). * design_matrix(x_train, M)) * design_matrix(x_train, M).T * y_train
err = mean_squared_error(x_train, y_train, w)
return (w, err)
``````

`design_matrix`
and
`mean_squared_error`
are working just fine.
Traceback:

``````ERROR: test_least_squares_err (test.TestLeastSquares)
----------------------------------------------------------------------
Traceback (most recent call last):
File "\content.py", line 48, in least_squares
w = np.linalg.inv(design_matrix(x_train, M).T * design_matrix(x_train, M)) * design_matrix(x_train, M).T * y_train
ValueError: operands could not be broadcast together with shapes (7,20) (20,7)
``````

Assuming that `design_matrix` returns a matrix, this code

``````design_matrix(x_train, M).T * design_matrix(x_train, M)
``````

most likely does not do what is intended since `*` is performing element-wise multiplication (Hadamard product of two matrices). Because your matrices are not square, it thus complains about incompatible shape.

To obtain matrix-matrix product, one might do (assuming numpy was imported as `import numpy as np`):

``````np.dot(design_matrix(x_train, M).T, design_matrix(x_train, M))
``````

Similar reasoning then applies to the rest of the statement `* design_matrix(x_train, M).T * y_train`...

Also, you might want to evaluate `design_matrix` only once, e.g., to put something like

``````mat = design_matrix(x_train, M)
``````

before the line calculating `w` and then use merely `mat`.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download