z.Lam z.Lam - 1 month ago 17
Python Question

adding broadcastble matrix in theano

I have two matrices which have shape (1,3) and (3,1)
And i want to add them and output a matrix (3,3)
In numpy, it works like this:

import numpy as np
a = np.array([0,1,2])
b = a.reshape(3,1)
a+b


And it outputs:

array([0,1,2],
[1,2,3],
[2,3,4]]


Now i would like use theano to do the same things so as to speed up the code.
My code is shown as follow:

label_vec1 = T.imatrix('label_vector')
label_vec2 = T.imatrix('label_vector')
alpha_matrix = T.add(label_vec1, label_vec2)
alpha_matrix_compute = theano.function([label_vec1,label_vec2],alpha_matrix)

a = numpy.array([[0,1,2]])
b = numpy.array([[0],[1],[2]])#
a1=theano.shared(numpy.asarray(a), broadcastable =(True,False))
b1 = theano.shared(numpy.asarray(b),broadcastable=(False, True))
c = alpha_matrix_compute(a1,b1)


But it outputs

TypeError: ('Bad input argument to theano function at index 0(0-based)', 'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?')


I am confused, why would it happens?
BTW, will it be faster using theano with GPU than using numpy?

Answer

After serval hours searching and reading, i found an answer here.

When define a numpy array to shared variable, it becomes symbolic variable and are not numeric array anymore.

To compute with shared variables, the code should be modified as follow:

a = numpy.array([[0,1,2]])
b = numpy.array([[0],[1],[2]])#
#b = a.reshape(a.shape[1],a.shape[0])
a1=theano.shared(numpy.asarray(a), broadcastable =(True,False), borrow =True)
b1 = theano.shared(numpy.asarray(b),broadcastable=(False, True),borrow = True)

alpha_matrix = T.add(a1, b1)
alpha_matrix_compute = theano.function([], alpha_matrix)

s_t_1 = timeit.default_timer()
for i in range(10000):
    c = alpha_matrix_compute()
e_t_1 = timeit.default_timer()
for i in range(10000):
    c = numpy.add(a,b)
e_t_2 = timeit.default_timer()
print('t1:',e_t_1-s_t_1)
print('t2:',e_t_2-e_t_1)

Also, I compared the time consuming of broadcastable matrix add using theano and numpy. The result is

t1: 0.25077449798077067
t2: 0.022930744192201424

It seems that numpy is faster. In fact, the data transfer between GPU and CPU took a lot of time. That is the reason why t2 is much smaller that t1.