Karl - 6 months ago 57

Python Question

I have two square matrices of the same size and the dimensions of a square patch. I'd like to compute the dot product between every pair of patches. Essentially I would like to implement the following operation:

`def patch_dot(A, B, patch_dim):`

res_dim = A.shape[0] - patch_dim + 1

res = np.zeros([res_dim, res_dim, res_dim, res_dim])

for i in xrange(res_dim):

for j in xrange(res_dim):

for k in xrange(res_dim):

for l in xrange(res_dim):

res[i, j, k, l] = (A[i:i + patch_dim, j:j + patch_dim] *

B[k:k + patch_dim, l:l + patch_dim]).sum()

return res

Obviously this would be an extremely inefficient implementation. Tensorflow's tf.nn.conv2d seems like a natural solution to this as I'm essentially doing a convolution, however my filter matrix isn't fixed. Is there a natural solution to this in Tensorflow, or should I start looking at implementing my own tf-op?

Answer

The natural way to do this is to first extract overlapping image patches of matrix B using tf.extract_image_patches, then to apply the tf.nn.conv2D function on A and each B sub-patch using tf.map_fn.

Note that prior to use tf.extract_image_patches and tf.nn.conv2D you need to reshape you matrices as 4D tensors of shape `[1, width, height, 1]`

using tf.reshape.

Also, prior to use tf.map_fn, you would also need to use the tf.transpose op so that the B sub-patches are indexed by the first dimension of the tensor you use as the `elems`

argument of tf.map_fn.