florianletsch florianletsch - 4 months ago 30
Python Question

How to read in sequence of images as 3d tensor?

As an input to a CNN I am working on, I want to use a sequence of images (with 3D convolution in the conv layers).

However, I already fail in reading in the images as a 3D tensor that I can use for the computation.

Here is my primitive attempt:

def get_sequence_as_tensor(folder):
images = [folder + "/depth-%i.png" for i in range(15)]

tensor = tf.zeros(shape=(480, 640, 15), dtype=tf.float32)
for i, image in enumerate(images):
img = tf.image.decode_png(image)
img_float = tf.cast(img, tf.float32)
img_float = tf.reshape(img_float, (480, 640))
tensor[:, :, i] = img_float

return tensor


Which already fails because I cannot use index notation with tensors as I would expect from numpy arrays.

TypeError: 'Tensor' object does not support item assignment


What would be a proper way of reading in a sequence of images as a 3d tensor?

Answer

I would recommend combining the images into a numpy array outside of tensorflow and then passing them in to a placeholder.

Something like this should work.

filename = tf.placeholder("string")
png_string = tf.read_file(filename)
img = tf.image.decode_png(png_string)
img_float = tf.cast(img, tf.float32)
img_float = tf.reshape(img_float, (480, 640))

images = tf.placeholder("float", (480, 640, 15))
output = some_operation(images)

sess = tf.Session()

images_array = np.zeros((480, 640, 15), np.float32)
for i, image in enumerate(filenames):
    images_array[:,:,i] = sess.run(img_float, feed_dict{filename: image})

out = sess.run(output, feed_dict={images: images_array})

I hope this helps.