W.P. McNeill W.P. McNeill - 3 months ago 72
Python Question

How do I pass a scalar via a TensorFlow feed dictionary

My TensorFlow model uses

tf.random_uniform
to initialize a variable. I would like to specify the range when I begin training, so I created a placeholder for the initialization value.

init = tf.placeholder(tf.float32, name="init")
v = tf.Variable(tf.random_uniform((100, 300), -init, init), dtype=tf.float32)
initialize = tf.initialize_all_variables()


I initialize variables at the start of training like so.

session.run(initialize, feed_dict={init: 0.5})


This gives me the following error:

ValueError: initial_value must have a shape specified: Tensor("Embedding/random_uniform:0", dtype=float32)


I cannot figure out the correct
shape
parameter to pass to
tf.placeholder
. I would think for a scalar I should do
init = tf.placeholder(tf.float32, shape=0, name="init")
but this gives the following error:

ValueError: Incompatible shapes for broadcasting: (100, 300) and (0,)


If I replace
init
with the literal value
0.5
in the call to
tf.random_uniform
it works.

How do I pass this scalar initial value via the feed dictionary?

Answer

TL;DR: Define init with a scalar shape as follows:

init = tf.placeholder(tf.float32, shape=(), name="init")

This looks like an unfortunate implementation detail of tf.random_uniform(): it currently uses tf.add() and tf.mul() to rescale the random value from [-1, +1] to [minval, maxval], but if the shape of minval or maxval is unknown, tf.add() and tf.mul() can't infer the proper shapes, because there might be broadcasting involved.

By defining init with a known shape (where a scalar is () or [], not 0), TensorFlow can draw the proper inferences about the shape of the result of tf.random_uniform(), and your program should work as intended.