W.P. McNeill W.P. McNeill - 11 months ago 364
Python Question

How do I pass a scalar via a TensorFlow feed dictionary

My TensorFlow model uses

to initialize a variable. I would like to specify the range when I begin training, so I created a placeholder for the initialization value.

init = tf.placeholder(tf.float32, name="init")
v = tf.Variable(tf.random_uniform((100, 300), -init, init), dtype=tf.float32)
initialize = tf.initialize_all_variables()

I initialize variables at the start of training like so.

session.run(initialize, feed_dict={init: 0.5})

This gives me the following error:

ValueError: initial_value must have a shape specified: Tensor("Embedding/random_uniform:0", dtype=float32)

I cannot figure out the correct
parameter to pass to
. I would think for a scalar I should do
init = tf.placeholder(tf.float32, shape=0, name="init")
but this gives the following error:

ValueError: Incompatible shapes for broadcasting: (100, 300) and (0,)

If I replace
with the literal value
in the call to
it works.

How do I pass this scalar initial value via the feed dictionary?

Answer Source

TL;DR: Define init with a scalar shape as follows:

init = tf.placeholder(tf.float32, shape=(), name="init")

This looks like an unfortunate implementation detail of tf.random_uniform(): it currently uses tf.add() and tf.mul() to rescale the random value from [-1, +1] to [minval, maxval], but if the shape of minval or maxval is unknown, tf.add() and tf.mul() can't infer the proper shapes, because there might be broadcasting involved.

By defining init with a known shape (where a scalar is () or [], not 0), TensorFlow can draw the proper inferences about the shape of the result of tf.random_uniform(), and your program should work as intended.