Karnivaurus - 2 months ago 70

Python Question

I am looking at the TensorFlow "MNIST For ML Beginners" tutorial, and I want to print out the training loss after every training step.

My training loop currently looks like this:

`for i in range(100):`

batch_xs, batch_ys = mnist.train.next_batch(100)

sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

Now,

`train_step`

`train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)`

Where

`cross_entropy`

`cross_entropy = -tf.reduce_sum(y_ * tf.log(y))`

One way to print this would be to explicitly compute

`cross_entropy`

`for i in range(100):`

batch_xs, batch_ys = mnist.train.next_batch(100)

cross_entropy = -tf.reduce_sum(y_ * tf.log(y))

print 'loss = ' + str(cross_entropy)

sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

I now have two questions regarding this:

- Given that is already computed during
`cross_entropy`

, it seems inefficient to compute it twice, requiring twice the number of forward passes of all the training data. Is there a way to access the value of`sess.run(train_step, ...)`

when it was computed during`cross_entropy`

?`sess.run(train_step, ...)`

- How do I even print a ? Using
`tf.Variable`

gives me an error...`str(cross_entropy)`

Thank you!

Answer

You can fetch the value of `cross_entropy`

by adding it to the list of arguments to `sess.run(...)`

. For example, your `for`

-loop could be rewritten as follows:

```
for i in range(100):
batch_xs, batch_ys = mnist.train.next_batch(100)
cross_entropy = -tf.reduce_sum(y_ * tf.log(y))
_, loss_val = sess.run([train_step, cross_entropy],
feed_dict={x: batch_xs, y_: batch_ys})
print 'loss = ' + loss_val
```

The same approach can be used to print the current value of a variable. Let's say, in addition to the value of `cross_entropy`

, you wanted to print the value of a `tf.Variable`

called `W`

, you could do the following:

```
for i in range(100):
batch_xs, batch_ys = mnist.train.next_batch(100)
cross_entropy = -tf.reduce_sum(y_ * tf.log(y))
_, loss_val, W_val = sess.run([train_step, cross_entropy, W],
feed_dict={x: batch_xs, y_: batch_ys})
print 'loss = %s' % loss_val
print 'W = %s' % W_val
```

Source (Stackoverflow)

Comments