Steven Steven - 2 months ago 31
Python Question

Reset tensorflow Optimizer

I am loading from a saved model and I would like to be able to reset a tensorflow optimizer such as an Adam Optimizer. Ideally something like:

sess.run([tf.initialize_variables(Adamopt)])


or

sess.run([Adamopt.reset])


I have tried looking for an answer but have yet to find any way to do it. Here's what I've found which don't address the issue:
https://github.com/tensorflow/tensorflow/issues/634

In TensorFlow is there any way to just initialize uninitialised variables?

Tensorflow: Using Adam optimizer

I basically just want a way to reset the "slot" variables in the Adam Optimizer.

Thanks

Answer

The simplest way I found was to give the optimizer its own variable scope and then run

optimizer_scope = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,
                                 "scope/prefix/for/optimizer")
sess.run(tf.initialize_variables(optimizer_scope))

idea from freeze weights