ckorzhik ckorzhik - 3 months ago 35
Python Question

TensorFlow: unset 'reuse' in variable scope

Is there way to unset

reuse
in variable scope?
I tried following commands:

In [1]: import tensorflow as tf

In [2]: tf.get_variable_scope().reuse
Out[2]: False

In [3]: tf.get_variable_scope().reuse_variables
Out[3]: <bound method VariableScope.reuse_variables of <tensorflow.python.ops.variable_scope.VariableScope object at 0x7fd9cc46c4d0>>

In [4]: tf.get_variable_scope().reuse_variables()

In [5]: tf.get_variable_scope().reuse
Out[5]: True

In [6]: tf.get_variable_scope().reuse_variables()

In [7]: tf.get_variable_scope().reuse
Out[7]: True

In [8]: tf.get_variable_scope().reuse_variables(False)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-8-ba19ed12625c> in <module>()
----> 1 tf.get_variable_scope().reuse_variables(False)

TypeError: reuse_variables() takes exactly 1 argument (2 given)

In [9]: tf.get_variable_scope().reuse = False
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-9-ddd0b37e4f0e> in <module>()
----> 1 tf.get_variable_scope().reuse = False

AttributeError: can't set attribute

In [10]: tf.get_variable_scope().reuse_variables = False

In [11]: tf.get_variable_scope().reuse_variables()
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-d03dc0fb25b6> in <module>()
----> 1 tf.get_variable_scope().reuse_variables()

TypeError: 'bool' object is not callable


As you can see, I can't unset
reuse
by calling
reuse_variables
even number of times, I can't set it with
=
operator and for some reason I can set whatever I want instead
reuse_variables
function (is it a bug?).

Answer

Once you're in a scope I think it's by design that you can't change the reuse state until you reopen the scope again with the flag set the other way:

with tf.variable_scope("scope"):
    a = tf.get_variable("var_a", 1) 
    print(a.name)

with tf.variable_scope("scope", reuse = True):
    b = tf.get_variable("var_a")
    print b.name 
    #c = tf.get_variable("var_b") # won't work
    # can't reuse something that doesn't exist
    # probably enforced so you don't make unintended variables

with tf.variable_scope("scope"): #reuse False
    #c = tf.get_variable("var_a") # won't work
    # there is another variable with the same name
    # makes sure you don't override the previous variable                    
    c = tf.get_variable("var_b",2)
    print c.name

you could hack around it this way:

with tf.variable_scope("scope") as scope:
    a = tf.get_variable("var_a", 1)  
    print(a.name)

with tf.variable_scope("scope", reuse = True):
    b = tf.get_variable("var_a")
    print b.name 
    with tf.variable_scope(scope):
        c = tf.get_variable("var_b", 1)
        print c.name

I guess you could do it this way:

print(tf.get_variable_scope().reuse) #False
tf.get_variable_scope().reuse_variables()
print(tf.get_variable_scope().reuse) #True

with tf.variable_scope(tf.get_variable_scope(), reuse=False):
    print(tf.get_variable_scope().reuse) #False
Comments