OHTO - 4 months ago 91

Python Question

Im trying to adapt the Keras VAE example to a deep network by adding one more layer.

Original code: Original VAE code

CHANGES:

`batch_size = 200`

original_dim = 784

latent_dim = 2

intermediate_dim_deep = 384 # <<<<<<<

intermediate_dim = 256

nb_epoch = 20

#

x = Input(batch_shape=(batch_size, original_dim))

x = Dense(intermediate_dim_deep, activation='relu')(x) # NEW LAYER <<<<<<

h = Dense(intermediate_dim, activation='relu')(x)

z_mean = Dense(latent_dim)(h)

z_log_var = Dense(latent_dim)(h)

#

def sampling(args):

z_mean, z_log_var = args

epsilon = K.random_normal(shape=(batch_size, latent_dim), mean=0.)

return z_mean + K.exp(z_log_var / 2) * epsilon

# note that "output_shape" isn't necessary with the TensorFlow backend

z = Lambda(sampling, output_shape=(latent_dim,))([z_mean, z_log_var])

#

# we instantiate these layers separately so as to reuse them later

decoder_h = Dense(intermediate_dim, activation='relu')

decoder_d = Dense(intermediate_dim_deep, activation='rely') # NEW LAYER <<<<<<

decoder_mean = Dense(original_dim, activation='sigmoid')

h_decoded = decoder_h(z)

d_decoded = decoder_d(h_decoded) # ADDED ONE MORE STEP HERE <<<<<<<

x_decoded_mean = decoder_mean(d_decoded)

#

def vae_loss(x, x_decoded_mean):

xent_loss = original_dim * objectives.binary_crossentropy(x, x_decoded_mean)

kl_loss = - 0.5 * K.sum(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis=-1)

return xent_loss + kl_loss

#

vae = Model(x, x_decoded_mean)

vae.compile(optimizer='rmsprop', loss=vae_loss)

#####

Compile I've me this error:

`/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py:1615: UserWarning: Model inputs must come from a Keras Input layer, they cannot be the output of a previous non-Input layer. Here, a tensor specified as input to "model_1" was not an Input tensor, it was generated by layer dense_1.`

Note that input tensors are instantiated via `tensor = Input(shape)`.

The tensor that caused the issue was: None

str(x.name))

---------------------------------------------------------------------------

Exception Traceback (most recent call last)

<ipython-input-8-c9010948cdee> in <module>()

----> 1 vae = Model(x, x_decoded_mean)

2 vae.compile(optimizer='rmsprop', loss=vae_loss)

/usr/local/lib/python2.7/dist-packages/keras/engine/topology.pyc in __init__(self, input, output, name)

1788 'The following previous layers '

1789 'were accessed without issue: ' +

-> 1790 str(layers_with_complete_input))

1791 for x in node.output_tensors:

1792 computable_tensors.append(x)

Exception: Graph disconnected: cannot obtain value for tensor input_1 at layer "input_1". The following previous layers were accessed without issue: []

I have the other examples in the repo and it seems a valid way to do it.

Am I missing something?

Answer

When adding the new hidden layer you're overriding the `x`

variable so you're left without an input layer. Also, is 'rely' a valid activation option?