Samuel Varghese - 4 months ago 50

Python Question

I'm trained a model in Keras, only Dense layers. However, when i try to predict it gives me the same answer all the time even with different values.

`import numpy`

from keras.models import Sequential

from keras.layers import Dense

from keras.layers import LSTM

from keras.layers import Dropout

from keras.layers.embeddings import Embedding

from keras.optimizers import Adam

import pandas as pd

import tensorflow as tf

tf.python.control_flow_ops = tf

df = pd.read_csv('/home/sam/Documents/data.csv')

dfX = df[['Close']]

dfY = df[['Y']]

bobX = dfX.as_matrix()

boby = dfY.as_matrix()

model = Sequential()

model.add(Dense(200, input_dim=1))

model.add(Activation('sigmoid'))

model.add(Dense(75))

model.add(Activation('sigmoid'))

model.add(Dense(10))

model.add(Activation('sigmoid'))

model.add(Dense(1))

adam = Adam(lr=0.1)

model.compile(loss='mse', optimizer= adam)

print(model.summary())

model.fit(bobX, boby, nb_epoch=2500, batch_size=500, verbose=0)

model.predict(np.array([[210.99]]))

Answer

Your learning rate is WAY to high for Adam. Actually 0.1 is too high for most optimizers I have used. You should use 1e-3 or 1e-4 as the learning rate. These usually work well for me. When you use that high of a learning rate the model will fail to converge. From my experience it often just settles for the constant average value of the problem.