noobiejp noobiejp -4 years ago 137
Python Question

Activation function error in a 1D CNN in Keras

I'm creating a model to classify if the input waverform contains rising edge of SDA of I2C line.

My input has 20000 datapoints and 100 training data.

I've initially found an answer regarding the input in here Keras 1D CNN: How to specify dimension correctly?

However, I'm getting an error in the activation function:

ValueError: Error when checking target: expected activation_1 to have 3 dimensions, but got array with shape (100, 1)


My model is:

model.add(Conv1D(filters=n_filter,
kernel_size=input_filter_length,
strides=1,
activation='relu',
input_shape=(20000,1)))
model.add(BatchNormalization())
model.add(MaxPooling1D(pool_size=4, strides=None))

model.add(Dense(1))
model.add(Activation("sigmoid"))

adam = Adam(lr=learning_rate)

model.compile(optimizer= adam, loss='binary_crossentropy', metrics=['accuracy'])

model.fit(train_data, train_label,
nb_epoch=10,
batch_size=batch_size, shuffle=True)

score = np.asarray(model.evaluate(test_new_data, test_label, batch_size=batch_size))*100.0


I can't determine the problem in here. On why the activation function expects a 3D tensor.

Answer Source

The problem lies in the fact that starting from keras 2.0, a Dense layer applied to a sequence will apply the layer to each time step - so given a sequence it will produce a sequence. So your Dense is actually producing a sequence of 1-element vectors and this causes your problem (as your target is not a sequence).

There are several ways on how to reduce a sequence to a vector and then apply a Dense to it:

  1. GlobalPooling:

    You may use GlobalPooling layers like GlobalAveragePooling1D or GlobalMaxPooling1D, eg.:

    model.add(Conv1D(filters=n_filter,
             kernel_size=input_filter_length,
             strides=1,
             activation='relu',
             input_shape=(20000,1)))
    model.add(BatchNormalization())
    model.add(GlobalMaxPooling1D(pool_size=4, strides=None))
    
    model.add(Dense(1))
    model.add(Activation("sigmoid"))
    
  2. Flattening:

    You might colapse the whole sequence to a single vector using Flatten layer:

    model.add(Conv1D(filters=n_filter,
             kernel_size=input_filter_length,
             strides=1,
             activation='relu',
             input_shape=(20000,1)))
    model.add(BatchNormalization())
    model.add(MaxPooling1D(pool_size=4, strides=None))
    model.add(Flatten())
    
    model.add(Dense(1))
    model.add(Activation("sigmoid"))
    
  3. RNN Postprocessing:

    You could also add a recurrent layer on a top of your sequence and make it to return only the last output:

    model.add(Conv1D(filters=n_filter,
             kernel_size=input_filter_length,
             strides=1,
             activation='relu',
             input_shape=(20000,1)))
    model.add(BatchNormalization())
    model.add(MaxPooling1D(pool_size=4, strides=None))
    model.add(SimpleRNN(10, return_sequences=False))
    
    model.add(Dense(1))
    model.add(Activation("sigmoid"))
    
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download