I'm fitting a neural network in Python Keras.
To avoid overfitting I would like to monitor the training/validation loss and create a proper callback which stops computations when training loss is too much less than validation loss.
An example of a callback is:
callback = [EarlyStopping(monitor='val_loss', value=45, verbose=0, mode='auto')]
You can create a custom callback class for your purpose.
I have created one that should correspond to your need :
class CustomEarlyStopping(Callback): def __init__(self, ratio=0.0, patience=0, verbose=0): super(EarlyStopping, self).__init__() self.ratio = ratio self.patience = patience self.verbose = verbose self.wait = 0 self.stopped_epoch = 0 self.monitor_op = np.greater def on_train_begin(self, logs=None): self.wait = 0 # Allow instances to be re-used def on_epoch_end(self, epoch, logs=None): current_val = logs.get('val_loss') current_train = logs.get('loss') if current_val is None: warnings.warn('Early stopping requires %s available!' % (self.monitor), RuntimeWarning) # If ratio current_loss / current_val_loss > self.ratio if self.monitor_op(np.divide(current_train,current_val),self.ratio): self.wait = 0 else: if self.wait >= self.patience: self.stopped_epoch = epoch self.model.stop_training = True self.wait += 1 def on_train_end(self, logs=None): if self.stopped_epoch > 0 and self.verbose > 0: print('Epoch %05d: early stopping' % (self.stopped_epoch))
I took the liberty to interpret that you wanted to stop if the ratio between the
train_loss and the
validation_loss goes under a certain ratio threshold. This ratio argument should be between
1.0 is dangerous as the validation loss and the training loss might fluctuate a lot in an erratic way at the beginning of the training.
You can add a patience argument which will wait to see if the breaking of your threshold is staying for a certain number of epochs.
The way to use this is for exampe :
callbacks = [CustomEarlyStopping(ratio=0.5, patience=2, verbose=1), ... Other callbacks ...] ... model.fit(..., callbacks=callbacks)
In this case it will stop if the training loss stays lower than
0.5*val_loss for more than 2 epochs.
Does that help you?