reese0106 reese0106 - 6 months ago 69
Python Question

add custom loss to eval_metric_ops

I have defined my own loss function using sequence_loss

loss = tf.contrib.legacy_seq2seq.sequence_loss(logits, labels, weights)

I am hoping to add this to eval_metric_ops so that in my ML engine package I can display the evaluation loss in tensorboard continuously (defaults are just accuracy).I tried adding this as a custom eval_metric_ops

eval_metric_ops = {
'loss': loss # this has already been coputed for Modes.EVAL

However, I get the error "TypeError: Values of eval_metric_ops must be (metric_value, update_op) tuples, given: Tensor("sequence_loss/truediv:0", shape=(), dtype=float32) for key: loss" -- what do I need to do to pass the loss as an eval_metric_op? I guessing that my current loss should be the metric value, but I'm not sure what the update_op should be?

Answer Source

The metric function in your case can be implemented using tf.metrics using an moving average of the loss:

def metric_fn(labels, predict):
   loss = tf.contrib.legacy_seq2seq.sequence_loss(logits, labels, weights)
   mean, op = tf.metrics.mean(loss)
   return mean, op
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download