Cost versus Loss

It is helpful to me to establish conventional language. While working in machine learning, I’ve been fairly casual about my use of the terms ‘loss function’ and ‘cost function.’ This post is mostly aimed towards helping out myself in the future. Putting down the definition in writing will help my language precision.

I will follow the convention Andrew Ng uses in this Coursera lesson Logistic Regression Cost Function at 6:10.

The loss function measures how well you are doing on a single training example.

The cost function measures how well you are doing on the entire training set.