Huber Loss Function

From GM-RKB
Jump to navigation Jump to search

A Huber Loss Function is a Loss Function that is used for regression.



References

2021a

  • (Wikipedia, 2021) ⇒ https://en.wikipedia.org/wiki/Huber_loss Retrieved:2021-3-7.
    • In statistics, the Huber loss is a loss function used in robust regression, that is less sensitive to outliers in data than the squared error loss. A variant for classification is also sometimes used.

      (...)The Huber loss function describes the penalty incurred by an estimation procedure . Huber (1964) defines the loss function piecewise by  : L_\delta (a) = \begin{cases} \frac{1}{2}{a^2} & \text{for } |a| \le \delta, \\ \delta (|a| - \frac{1}{2}\delta), & \text{otherwise.} \end{cases} This function is quadratic for small values of , and linear for large values, with equal values and slopes of the different sections at the two points where |a| = \delta . The variable often refers to the residuals, that is to the difference between the observed and predicted values a = y - f(x) , so the former can be expanded to Compared to Hastie et al., the loss is scaled by a factor of ½, to be consistent with Huber's original definition given earlier.  : L_\delta(y, f(x)) = \begin{cases} \frac{1}{2}(y - f(x))^2 & \textrm{for } |y - f(x)| \le \delta, \\ \delta\, |y - f(x)| - \frac{1}{2}\delta^2 & \textrm{otherwise.} \end{cases}

2021b

  • (ML Glossary, 2021) ⇒ https://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html Retrieved:2021-03-06.
    • QUOTE: Typically used for regression. It’s less sensitive to outliers than the MSE as it treats error as square only inside an interval.

      $L_{\delta}=\left\{\begin{array}{cc}\frac{1}{2}(y-\hat{y})^{2} & i f|(y-\hat{y})|<\delta \\\delta\left((y-\hat{y})-\frac{1}{2} \delta\right) & \text { otherwise }\end{array}\right.$

      Code

      def Huber(yHat, y, delta=1.):

         return np.where(np.abs(y-yHat) < delta,.5*(y-yHat)**2 , delta*(np.abs(y-yHat)-0.5*delta))