Open main menu

GM-RKB β

Structural Risk Minimization (SRM) Task

(Redirected from Structural Risk Minimization)

References

2020

  • (Wikipedia, 2020) ⇒ https://en.wikipedia.org/wiki/Structural_risk_minimization Retrieved:2020-2-28.
    • Structural risk minimization (SRM) is an inductive principle of use in machine learning. Commonly in machine learning, a generalized model must be selected from a finite data set, with the consequent problem of overfitting – the model becoming too strongly tailored to the particularities of the training set and generalizing poorly to new data. The SRM principle addresses this problem by balancing the model's complexity against its success at fitting the training data.

      In practical terms, Structural Risk Minimization is implemented by minimizing [math] E_{train} + \beta H(W) [/math] , where [math] E_{train} [/math] is the train error, the function [math] H(W) [/math] is called a regularization function, and [math] \beta [/math] is a constant. [math] H(W) [/math] is chosen such that it takes large values on parameters [math] W [/math] that belong to high-capacity subsets of the parameter space. Minimizing [math] H(W) [/math] in effect limits the capacity of the accessible subsets of the parameter space, thereby controlling the trade-off between minimizing the training error and minimizing the expected gap between the training error and test error.

      The SRM principle was first set out in a 1974 paper by Vladimir Vapnik and Alexey Chervonenkis and uses the VC dimension.

2017a

2017b

The basic lemma underlying the Structural Risk Minimization procedure is now easy for us to prove if we work through the definitions and use the union bound.

2012

2010

2008

Figure 1 (page 2) gives a diagrammatic representation of SRM.

 
Figure 1: Structural risk minimization.
  1. Vapnik, V. N., and A. Ya. Chervonenkis, 1974. Teoriya Raspoznavaniya Obrazov: Statisticheskie Problemy Obucheniya. (Russian) [Theory of Pattern Recognition: Statistical Problems of Learning]. Moscow: Nauka

2003

1992a

1992b