Iterative Scaling Algorithm

From GM-RKB
(Redirected from iterative scaling algorithm)
Jump to navigation Jump to search

A Iterative Scaling Algorithm is an optimization algorithm that ...



References

2007

2003

  • (Jin et al., 2003) ⇒ Rong Jin, Rong Yan, Jian Zhang, and Alexander G. Hauptmann. (2003). “A Faster Iterative Scaling Algorithm for Conditional Exponential Model.” In: Proceedings of ICML 2003.
    • QUOTE: To find the optimal conditional exponential model for given training data, two groups of approaches have been used in the past research. One is named iterative scaling approach (Brown, 1959), including the Generalized Iterative Scaling (GIS) (Darroch & Ratcli, 1972) and the Improved Iterative Scaling (IIS) (Berger, 1997). The underlying idea for iterative scaling approaches is similar to the idea of Expectation- Maximization (EM) approach: by approximating the log-likelihood function of the conditional exponential model as some kind of ‘simple’ auxiliary function, the iterative scaling methods are able to decouple the correlation between the parameters and the search for the maximum point can be operated along many directions simultaneously. By carrying out this procedure iteratively, the approximated optimal point found over the ‘simplified’ function is guaranteed to converge to the true optimal point due to the convexity of the objective function. The distinction between GIS and IIS is that the GIS method requires the sum of input features to be a constant over all the examples while the IIS method doesn’t.

2000

1997

1996

1972

1959

  • (Brown, 1959) ⇒ D. Brown. (1959). “A Note on Approximations to Discrete Probability Distributions.” In: Information and Control.