1992 AccelerationofStochasticApproxi

• (Polyak et al., 1992) ⇒ Boris T. Polyak, and Anatoli B. Juditsky. (1992). “Acceleration of Stochastic Approximation by Averaging.” In: SIAM Journal on Control and Optimization, 30(4). doi:10.1137/0330046

Subject Headings: Stochastic Gradient Descent Algorithm.

Cited By

• http://scholar.google.com/scholar?q=%22Acceleration+of+stochastic+approximation+by+averaging%22+1992
• http://dl.acm.org/citation.cfm?id=131092.131098&preflayout=flat#citedby

Quotes

Author Keywords

• optimal algorithms; recursive estimation; stochastic approximation; stochastic optimization

Abstract

A new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated. Convergence with probability one is proved for a variety of classical optimization and identification problems. It is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.

1. Introduction.

The methods of stochastic approximation originate in the works [29], [12] and are currently well studied [5], [21], [14], [16], [40].

References

• …
• [12] E. KIEFER AND J. WOLFOVITZ, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist., 23 (1952), pp. 462-466.
• …
• [29] H. ROBBINS AND S. MONROE, A stochastic-approximation method, Ann. Math. Statist., 22 (1951), pp. 400-407.
• …,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
1992 AccelerationofStochasticApproxi	Boris T. Polyak Anatoli B. Juditsky			Acceleration of Stochastic Approximation by Averaging				10.1137/0330046