Kernel Regression Algorithm

A Kernel Regression Algorithm is a non-parametric regression algorithm that can be implemented by a kernel regression system to solve a kernel regression task.

• Example(s):
  • Nadaraya-Watson Kernel Regression Algorithm.
  • …
• Counter-Example(s):
  • k-Nearest Neighbor Algorithm.
• See: Conditional Expectation, Kernel Principal Component Analysis.

---

---

References

2017

• (Statsmodels, 2017) ⇒ http://www.statsmodels.org/dev/generated/statsmodels.nonparametric.kernel_regression.KernelReg.html
  • QUOTE: … Calculates the conditional mean [math]\displaystyle{ E[y|X] }[/math] where [math]\displaystyle{ g(X)+e }[/math]. Note that the “local constant” type of regression provided here is also known as Nadaraya-Watson kernel regression; “local linear” is an extension of that which suffers less from bias issues at the edge of the support(...)

2015

• (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/kernel_regression Retrieved:2015-1-14.
  • The kernel regression is a non-parametric technique in statistics to estimate the conditional expectation of a random variable. The objective is to find a non-linear relation between a pair of random variables X and Y.

    In any nonparametric regression, the conditional expectation of a variable [math]\displaystyle{ Y }[/math] relative to a variable [math]\displaystyle{ X }[/math] may be written:

    [math]\displaystyle{ \operatorname{E}(Y | X) = m(X) }[/math]

    where [math]\displaystyle{ m }[/math] is an unknown function.

2007

• (Weinberger & Tesauro, 2007) ⇒ K.Q. Weinberger, G. Tesauro. (2007). “Metric Learning for Kernel Regression.” In: Proceedings of International Workshop on Artificial Intelligence and Statistics.

2003

• (Gartner, 2003) ⇒ Thomas Gärtner. (2003). “A Survey of Kernels for Structured Data.” In: ACM SIGKDD Explorations Newsletter, 5(1).

1999

• (Jaakkola & Haussler, 1999) ⇒ T. Jaakkola and D. Haussler. (1999). “Probabilistic Kernel Regression Models". In: Proceedings of the 1999 Conference on AI and Statistics, 1999.

1997

• (Caruana, 1997) ⇒ Rich Caruana. (1997). “Multitask Learning.” In: Machine Learning, 28(1) doi:10.1023/A:1007379606734
  • QUOTE: This paper reviews prior work on MTL, presents new evidence that MTL in backprop nets discovers task relatedness without the need of supervisory signals, and presents new results for MTL with k-nearest neighbor and kernel regression.

---