Nadaraya-Watson Kernel Regression Task

A Nadaraya-Watson Kernel Regression Task is a Kernel Regression Task that is based on Nadaraya-Watson Estimator.

• AKA: Nadaraya-Watson Kernel Estimation Task.
• Context:
  • Task Input:

    a N-observed Numerically-Labeled Training Dataset [math]\displaystyle{ D=\{(x_1,y_1,z_1,...),(x_2,y_2,z_2,...),\cdots(x_n,y_n,z_n,...)\} }[/math] that can be represented by

    • [math]\displaystyle{ \mathbf{Y} }[/math] response variable continuous dataset.
    • [math]\displaystyle{ \mathbf{X} }[/math] predictor variables continuous dataset.
  • Task Output:
    • [math]\displaystyle{ Y^* }[/math], predited reponse variable values
  • Task Requirements
    • It requires to calculate using nonparametric regression function distribution of the form:

      [math]\displaystyle{ f(x)=\frac{1}{n}\sum_{i=1}^n K_h(x-x_i) }[/math]

      that can solve the [math]\displaystyle{ y_i=f(x_i)+ \epsilon_i }[/math] and it is based on the Nadaray-Watson estimator.

    • It may require a regression diagnostic test to determine goodness of fit of the regression model.
  • It can be solved by a Kernel Regression System that implements a Kernel Regression Algorithm.
  • It can range from being Locally Weighted Regression Task to being a Nonlinear Regression Task.
• Example(s):
  • …
• Counter-Example(s):
  • A Linear Regression Task.
  • A Regularized Linear Regression Task.
  • A Linear Least-Squares Regression Task.
• See: Conditional Expectation, Non-Parametric, Random Variable, Nonparametric Regression, Gaussian Process Regression, Kernel Density Estimation.

---

---

References

2017

• (Wikipedia, 2017) ⇒ https://en.wikipedia.org/wiki/Kernel_regression#Nadaraya Retrieved:2017-9-3.
  • Nadaraya and Watson, both in 1964, proposed to estimate [math]\displaystyle{ m }[/math] as a locally weighted average, using a kernel as a weighting function. The Nadaraya–Watson estimator is: [math]\displaystyle{ \widehat{m}_h(x)=\frac{\sum_{i=1}^n K_h(x-x_i) y_i}{\sum_{i=1}^nK_h(x-x_i)} }[/math] where [math]\displaystyle{ K }[/math] is a kernel with a bandwidth [math]\displaystyle{ h }[/math] . The denominator is a weighting term with sum 1.

    Derivation

    [math]\displaystyle{ \operatorname{E}(Y | X=x) = \int y f(y|x) dy = \int y \frac{f(x,y)}{f(x)} dy }[/math] Using the kernel density estimation for the joint distribution f(x,y) and f(x) with a kernel K, [math]\displaystyle{ \hat{f}(x,y) = \frac{1}{n}\sum_{i=1}^{n} K_h\left(x-x_i\right) K_h\left(y-y_i\right) }[/math] ,
    [math]\displaystyle{ \hat{f}(x) = \frac{1}{n} \sum_{i=1}^{n} K_h\left(x-x_i\right) }[/math] ,

    we get [math]\displaystyle{ \operatorname{\hat E}(Y | X=x) = \int \frac{y \sum_{i=1}^{n} K_h\left(x-x_i\right) K_h\left(y-y_i\right)}{\sum_{i=1}^{n} K_h\left(x-x_i\right)} dy, }[/math] [math]\displaystyle{ \operatorname{\hat E}(Y | X=x) = \frac{\sum_{i=1}^{n} K_h\left(x-x_i\right) \int y \, K_h\left(y-y_i\right) dy}{\sum_{i=1}^{n} K_h\left(x-x_i\right)}, }[/math] [math]\displaystyle{ \operatorname{\hat E}(Y | X=x) = \frac{\sum_{i=1}^{n} K_h\left(x-x_i\right) y_i}{\sum_{i=1}^{n} K_h\left(x-x_i\right)}, }[/math] which is the Nadaraya–Watson estimator.

---