# Reproducing Kernel Hilbert Space

Finally arrive at reproducing kernel Hilbert space. https://nzer0.github.io/reproducing-kernel-hilbert-space.html

The above post introduces RKHS in Korean. It was helpful. I had struggled to understand some concepts in RKHS. What does mean Hilbert space in terms of feature expansion? ($$f:\mathcal{X} \to \mathbb{R}$$, $$f \in \mathcal{H}_K$$) It was confusing the difference between $$f$$ and $$f(x)$$. $$f$$ means the function in Hilbert space and $$f(x)$$ is evaluation.

I thought that the function can be represented by the inner product of the basis of feature space $$K(\cdot,x)$$ and coefficients $$f$$, and the coefficients are vectors in feature space.

The reproducing property of Kernel is $$\langle f, K(\cdot,x)\rangle_{\mathcal{H}} = f(x)$$. Thus $$K(\cdot,x) \in \mathcal{H}_K$$. $$K(\cdot,x)$$ is a $$x$$ specified function in Hilbert space $$\mathcal{H}_K$$ and an evaluator of the specific point x. This means the inner product of $$f$$ and $$K_{x}$$ is the value of $$f$$ at point $$x$$, $$f(x)$$.

In a nutshell, kenel method is a different way of evaluating f in a specific point $$x$$. Evaluating a function $$f$$ at a point $$x$$ is inner product of $$f$$ and $$L_x$$, where $$L_x \in \mathcal{H}_K$$ is a evaluation functional which is a kernal function and linear $$K(\cdot, x)$$. Reproducing property of $$\mathcal{H}_K$$ can be achieved if all $$f \in \mathcal{H}$$ has bounded evaluation functionals ($$L_x$$).

In least square methods, the parameters ($$\hat{\beta}$$) are determined by inner product of $$X$$ $$\hat{\beta} = (X^{T}X)^{-1}X^{T}y$$. In Kernel method, $$\hat{\beta}$$ is determined $$\langle K(\cdot,x_i), K(\cdot,x_j), \rangle_{\mathcal{H}_K} = K(x_i, x_j)$$. Each $$K(\cdot, x)$$ is a parameter and a argument (variable like $$x$$).

Some subclass of the loss function and penalty functions can be generated by a positive definite kernel. A Kernel accepts two arguments and a Kernel function does one argument and the other argument becomes parameter. Reproducing Kernel Hilbert space is a function space with Kernal function space with the evaluation functional as a Kernel. The feature expansion into the RKHS can use the Kernel matrix instead of the inner product of each variable $$X^TX$$.

The important concepts are Hilbert space, inner product, Kernel function, evaluation functional, feature expansion, Fourier transformation, Reisz representation theorem (dual space $$\mathcal{H}_{K}^*$$ of Hibert space $$\mathcal{H}_K$$)

##### Jun Kang
###### Clinical Assistant Professor of Hospital Pathology

My research interests include pathology, oncology and statistics.