Skip to main content

Question

Problem 5.2 Mean and variance of a kernel density estimate

In this problem, we will study the mean and variance of the kernel density estimate, i.e., the distribution p^(x)\hat{p}(x). Let X={x1,⋯ ,xn}X = \{x_1, \cdots, x_n\} be the set of samples, and k~(x)\tilde{k}(x) be the kernel with bandwidth included. The estimated probability distribution is p^(x)=1n∑∗i=1nk~(x−xi).(5.5)\hat{p}(x) = \frac{1}{n} \sum*{i=1}^n \tilde{k}(x - x_i). \quad (5.5) Suppose that the kernel function k~(x)\tilde{k}(x) has zero mean and covariance HH, i.e., E∗k~[x]=∫k~(x)xdx=0,(5.6)\mathbb{E}*{\tilde{k}}[x] = \int \tilde{k}(x) x dx = 0, \quad (5.6) cov⁡k~(x)=∫k~(x)(x−Ek~[x])(x−E_k~[x])Tdx=H.(5.7)\operatorname{cov}_{\tilde{k}}(x) = \int \tilde{k}(x) (x - \mathbb{E}_{\tilde{k}}[x])(x - \mathbb{E}\_{\tilde{k}}[x])^T dx = H. \quad (5.7)

(b) Show that the covariance of the distribution p^(x)\hat{p}(x) is Σ^=cov⁡p^(x)=H+1n∑i=1n(xi−μ^)(xi−μ^)T,(5.9)\hat{\Sigma} = \operatorname{cov}_{\hat{p}}(x) = H + \frac{1}{n} \sum_{i=1}^n (x_i - \hat{\mu})(x_i - \hat{\mu})^T, \quad (5.9) where the second term on the right hand side is the sample covariance.