Skip to main content

Answer

Pre-required Knowledge​

  • Variance of mean of i.i.d. variables: Var(1n∑Xi)=1n2∑Var(Xi)=1nVar(X1)\text{Var}(\frac{1}{n} \sum X_i) = \frac{1}{n^2} \sum \text{Var}(X_i) = \frac{1}{n} \text{Var}(X_1).
  • Variance inequality: Var(X)=E[X2]−(E[X])2≤E[X2]\text{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 \le \mathbb{E}[X^2].
  • Kernel property: Let M=max⁡uk(u)M = \max_u k(u). Then k(x−xih)≤Mk(\frac{x-x_i}{h}) \le M.

Step-by-Step Answer​

  1. Expression for Variance: var(p^(x))=var(1n∑i=1nk~(x−xi))\text{var}(\hat{p}(x)) = \text{var} \left( \frac{1}{n} \sum_{i=1}^n \tilde{k}(x - x_i) \right) Because xix_i are i.i.d., the variance of the sum is the sum of variances (covariance terms are zero). var(p^(x))=1n2∑i=1nvar(k~(x−xi))=1nvar(k~(x−x1))\text{var}(\hat{p}(x)) = \frac{1}{n^2} \sum_{i=1}^n \text{var}(\tilde{k}(x - x_i)) = \frac{1}{n} \text{var}(\tilde{k}(x - x_1))

  2. Apply Variance Inequality: Using var(Y)≤E[Y2]\text{var}(Y) \le \mathbb{E}[Y^2]: var(p^(x))≤1nE[(k~(x−x1))2]\text{var}(\hat{p}(x)) \le \frac{1}{n} \mathbb{E} [ (\tilde{k}(x - x_1))^2 ]

  3. Expand the square term: Recall k~(u)=1hdk(u/h)\tilde{k}(u) = \frac{1}{h^d} k(u/h).

    (k~(x−x1))2=(1hdk(x−x1h))2=1hdk(x−x1h)⋅1hdk(x−x1h)(\tilde{k}(x - x_1))^2 = \left( \frac{1}{h^d} k\left(\frac{x - x_1}{h}\right) \right)^2 = \frac{1}{h^d} k\left(\frac{x - x_1}{h}\right) \cdot \frac{1}{h^d} k\left(\frac{x - x_1}{h}\right)

  4. Apply the bound on Kernel: Using the hint k(u)≤max⁡xk(x)k(u) \le \max_x k(x), let M=max⁡xk(x)M = \max_x k(x).

    (k~(x−x1))2≤1hdk(x−x1h)⋅1hdM=Mhdk~(x−x1)(\tilde{k}(x - x_1))^2 \le \frac{1}{h^d} k\left(\frac{x - x_1}{h}\right) \cdot \frac{1}{h^d} M = \frac{M}{h^d} \tilde{k}(x - x_1)

  5. Compute Expectation:

    E[(k~(x−x1))2]≤E[Mhdk~(x−x1)]=MhdE[k~(x−x1)]\mathbb{E} [ (\tilde{k}(x - x_1))^2 ] \le \mathbb{E} \left[ \frac{M}{h^d} \tilde{k}(x - x_1) \right] = \frac{M}{h^d} \mathbb{E} [\tilde{k}(x - x_1)]

    From part (a), we know E[k~(x−x1)]=E[p^(x)]\mathbb{E} [\tilde{k}(x - x_1)] = \mathbb{E}[\hat{p}(x)].

    E[(k~(x−x1))2]≤MhdE[p^(x)]\mathbb{E} [ (\tilde{k}(x - x_1))^2 ] \le \frac{M}{h^d} \mathbb{E}[\hat{p}(x)]

  6. Final Result: Substitute back into the variance inequality from step 2:

    var(p^(x))≤1n(MhdE[p^(x)])=1nhdME[p^(x)]\text{var}(\hat{p}(x)) \le \frac{1}{n} \left( \frac{M}{h^d} \mathbb{E}[\hat{p}(x)] \right) = \frac{1}{nh^d} M \mathbb{E}[\hat{p}(x)]