Skip to main content

Problem 1.10 Completing the square - Answer

Prerequisite Knowledge​

To solve this problem, you need to be familiar with the following linear algebra concepts:

  1. Symmetric Matrix: A matrix AA is symmetric if AT=AA^T = A. In this problem, A∈SnA \in \mathbb{S}^n, so this property holds.
  2. Transpose Properties:
    • (A+B)T=AT+BT(A + B)^T = A^T + B^T
    • (AB)T=BTAT(AB)^T = B^T A^T
  3. Scalar Transpose: If a result is a scalar (a 1×11 \times 1 matrix), its transpose is equal to itself. For vectors x,b∈Rnx, b \in \mathbb{R}^n, the dot product xTbx^T b is a scalar, so xTb=(xTb)T=bTxx^T b = (x^T b)^T = b^T x.
  4. Matrix Inverse: AA−1=A−1A=IA A^{-1} = A^{-1} A = I.

Step-by-Step Derivation​

We want to show that the quadratic form: f(x)=xTAx−2xTb+c(1.28)f(x) = x^T A x - 2x^T b + c \qquad (1.28) can be rewritten as: f(x)=(x−d)TA(x−d)+e(1.29)f(x) = (x - d)^T A (x - d) + e \qquad (1.29) given: d=A−1b(1.30)d = A^{-1}b \qquad (1.30) e=c−dTAd=c−bTA−1b(1.31)e = c - d^T A d = c - b^T A^{-1} b \qquad (1.31)

Let's start by expanding the proposed form (1.29)(1.29) and substituting the values of dd and ee to see if we arrive at (1.28)(1.28).

Step 1: Expand the quadratic term (x−d)TA(x−d)(x - d)^T A (x - d)​

Using the transpose property (A+B)T=AT+BT(A+B)^T = A^T + B^T: (x−d)T=xT−dT(x - d)^T = x^T - d^T

So, (x−d)TA(x−d)=(xT−dT)A(x−d)(x - d)^T A (x - d) = (x^T - d^T) A (x - d)

Distribute the terms: =(xTA−dTA)(x−d)= (x^T A - d^T A) (x - d) =xTAx−xTAd−dTAx+dTAd= x^T A x - x^T A d - d^T A x + d^T A d

Step 2: Simplify the cross terms​

The two middle terms are −xTAd-x^T A d and −dTAx-d^T A x. Note that these terms are scalars (1×11 \times 1). Let's look at the term dTAxd^T A x. Since it is a scalar, it equals its transpose: (dTAx)T=xTAT(dT)T=xTATd(d^T A x)^T = x^T A^T (d^T)^T = x^T A^T d

Since AA is symmetric (A∈SnA \in \mathbb{S}^n), we know AT=AA^T = A. Therefore: (dTAx)T=xTAd(d^T A x)^T = x^T A d

So, dTAx=xTAdd^T A x = x^T A d. The cross terms are identical. The expression becomes: (x−d)TA(x−d)=xTAx−2xTAd+dTAd(x - d)^T A (x - d) = x^T A x - 2x^T A d + d^T A d

Step 3: Substitute expressions for dd and ee back into original equation (1.29)​

Now substitute this expansion back into (1.29)(1.29): f(x)=(xTAx−2xTAd+dTAd)+ef(x) = \left( x^T A x - 2x^T A d + d^T A d \right) + e

Substitute d=A−1bd = A^{-1}b into the term −2xTAd-2x^T A d: −2xTAd=−2xTA(A−1b)-2x^T A d = -2x^T A (A^{-1}b) =−2xT(AA−1)b= -2x^T (A A^{-1}) b =−2xTIb= -2x^T I b =−2xTb= -2x^T b

Substitute the definition of ee from (1.31)(1.31): e=c−dTAde = c - d^T A d

So the full expression becomes: f(x)=xTAx−2xTb+dTAd+(c−dTAd)f(x) = x^T A x - 2x^T b + d^T A d + (c - d^T A d)

Step 4: Final Cancellation​

The dTAdd^T A d terms cancel out: f(x)=xTAx−2xTb+dTAd−dTAd⏟_0+cf(x) = x^T A x - 2x^T b + \underbrace{d^T A d - d^T A d}\_{0} + c f(x)=xTAx−2xTb+cf(x) = x^T A x - 2x^T b + c

This matches equation (1.28)(1.28). Thus, we have shown that f(x)f(x) can indeed be rewritten in the form (1.29)(1.29) with the given parameters.