[SOLVED] Ece490 homework 0

This problem set, which will not be graded and which you do not have to turn in, is meant to give you a rough idea of the material from multivariable calculus and linear algebra you should be comfortable with in order to do well in ECE 490.
1 Let f : Rn →R be a real-valued function of n real variables x1,…,xn, differentiable in each of these variables. Prove that the following identity holds for all x = (x1,…,xn)T and y = (y1,…,yn)T:

where is the gradient of f evaluated at x.
Hint: With x and y fixed, consider the function g(t) := f((1 − t)x + ty) of a single real variable t. Use the fundamental theorem of calculus to write

where g0(t) denotes the derivative of g with respect to t.
2 Consider the quadratic function, where x takes values in Rn and A is a given n×n matrix. Compute the gradient ∇f(x) and the Hessian ∇2f(x) (the n × n matrix of second-order
partial derivatives
3 Compute the gradient and the Hessian of the log-sum-exp function
,
where log denotes natural logarithm.
4 Let a matrix A ∈Rm×n and a vector b ∈Rm be given, such that ATA is invertible. Find the unique vector x∗ ∈Rn that minimizes the function
f(x) = kAx − bk2 = (Ax − b)T(Ax − b)
and prove that it is, indeed, unique (i.e., f(x) > f(x∗) for all x 6= x∗).
5 Let f : Rn →R be a differentiable function, such that the inequality
f(y) ≥ f(x) + ∇f(x)T(y − x)
holds for all vectors x,y ∈Rn. Prove that
f(tx + (1 − t)y) ≤ tf(x) + (1 − t)f(y)
for all x,y ∈Rn and all 0 ≤ t ≤ 1.
1
ECE 490 Homework 0 (not for turning in)

6 Let e1,…,en denote the standard unit vectors in Rn, i.e., the ith coordinate of ei is equal to 1 and all other coordinates are equal to 0. Determine the eigenvalues of the matrix
.
7 Let u and v be vectors in Rn, such that uTv = 06 . Show that u is an eigenvector of the matrix A = uvT and find the corresponding eigenvalue.
8 The singular values of a matrix A Rm×n are the nonnegative square roots of the eigenvalues of the symmetric matrix ATA. Consider a square matrix A Rn×n that can be written in the form
n
A = XσiuiviT, i=1
where σi are nonnegative real numbers and where {u1,…,un} and {v1,…,vn} are two orthonormal bases of
Rn, i.e.,
uTi uj = δij and viTvj = δij.
Here, δij is the Kronecker symbol, i.e., δij = 1 if i = j and 0 otherwise. Find the singular values of A in terms of the σi’s.
2