Mastering the SVD: From Theory to Engineering Applications in Haskell

Introduction: Why SVD Matters in Modern Engineering

In the age of AI, data compression, and recommendation systems, the Singular Value Decomposition (SVD) is a cornerstone of linear algebra. From Netflix prize-winning algorithms to Google's PageRank, SVD powers many modern technologies. In this tutorial, we'll demystify the full and reduced SVD, explore key properties, and implement them in Haskell. Whether you're studying for the ENGG7302 exam or building your next AI app, understanding SVD is essential.

What is the Singular Value Decomposition?

The SVD of a real or complex matrix A is a factorization of the form A = U Σ VH, where:

• U and V are unitary matrices (orthogonal if real).
• Σ is a diagonal matrix with non-negative real numbers (singular values) in descending order.

The full SVD includes all singular values, while the reduced SVD (also called thin SVD) only keeps the non-zero singular values and corresponding columns. For an m×n matrix of rank r, the full SVD yields U (m×m), Σ (m×n), V (n×n); the reduced SVD yields U (m×r), Σ (r×r), V (n×r).

Key Properties of SVD

From the exam questions, we learn that:

• U and V are not necessarily the same – they are different unitary matrices (contradicting statement [1] in Question 1).
• A-1 = V Σ-1 UH for invertible matrices (statement [2] is correct for square invertible matrices).
• Σ is unique up to ordering – the singular values are unique, but the matrices U and V are not unique (statement [3] in Question 8 is correct).

Understanding these properties helps avoid common pitfalls in exam questions.

Computing SVD in Haskell

We'll use the hmatrix library for linear algebra. Install it via cabal install hmatrix.

import Numeric.LinearAlgebra

-- Full SVD
fullSVD :: Matrix Double -> (Matrix Double, Vector Double, Matrix Double)
fullSVD m = let (u, s, v) = svd m in (u, s, v)

-- Reduced SVD (thin)
reducedSVD :: Matrix Double -> (Matrix Double, Vector Double, Matrix Double)
reducedSVD m = let (u, s, v) = thinSVD m in (u, s, v)

Example: Full vs Reduced SVD

Consider a rank-2 matrix:

let a = (3 >< 3) [1,0,0, 0,2,0, 0,0,0]
let (u_full, s_full, v_full) = fullSVD a
let (u_thin, s_thin, v_thin) = reducedSVD a

The full SVD returns a 3×3 U, 3×3 Σ, 3×3 V. The reduced SVD returns 3×2 U, 2×2 Σ, 3×2 V. The singular values are [2,1,0] – note the zero is omitted in reduced SVD.

Real-World Analogy: AI Recommendation Systems

In a movie recommendation system, the user-item matrix is decomposed via SVD to capture latent factors. The reduced SVD keeps only the top k singular values, compressing the data while preserving the most important patterns. This is similar to how Spotify or Netflix suggest content – by reducing dimensionality, they focus on the strongest signals.

Exam-Style Question Walkthrough

Question 1 (from exam): Which statements about full and reduced SVD are correct?

Let's test in Haskell:

let a = (2 >< 2) [1,0, 0,2]
let (u_full, s_full, v_full) = fullSVD a
let (u_thin, s_thin, v_thin) = reducedSVD a
-- Check if U and V are the same
print (u_full == v_full) -- False
-- Check if Σ are different
print (s_full == s_thin) -- True (since all singular values non-zero)

Thus, only [4] (U, V may have the same rank) is correct in some contexts, but the correct answer is (d) None – because [4] is false? Actually, U and V always have the same rank as A. So [4] is true. But careful: the statements in the exam are nuanced. The correct answer is (c) Only [4].

Pseudo-inverse and Condition Number

The pseudo-inverse A+ = V Σ+ UH, where Σ+ has reciprocals of non-zero singular values. In Haskell:

let pinv = pinv a
let condNum = (maximum s) / (minimum (filter (>0) s))

For the matrix in Question 11, A = diag [2,0,0], the pseudo-inverse is diag [0.5,0,0] and condition number is infinite (division by zero).

Trending Context: AI and Data Compression

In 2026, AI models like GPT-5 use SVD for weight compression. By applying reduced SVD to weight matrices, engineers reduce model size without significant accuracy loss. This technique, called low-rank approximation, is also used in image compression (JPEG-like) and natural language processing.

Common Mistakes in SVD

• Confusing full and reduced SVD – remember reduced SVD removes zero singular values and corresponding columns.
• Assuming U and V are the same – they are generally different.
• Thinking Σ is unique – the singular values are unique, but U and V are not.

Conclusion

Mastering SVD is crucial for advanced computational techniques in engineering. By implementing it in Haskell, you gain practical insight into its properties. Whether you're preparing for the ENGG7302 final exam or building the next AI app, SVD is a tool you'll use again and again.