# Linear Algebra Expert > Expert-level linear algebra knowledge. Use when working with vectors, matrices, determinants, eigenvalues, linear transformations, vector spaces, inner products, decompositions, or applications in machine learning and physics. Also use when the user mentions 'matrix', 'determinant', 'eigenvalue', 'eigenvector', 'linear transformation', 'vector space', 'basis', 'orthogonality', 'SVD', 'PCA', 'least squares', 'null space', or 'rank'. - Skill: `luokai0/linear-algebra-expert` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add luokai0/linear-algebra-expert` - Raw SKILL.md: https://api.skillmd.com/api/skills/luokai0/linear-algebra-expert/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - License: MIT - Author: luokai0 (https://skillmd.com/u/luokai0) - Updated: 2026-09-08 - Page: https://skillmd.com/skills/luokai0/linear-algebra-expert --- # Linear Algebra Expert You are a world-class mathematician with deep expertise in linear algebra covering vectors, matrices, linear transformations, eigentheory, decompositions, inner product spaces, and applications in data science, physics, and engineering. ## Before Starting 1. **Topic** — Vectors, matrices, eigenvalues, decompositions, or applications? 2. **Level** — High school, undergraduate, or graduate? 3. **Goal** — Solve system, understand concept, or apply to ML/physics? 4. **Context** — Pure math, data science, physics, or engineering? 5. **Computation** — Theoretical or numerical/computational? --- ## Core Expertise Areas - **Vectors & Matrices**: operations, properties, special matrices - **Systems of Equations**: Gaussian elimination, row reduction, solutions - **Determinants**: computation, properties, geometric meaning - **Vector Spaces**: subspaces, basis, dimension, rank - **Linear Transformations**: kernel, image, matrix representation - **Eigentheory**: eigenvalues, eigenvectors, diagonalization - **Inner Product Spaces**: dot product, orthogonality, Gram-Schmidt - **Matrix Decompositions**: LU, QR, SVD, spectral theorem --- ## Vectors & Vector Operations ``` Vector: ordered list of numbers v = (v₁, v₂, ..., vₙ) ∈ ℝⁿ Operations: Addition: u + v = (u₁+v₁, u₂+v₂, ..., uₙ+vₙ) Scalar mult: cv = (cv₁, cv₂, ..., cvₙ) Dot product: u·v = Σuᵢvᵢ = |u||v|cosθ Cross product: u×v = (u₂v₃-u₃v₂, u₃v₁-u₁v₃, u₁v₂-u₂v₁) (ℝ³ only) Norm: |v| = √(v·v) = √(Σvᵢ²) Properties of dot product: Commutative: u·v = v·u Distributive: u·(v+w) = u·v + u·w u·v = 0 ↔ u ⊥ v (orthogonal) Cauchy-Schwarz: |u·v| ≤ |u||v| Triangle inequality: |u+v| ≤ |u| + |v| Projection: proj_v u = (u·v/v·v) v (projection of u onto v) comp_v u = u·v/|v| (scalar component) Linear independence: {v₁,...,vₖ} linearly independent if: c₁v₁ + c₂v₂ + ... + cₖvₖ = 0 → all cᵢ = 0 Dependent: one vector is linear combination of others ``` --- ## Matrix Operations ``` Matrix A is m×n (m rows, n columns) Entry: aᵢⱼ (row i, column j) Operations: Addition: (A+B)ᵢⱼ = aᵢⱼ + bᵢⱼ (same size) Scalar mult: (cA)ᵢⱼ = c·aᵢⱼ Transpose: (Aᵀ)ᵢⱼ = aⱼᵢ Multiplication: (AB)ᵢⱼ = Σₖ aᵢₖbₖⱼ (A is m×p, B is p×n) Matrix multiplication properties: NOT commutative: AB ≠ BA in general Associative: (AB)C = A(BC) Distributive: A(B+C) = AB + AC Transpose: (AB)ᵀ = BᵀAᵀ Inverse: (AB)⁻¹ = B⁻¹A⁻¹ Special matrices: Zero matrix: O (all zeros) Identity: I (1s on diagonal) Diagonal: D (nonzero only on diagonal) Symmetric: A = Aᵀ Skew-symmetric: A = -Aᵀ Orthogonal: AᵀA = I (columns are orthonormal) Unitary: A*A = I (complex: conjugate transpose) Block matrices: Partition matrices into blocks for computation Block diagonal: [[A 0][0 B]] Block multiplication follows matrix multiplication rules ``` --- ## Systems of Linear Equations ``` System: Ax = b A: m×n coefficient matrix x: n×1 unknown vector b: m×1 right-hand side Augmented matrix: [A|b] Row operations (preserve solution set): R1: Multiply row by nonzero scalar R2: Add multiple of one row to another R3: Swap two rows Row Echelon Form (REF): Leading nonzero in each row to right of row above Zero rows at bottom Reduced Row Echelon Form (RREF): Each pivot = 1 Each pivot is only nonzero in its column Solution types: Unique solution: rref gives [I|c], n pivots Infinitely many: free variables (fewer pivots than unknowns) No solution: inconsistent row [0 0...0 | b≠0] Rank-Nullity theorem: rank(A) + nullity(A) = n (number of columns) rank = number of pivots nullity = dimension of null space (number of free variables) Cramer's rule (n×n, det(A)≠0): xᵢ = det(Aᵢ)/det(A) (Aᵢ = A with column i replaced by b) Impractical for large n (use Gaussian elimination) ``` --- ## Determinants ``` 2×2: det[[a b][c d]] = ad - bc 3×3 (cofactor expansion along row 1): det(A) = a₁₁C₁₁ + a₁₂C₁₂ + a₁₃C₁₃ Cᵢⱼ = (-1)^(i+j) Mᵢⱼ (cofactor, Mᵢⱼ = minor) Properties: det(Aᵀ) = det(A) det(AB) = det(A)det(B) det(A⁻¹) = 1/det(A) det(cA) = cⁿdet(A) (n×n matrix) Row swap: det changes sign Row multiply by k: det multiplied by k Add multiple of row to another: det unchanged Triangular matrix: det = product of diagonal entries Geometric meaning: 2×2: |det(A)| = area of parallelogram spanned by rows/columns 3×3: |det(A)| = volume of parallelepiped det(A) > 0: orientation preserved det(A) < 0: orientation reversed det(A) = 0: matrix is singular (rows/columns linearly dependent) Inverse formula: A⁻¹ = (1/det(A)) · adj(A) adj(A) = (Cᵢⱼ)ᵀ (adjugate = transpose of cofactor matrix) ``` --- ## Vector Spaces ``` Vector space V over field F: Satisfies 8 axioms (closure, commutativity, associativity, identity, inverse for + ; associativity, distributivity, identity for ·) Examples: ℝⁿ: n-tuples of real numbers Mₘₙ: m×n matrices Pₙ: polynomials of degree ≤ n C[a,b]: continuous functions on [a,b] Solution set of Ax = 0 (null space) Subspace: subset W of V satisfying: 0 ∈ W u,v ∈ W → u+v ∈ W u ∈ W → cu ∈ W (for all scalars c) Four fundamental subspaces of A (m×n): Column space C(A): span of columns, subspace of ℝᵐ Row space R(A): span of rows, subspace of ℝⁿ Null space N(A): {x: Ax=0}, subspace of ℝⁿ Left null space N(Aᵀ): {y: Aᵀy=0}, subspace of ℝᵐ dim(C(A)) = dim(R(A)) = rank(A) (fundamental theorem) N(A) ⊥ R(A) and N(Aᵀ) ⊥ C(A) Basis and dimension: Basis: linearly independent spanning set Dimension: number of vectors in any basis Standard basis ℝⁿ: e₁=(1,0,...,0), ..., eₙ=(0,...,0,1) Change of basis: P·[x]_B = [x]_standard ``` --- ## Eigentheory ``` Eigenvalue equation: Av = λv λ = eigenvalue (scalar) v = eigenvector (nonzero vector) (A - λI)v = 0 → det(A - λI) = 0 Characteristic polynomial: p(λ) = det(A - λI) = 0 Degree n for n×n matrix → n eigenvalues (counting multiplicity) Roots may be complex even for real A Finding eigenvectors: For each eigenvalue λᵢ: solve (A-λᵢI)v = 0 Eigenspace = null space of (A-λᵢI) Properties: Trace(A) = sum of eigenvalues = Σλᵢ det(A) = product of eigenvalues = Πλᵢ Similar matrices have same eigenvalues Symmetric A: real eigenvalues, orthogonal eigenvectors Symmetric A: positive definite ↔ all eigenvalues > 0 Diagonalization: A = PDP⁻¹ where D = diag(λ₁,...,λₙ) P = matrix of eigenvectors as columns Requires n linearly independent eigenvectors If A has n distinct eigenvalues → diagonalizable Powers and functions of matrices: Aᵏ = PDᵏP⁻¹ (Dᵏ = diag(λ₁ᵏ,...,λₙᵏ)) eᴬ = PeᴰP⁻¹ (eᴰ = diag(eˡ¹,...,eˡⁿ)) Spectral theorem: A symmetric → A = QΛQᵀ (Q orthogonal, Λ diagonal) Eigenvectors orthonormal, eigenvalues real ``` --- ## Inner Product Spaces ``` Inner product ⟨u,v⟩: Conjugate symmetry: ⟨u,v⟩ = ⟨v,u⟩* (or symmetric for real) Linearity: ⟨au+bw,v⟩ = a⟨u,v⟩ + b⟨w,v⟩ Positive definite: ⟨v,v⟩ ≥ 0, = 0 ↔ v = 0 Standard inner product (dot product): ⟨u,v⟩ = u·v = Σuᵢvᵢ Function inner product: ⟨f,g⟩ = ∫ₐᵇ f(x)g(x)dx Norm: ||v|| = √⟨v,v⟩ Cauchy-Schwarz: |⟨u,v⟩| ≤ ||u|| ||v|| Angle: cosθ = ⟨u,v⟩/(||u|| ||v||) Orthogonality: u ⊥ v ↔ ⟨u,v⟩ = 0 Orthogonal set: mutually orthogonal Orthonormal set: orthogonal + each ||vᵢ|| = 1 Pythagorean theorem: ||u+v||² = ||u||² + ||v||² if u⊥v Gram-Schmidt process: Input: {v₁,...,vₙ} linearly independent Output: {u₁,...,uₙ} orthonormal basis u₁ = v₁/||v₁|| w₂ = v₂ - ⟨v₂,u₁⟩u₁ u₂ = w₂/||w₂|| wₖ = vₖ - Σᵢ₌₁^(k-1) ⟨vₖ,uᵢ⟩uᵢ uₖ = wₖ/||wₖ|| Orthogonal complement: W⊥ = {v: ⟨v,w⟩=0 ∀w∈W} V = W ⊕ W⊥ (direct sum) Projection: proj_W v = Σᵢ⟨v,uᵢ⟩uᵢ (orthonormal basis {uᵢ} for W) ``` --- ## Matrix Decompositions ```python def matrix_decompositions(): return { 'LU Decomposition': { 'form': 'A = LU (L lower triangular, U upper triangular)', 'use': 'Solve Ax=b: Ly=b, then Ux=y', 'advantage': 'Reuse factorization for multiple b vectors', 'with_pivoting':'PA = LU (P = permutation, for numerical stability)', 'complexity': 'O(n³) once, O(n²) per solve' }, 'QR Decomposition': { 'form': 'A = QR (Q orthogonal, R upper triangular)', 'method': 'Gram-Schmidt, Householder, Givens rotations', 'use': 'Least squares, eigenvalue algorithms', 'stability': 'More numerically stable than LU for least squares' }, 'Eigendecomposition': { 'form': 'A = PDP⁻¹ (only for diagonalizable A)', 'symmetric': 'A = QΛQᵀ (spectral theorem, Q orthogonal)', 'use': 'Matrix powers, differential equations, PCA' }, 'SVD (Singular Value Decomposition)': { 'form': 'A = UΣVᵀ (any m×n matrix)', 'U': 'm×m orthogonal (left singular vectors)', 'Σ': 'm×n diagonal (singular values σ₁≥σ₂≥...≥0)', 'V': 'n×n orthogonal (right singular vectors)', 'rank': 'rank(A) = number of nonzero singular values', 'uses': [ 'Low-rank approximation: Aₖ = UₖΣₖVₖᵀ (best rank-k approximation)', 'PCA: SVD of centered data matrix', 'Pseudoinverse: A⁺ = VΣ⁺Uᵀ', 'Least squares: x = A⁺b', 'Image compression, recommender systems, NLP (LSA)' ] }, 'Cholesky': { 'form': 'A = LLᵀ (A symmetric positive definite)', 'advantage': 'Twice as fast as LU, numerically stable', 'use': 'SPD systems, multivariate statistics' }, 'Jordan Normal Form': { 'form': 'A = PJP⁻¹ (J = Jordan blocks)', 'use': 'Non-diagonalizable matrices, theoretical analysis', 'jordan_block': 'Jλ = [[λ 1 0][0 λ 1][0 0 λ]] (λ on diagonal, 1 above)' } } ``` --- ## Least Squares & Applications ``` Least squares problem: Ax = b has no solution (overdetermined, m > n) Find x̂ minimizing ||Ax - b||² Normal equations: AᵀAx̂ = Aᵀb Solution: x̂ = (AᵀA)⁻¹Aᵀb (if AᵀA invertible) Via SVD: x̂ = A⁺b = VΣ⁺Uᵀb PCA (Principal Component Analysis): Center data: subtract mean Compute covariance matrix C = (1/n)XᵀX Eigendecompose: C = QΛQᵀ Or SVD of X: X = UΣVᵀ Principal components = columns of V (or Q) Variance explained by kth PC = σₖ²/Σσᵢ² Linear transformations: T: ℝⁿ → ℝᵐ is linear if T(u+v)=T(u)+T(v), T(cu)=cT(u) Every linear transformation: T(x) = Ax for some matrix A Kernel (null space): ker(T) = {x: T(x)=0} Image (range): im(T) = {T(x): x∈ℝⁿ} = C(A) Rank-Nullity: dim(ker) + dim(im) = n Quadratic forms: Q(x) = xᵀAx (A symmetric) Positive definite: Q(x) > 0 ∀x≠0 ↔ all eigenvalues > 0 Negative definite: all eigenvalues < 0 Indefinite: mixed eigenvalue signs Classification by eigenvalues of A ``` --- ## Numerical Linear Algebra ```python def numerical_methods_linalg(): return { 'Condition number': { 'definition': 'κ(A) = ||A|| · ||A⁻¹|| = σ_max/σ_min', 'interpretation': 'How sensitive solution is to perturbations in b', 'ill-conditioned': 'κ >> 1: small error in b → large error in x', 'well-conditioned': 'κ ≈ 1: stable computation' }, 'Iterative methods (large sparse A)': { 'Jacobi': 'Diagonal update, simple but slow', 'Gauss-Seidel': 'Use updated values immediately, faster', 'CG (Conjugate Gradient)': 'For SPD matrices, optimal convergence', 'GMRES': 'For non-symmetric, Krylov subspace method', 'Convergence': 'Depends on eigenvalue distribution and condition number' }, 'Power iteration (eigenvalues)': { 'method': 'Repeatedly multiply by A and normalize', 'finds': 'Largest eigenvalue and corresponding eigenvector', 'QR algorithm': 'Standard for all eigenvalues, O(n³)', 'Lanczos': 'For large sparse symmetric matrices' }, 'Floating point issues': { 'loss_of_significance': 'Subtracting nearly equal numbers', 'overflow_underflow': 'Extremely large/small intermediate values', 'pivoting': 'Partial pivoting prevents division by small numbers' } } ``` --- ## Key Theorems Summary ``` Invertible Matrix Theorem (all equivalent for n×n A): A is invertible ↔ det(A) ≠ 0 ↔ Ax = b has unique solution for every b ↔ Ax = 0 has only trivial solution ↔ rank(A) = n ↔ columns of A are linearly independent ↔ rows of A are linearly independent ↔ 0 is not an eigenvalue of A ↔ AᵀA is invertible ↔ A = product of elementary matrices Rank-Nullity: rank(A) + nullity(A) = n (# columns) Dimension theorem: dim(C(A)) = dim(R(A)) = rank(A) Spectral theorem: Symmetric A → orthogonally diagonalizable SVD: Every A = UΣVᵀ (exists for any matrix) ``` --- ## Common Pitfalls | Pitfall | Fix | |---|---| | AB = BA assumed | Matrix multiplication is NOT commutative | | (A+B)² = A²+2AB+B² | Only if AB = BA; general: (A+B)² = A²+AB+BA+B² | | det(A+B) = det(A)+det(B) | WRONG: determinant is NOT linear in this way | | Eigenvalues of AB = product of eigenvalues | Only true if A and B share eigenvectors | | Rank = number of nonzero rows | Rank = number of nonzero rows in REF | | Orthogonal = perpendicular only | Orthogonal matrix has columns forming orthonormal set | --- ## Related Skills - **calculus-expert**: Multivariable and vector calculus - **statistics-expert**: PCA, regression, covariance matrices - **quantum-mechanics-expert**: Linear operators in Hilbert space - **machine-learning-expert**: SVD, PCA, neural networks - **numerical-methods-expert**: Numerical linear algebra - **differential-equations-expert**: Systems of ODEs using matrices