第19讲 矩阵特征值问题和奇异值分解.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第19讲 矩阵特征值问题和奇异值分解"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 矩阵的特征值问题\n",
    "\n",
    "对于n阶方阵A，如果存在列向量Vec[i]和标量l[i]，使得\n",
    "\n",
    "A.Vec[i]=l[i]*Vec[i],   i=0,1,...n-1\n",
    "\n",
    "即在n阶方阵变换下方向保持不变、长度可变的向量称为该方阵的特征向量。\n",
    "\n",
    "称Vec[i]是矩阵A的第i个特征向量，称 l[i]是关联特征向量Vec[i]的特征值（即特征向量的放大因子）。\n",
    "\n",
    "寻找矩阵全部特征值及其对应特征向量的问题称为特征值问题（或本征值问题）。特征方程还可以写成\n",
    "\n",
    "     A.V=V.diag(l)  \n",
    "     \n",
    "  即   A.[Vec[0],Vec[1],...Vec[n-1]]=[Vec[0],Vec[1],...Vec[n-1]].diag(l)\n",
    "  \n",
    "其中矩阵V的第i列是矩阵A的第i个特征向量Vec[i]，对应l[i]。\n",
    "方阵diag(l)主对角线上依次是特征值l[0],l[1]...l[n-1]，diag(l)的主对角线以外的元素全为0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 矩阵特征值及其特征向量求解eig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 2  1  5]\n",
      " [ 3 -4  7]\n",
      " [11 -7 13]]\n",
      "147.0\n",
      "lamda= [14.627+0.j  -1.813+2.6j -1.813-2.6j]\n",
      "vec[0],vec[1],vec[2]= [[ 0.367+0.j     0.613+0.j     0.613-0.j   ]\n",
      " [ 0.378+0.j     0.034+0.598j  0.034-0.598j]\n",
      " [ 0.85 +0.j    -0.475+0.199j -0.475-0.199j]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np; # b.vec[i]=lamda[i]*vec[i]; \n",
    "\n",
    "b=np.array([[2,1,5],[3,-4,7],[11,-7,13]]);\n",
    "print(b)\n",
    "print(np.linalg.det(b).round(3))  #计算行列值\n",
    "\n",
    "val,vec=np.linalg.eig(b);  #矩阵特征值和特征向量\n",
    "\n",
    "print(\"lamda=\",val.round(3));\n",
    "\n",
    "print(\"vec[0],vec[1],vec[2]=\",vec.round(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0\n",
      "0.0,0.0,0.0,"
     ]
    }
   ],
   "source": [
    "D=np.diag(val);\n",
    "er1=np.dot(b,vec)-np.dot(vec,D);\n",
    "er=np.abs(er1).max();\n",
    "print(er.round(6))\n",
    "\n",
    "for i in range(3):\n",
    "    er2=np.dot(b,vec[:,i])-val[i]*vec[:,i];\n",
    "    er=np.abs(er2).max();\n",
    "    print(er.round(6),end=\",\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 对称方阵与反对称方阵特征向量的特点研究\n",
    "\n",
    "*  对称方阵的特征向量集合是正交归一集合。特征值为实数。\n",
    "* 反对称方阵的特征向量集合是归一集合，但不是正交集合。特征值为纯虚数。\n",
    "* 一般方阵的特征向量集合既不是正交的，也不是归一的。特征值为复数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 2.  2.  8.]\n",
      " [ 2. -4.  0.]\n",
      " [ 8.  0. 13.]]\n",
      "lamda= [17.249 -1.133 -5.116]\n",
      "v[0],v[1],v[2]=\n",
      " [[-0.469 -0.744  0.476]\n",
      " [-0.044 -0.519 -0.854]\n",
      " [-0.882  0.421 -0.21 ]]\n"
     ]
    }
   ],
   "source": [
    "bdc=(b+b.T)/2\n",
    "print(bdc)\n",
    "val,vec=np.linalg.eig(bdc); \n",
    "print(\"lamda=\",val.round(3));\n",
    "\n",
    "print(\"v[0],v[1],v[2]=\\n\",vec.round(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一般方阵的特征向量集合既不是正交的，也不是归一的。特征值为复数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a=\n",
      " [[  1   2   3   4]\n",
      " [  5   6   7   8]\n",
      " [  7  -5   3  -2]\n",
      " [ -3  11 -12  50]]\n",
      "det(a)= -4875.9999999999945\n",
      "l=\n",
      " [52.2371+0.j      4.9055+4.6385j  4.9055-4.6385j -2.048 +0.j    ]\n",
      "l.imag=\n",
      " [ 0.        4.638474 -4.638474  0.      ]\n",
      "v=\n",
      " [[-0.0806+0.j     -0.2592-0.0865j -0.2592+0.0865j  0.6569+0.j    ]\n",
      " [-0.1715+0.j     -0.8425+0.j     -0.8425-0.j      0.3381+0.j    ]\n",
      " [ 0.0458+0.j      0.07  -0.3951j  0.07  +0.3951j -0.6486+0.j    ]\n",
      " [-0.9808+0.j      0.216 -0.0887j  0.216 +0.0887j -0.1831+0.j    ]]\n",
      "v.T.dot(v)=\n",
      " [[ 1.    +0.j     -0.0433+0.0759j -0.0433-0.0759j  0.039 +0.j    ]\n",
      " [-0.0433+0.0759j  0.6571-0.0488j  1.    +0.j     -0.54  +0.2157j]\n",
      " [-0.0433-0.0759j  1.    +0.j      0.6571+0.0488j -0.54  -0.2157j]\n",
      " [ 0.039 +0.j     -0.54  +0.2157j -0.54  -0.2157j  1.    +0.j    ]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np \n",
    "ar=np.array;\n",
    "nll=np.linalg.linalg\n",
    "\n",
    "a=ar([[1,2,3,4],[5,6,7,8],[7,-5,3,-2],[-3,11,-12,50]]);\n",
    "\n",
    "print('a=\\n',a); \n",
    "\n",
    "print('det(a)=', nll.det(a))\n",
    "\n",
    "l,v=nll.eig(a)\n",
    "\n",
    "print('l=\\n',l.round(4)); \n",
    "print('l.imag=\\n',l.imag)\n",
    "print('v=\\n',v. round(4))\n",
    "\n",
    "vtv=v.T.dot(v). round(4)\n",
    "print('v.T.dot(v)=\\n', vtv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 矩阵函数f(A)=V.f(D).VI\n",
    "满秩方阵存在特征值问题。n阶方阵有n个特征值和n个特征向量。满秩n阶方阵可以分解为\n",
    "        \n",
    "        A=V.D.VI,  Vi=np.linalg.linalg.inv(V),  D=np.diag(l)\n",
    "        \n",
    "其中VI是V的逆矩阵，即V.VI=VI.V=I(单位矩阵),D是特征值l构造的对角矩阵。特别矩阵A的p次幂为：\n",
    "\n",
    "        Ap=V.Dp.VI      以至于    f(A)=V.f(D).VI=V.diag(f(l)).VI\n",
    "        \n",
    "其中D^p是特征值的p次幂构成的对角矩阵。矩阵A的函数f等于矩阵特征值对角阵的函数diag(f(l))再左乘V右乘VI。   \n",
    "\n",
    "利用这一特性，可以计算出以方阵为自变量的各类矩阵函数，前提是，矩阵是满秩方阵，函数可以展开成收敛幂级数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 矩阵奇异值分解svd\n",
    "\n",
    "不满秩的方阵称为奇异矩阵。奇异矩阵也可以分解为\n",
    "\n",
    "         A=S.diag(v).D   s-1.A.d-1=diag(v)\n",
    "         \n",
    "v是矩阵的奇异值，S和D与A矩阵的阶数相等。注意s与d的点积S.D不等于单位矩阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s=\n",
      " [[-2.54678171e-01  1.87454073e-01 -9.48683298e-01]\n",
      " [-5.92781826e-01 -8.05363090e-01  4.45954139e-16]\n",
      " [-7.64034512e-01  5.62362218e-01  3.16227766e-01]]\n",
      "v=\n",
      " [1.46452146e+01 1.58672309e+00 9.74960837e-16]\n",
      "d=\n",
      " [[-0.33580314 -0.55017784 -0.76455255]\n",
      " [-0.8488637  -0.17503621  0.49879128]\n",
      " [-0.40824829  0.81649658 -0.40824829]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np;\n",
    "\n",
    "A=np.array([[1,2,3],[4,5,6],[3,6,9]])\n",
    "\n",
    "s,v,d=np.linalg.linalg.svd(A);\n",
    "\n",
    "print('s=\\n',s);\n",
    "print('v=\\n',v);\n",
    "print('d=\\n',d)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 点积应用一例：向量集合的正交归一化\n",
    "\n",
    "线性独立向量集合v=[v0,v1,...vn-1]转化为正交归一向量集合u的算法为：\n",
    "\n",
    "\n",
    "其中(uj*.vk)=dot(uk*,vk)是两向量的点积运算。\n",
    "\n",
    "正交归一化需要的Python程序orth.py内容如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    def orth(v):\n",
    "    \"\"\"\n",
    "    v is a nonorthogonal set of vectors.\n",
    "    v is a square matrix.\n",
    "    Schmidt projection algorithm is used\n",
    "       to normalize v into vector set u. \n",
    "    \"\"\"\n",
    "    import numpy as np;\n",
    "    \n",
    "    u=v.copy()\n",
    "    r,n=v.shape\n",
    "    \n",
    "    us0[:,0].conjugate().dot(u[:,0]).real\n",
    "    \n",
    "    u[:,0]/=np.sqrt(us0)\n",
    "    \n",
    "    for i in range(1,n):\n",
    "            for j in range(i):\n",
    "                uv=u[:,j].conjugate().dot(v[:,i]); \n",
    "                u[:,i]-=uv*u[:,j]\n",
    "            \n",
    "            uin=u[:,i].conjugate().dot(u[:,i]).real\n",
    "            \n",
    "            u[:,i]/=np.sqrt(uin)\n",
    "    \n",
    "    udu=u.conjugate().T.dot(u)\n",
    "    print('uTu=\\n',udu.round(4))\n",
    "    return u\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}