From 2be52c86b1ba7663df8a05cc47f822ab5b5ef1e3 Mon Sep 17 00:00:00 2001 From: Razvan Deaconescu Date: Wed, 13 Sep 2023 05:45:18 +0300 Subject: [PATCH] Add content for vectors-matrix-ops/multivariate-calculus Add actual content to the skeleton of the `vectors-matrix-ops/multivariate-calculus` section. Signed-off-by: Eggert Karl Hafsteinsson Signed-off-by: Teodor Dutu Signed-off-by: Razvan Deaconescu --- .../multivariate-calculus/reading/README.md | 237 ++++++++++++++++++ 1 file changed, 237 insertions(+) diff --git a/chapters/vectors-matrix-ops/multivariate-calculus/reading/README.md b/chapters/vectors-matrix-ops/multivariate-calculus/reading/README.md index c9fcd23..01683fe 100644 --- a/chapters/vectors-matrix-ops/multivariate-calculus/reading/README.md +++ b/chapters/vectors-matrix-ops/multivariate-calculus/reading/README.md @@ -1 +1,238 @@ # Multivariate Calculus + +## Vector Functions of Several Variables + +A vector-valued function of several variables is a function + +$$f: \mathbb{R}^{m} \rightarrow \mathbb{R}^{n}$$ + +i.e. a function of $m$ -dimensional vectors, which returns $n$ dimensional vectors. + +### Examples + +:::info Example + +A real valued function of many variables: $f: \mathbb{R}^3\to\mathbb{R}$, $f(x_1,x_2,x_3)=2x_1+3x_2+4x_3$. + +::: + +:::note Note + +$f$ is linear and $f(x)=Ax$ where + +$$x=\begin{pmatrix} \\ x_1 \\ x_2 \\ x_3\end{pmatrix}$$ + +and + +$$A=\begin{bmatrix}2&3&4\end{bmatrix}$$ + +::: + +:::info Example + +Let + +$$f: \mathbb{R}^{2} \rightarrow \mathbb{R}^{2}$$ + +where + +$$f(x_1,x_2) = \left( \begin{array}{c} x_1+x_2 \\ x_1-x_2 \end{array} \right)$$ + +::: + +:::note Note + +Note that $f(x)=Ax$, where + +$$A=\begin{bmatrix} \\ 1&1 \\ 1&-1 \\ \end{bmatrix}$$ + +::: + +:::info Example + +Let + +$$f: \mathbb{R}^{3} \rightarrow \mathbb{R}^{4}$$ + +be defined by + +$$f(x) = \left( \begin{array}{c} x_1+x_2 \\ x_1-x_3 \\ y-z \\ x_1+x_2+x_3 \end{array} \right)$$ + +::: + +:::Note note + +$$f(x) = Ax$$ where $$A = \begin{bmatrix} 1 & 1 & 0\\ 1 & 0 & -1\\ 0 & 1 & -1\\ 1 & 1 & 1 \end{bmatrix}$$ + +::: + +:::info Example + +These multi-dimensional functions do not have to be linear, for example the function $f:\mathbb{R}^2\to\mathbb{R}^2$ + +$$f(x) = \left( \begin{array}{c} x_1x_2 \\ x_1^{2}+x_2^{2} \end{array} \right),$$ + +is obviously not linear. + +::: + +## The Gradient + +Suppose the real valued function $f:\mathbb{R}^m \rightarrow \mathbb{R}$ is differentiable in each coordinate. +Then the gradient of $f$, denoted $\nabla f$ is given by + +$$\nabla f(x)=\begin{pmatrix}\frac{\partial f}{\partial x_1},&\dots &,\frac{\partial f}{\partial x_1}\end{pmatrix}$$ + +### Details + +:::note Definition + +Suppose the real valued function $f:\mathbb{R}^m \rightarrow \mathbb{R}$ is differentiable in each coordinate. +Then the **gradient** of $f$, denoted $\nabla f$ is given by + +$$\nabla f(x)= \begin{pmatrix} \frac{\partial f}{\partial x_1},&\dots &,\frac{\partial f}{\partial x_1}\end{pmatrix}$$ + +where each partial derivative $\frac{\partial f}{\partial x_i}$ is computed by differentiating $f$ with respect to that variable, regarding the others as fixed. + +::: + +### Examples + +:::info Example + +Let + +$$f(\underline{x})= x^2+y^2+2xy.$$ + +Then the partial derivatives of $f$ are + +$$\frac{\partial f}{\partial x}=2x+2y$$ + +and + +$$\frac{\partial f}{\partial y}=2y+2x$$ + +and the gradient of $f$ is therefore + +$$\nabla f =\begin{pmatrix}2x+2y, & 2y+2x\end{pmatrix}$$ + +::: + +:::info Example + +Let + +$$f(\underline{x})=x_1-x_2$$ + +The gradient of $f$ is + +$$\nabla f= \begin{pmatrix}1, & -1\end{pmatrix}$$ + +::: + +## The Jacobian + +Now consider a function $f:\mathbb{R}^m\to\mathbb{R}^n$. +Write $f_i$ for the $i^{th}$ coordinate of $f$, so we can write $f(x)=(f_1(x),f_2(x),\ldots,f_n(x))$, where $x\in\mathbb{R}^m$. +If each coordinate function $f_i$ is differentiable in each variable we can form the *Jacobian matrix* of $f$: + +$$\begin{pmatrix}\nabla f_1\\ \vdots\\ \nabla f_n \end{pmatrix}$$ + +### Details + +Now consider a function $f:\mathbb{R}^m\to\mathbb{R}^n$. +Write $f_i$ for the $i^{th}$ coordinate of $f$, so we can write $f(x)=(f_1(x),f_2(x),\ldots,f_n(x))$, where $x\in\mathbb{R}^m$. +If each coordinate function $f_i$ is differentiable in each variable we can form the *Jacobian matrix* of $f$: + +$$\begin{pmatrix}\nabla f_1\\ \vdots\\ \nabla f_n \end{pmatrix}$$ + +In this matrix, the element in the $i^{th}$ row and $j^{th}$ column is $\frac{\partial f_i}{\partial x_j}$. + +### Examples + +:::info Example + +For the function + +$$f(x,y)=\begin{pmatrix} x^2 +y \\ x y \\ x \end{pmatrix}= \begin{pmatrix} f_1(x,y) \\ f_2(x,y) \\ f_3(x,y) \end{pmatrix}$$ + +the Jacobian matrix of $f$ is the matrix + +$$J= \begin{bmatrix} \nabla f_1 \\ \nabla f_2 \\ \nabla f_3 \end{bmatrix}= \begin{bmatrix} 2x & 2y \\ y & x \\ 1 & 0 \end{bmatrix}$$ + +::: + +## Univariate Integration By Substitution + +If $f$ is a continuous function and $g$ is strictly increasing and differentiable then, + +$$\int_{g(a)}^{g(b)} f(x)dx = \int_a^b f(g(t))g^\prime (t)dt.$$ + +### Details + +If $f$ is a continuous function and $g$ is strictly increasing and differentiable then, + +$$\int_{g(a)}^{g(b)} f(x)dx = \int_a^b f(g(t))g^\prime (t)dt$$ + +It follows that if $X$ is a continuous random variable with density $f$ + +and $Y = h(X)$ is a function of $X$ that has the inverse $g=h^{-1}$, so $X = g(Y)$, then the density of $Y$ is given by, + +$$f_Y(y) = f (g(y)) g^\prime (y)$$ + +This is a consequence of + +$$P [Y \leq b] = P [g(Y) \leq g(b)] = P [X \leq g(b)] = \int_{- \infty} ^{g(b)}f(x)dx = \int_{- \infty} ^b f (g(y))g^\prime (y)dy$$ + +## Multivariate Integration By Substitution + +Suppose $f$ is a continuous function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ and $g: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is a one-to-one function with continuous partial derivatives. +Then if $U \subseteq \mathbb{R}^n$ is a subset, + +$$\int_{g(U)} f(\mathbf {x})d\mathbf {x} = \int_{U}({g}(\mathbf {y}))|J|d\mathbf {y}$$ + +where $J$ is the Jacobian matrix and $|J|$ is the absolute value of it's determinant. + +$$J= \left|\begin{bmatrix} \frac{\partial g_1}{\partial y_1} & \frac{\partial g_1}{\partial y_2} & \cdots &\frac{\partial g_1}{\partial y_n} \\ \vdots & \vdots & \cdots & \vdots \\ \frac{\partial g_n}{\partial y_1} & \frac{\partial g_n}{\partial y_2} & \cdots & \frac{\partial g_n}{\partial y_n} \end{bmatrix}\right| = \left|\begin{bmatrix} \nabla g_1 \\ \vdots \\ \nabla g_n \end{bmatrix}\right|$$ + +### Details + +Suppose $f$ is a continuous function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ and $g: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is a one-to-one function with continuous partial derivatives. +Then if $U \subseteq \mathbb{R}^n$ is a subset, + +$$\int_{g(U)} f(\mathbf {x})d\mathbf {x} = \int_{U}({g}(\mathbf {y}))|J|d\mathbf {y}$$ + +where $J$ is the Jacobian determinant and \|J\| is its absolute value. + +$$J= \left|\begin{bmatrix} \frac{\partial g_1}{\partial y_1} & \frac{\partial g_1}{\partial y_2} & \cdots &\frac{\partial g_1}{\partial y_n} \\ \vdots & \vdots & \cdots & \vdots \\ \frac{\partial g_n}{\partial y_1} & \frac{\partial g_n}{\partial y_2} & \cdots & \frac{\partial g_n}{\partial y_n} \end{bmatrix}\right| = \left|\begin{bmatrix} \nabla g_1 \\ \vdots \\ \nabla g_n \end{bmatrix}\right|$$ + +Similar calculations as in 28.4 give us that if $X$ is a continuous multivariate random variable, $X = (X_1, \ldots, X_n)^\prime$ with density $f$ and $\mathbf{Y} = \mathbf{h} (\mathbf{X})$, where $\mathbf{h}$ is one-to-one with inverse $\mathbf g= \mathbf{h}^{-1}$. +So, $\mathbf{X} = g(\mathbf{Y})$, then the density of $\mathbf{Y}$ is given by; + +$$f_Y(\mathbf y) = f (g(\mathbf y)) |J|$$ + +### Examples + +:::info Example + +If $\mathbf{Y} = A \mathbf X$ where $A$ is an $n \times n$ matrix with $\det(A)\neq0$ and $X = (X_1, \ldots, X_n)^\prime$ are independent and identically distributed random variables, then we have the following results. + +The joint density of $X_1 \cdots X_n$ is the product of the individual (marginal) densities, + +$$f_X(\mathbf x)= f(x_1) f(x_2) \cdots f(x_n)$$ + +The matrix of partial derivatives corresponds to $\frac{\partial g}{\partial y}$ where $\mathbf X = \mathbf g(\mathbf{Y})$, i.e. these are the derivatives of the transformation: $\mathbf X = g (\mathbf{Y}) = A^{-1}\mathbf{Y}$, or $\mathbf X = B \mathbf{Y}$ where $B = A^{-1}$ + +But if $\mathbf X = B \mathbf{Y}$, then + +$$X_i = b_{i1}y_1 + b_{i2}y_2 + \cdots b_{ij}y_j\cdots b_{in}y_n$$ + +So, $\frac{\partial x_i}{\partial y_j} = b_{ij}$ and thus, + +$$J =\left|\frac{\partial d\mathbf x}{\partial d\mathbf y}\right| = |B| = |A^{-1}| = \frac {1}{|A|}$$ + +The density of $\mathbf{Y}$ is therefore; + +$$f_Y(\mathbf{y}) = f_X(g(\mathbf{y})) |J| = f_X(A^{-1}\mathbf{y}) |A^{-1}|$$ + +:::