Add content for vectors-matrix-ops/multivariate-calculus

Add actual content to the skeleton of the
`vectors-matrix-ops/multivariate-calculus` section.

Signed-off-by: Eggert Karl Hafsteinsson <[email protected]>
Signed-off-by: Teodor Dutu <[email protected]>
Signed-off-by: Razvan Deaconescu <[email protected]>

chapters/vectors-matrix-ops/multivariate-calculus/reading/README.md

@@ -1 +1,238 @@
 # Multivariate Calculus
+
+## Vector Functions of Several Variables
+
+A vector-valued function of several variables is a function
+
+$$f: \mathbb{R}^{m} \rightarrow \mathbb{R}^{n}$$
+
+i.e. a function of $m$ -dimensional vectors, which returns $n$ dimensional vectors.
+
+### Examples
+
+:::info Example
+
+A real valued function of many variables: $f: \mathbb{R}^3\to\mathbb{R}$, $f(x_1,x_2,x_3)=2x_1+3x_2+4x_3$.
+
+:::
+
+:::note Note
+
+$f$ is linear and $f(x)=Ax$ where
+
+$$x=\begin{pmatrix} \\ x_1 \\ x_2 \\ x_3\end{pmatrix}$$
+
+and
+
+$$A=\begin{bmatrix}2&3&4\end{bmatrix}$$
+
+:::
+
+:::info Example
+
+Let
+
+$$f: \mathbb{R}^{2} \rightarrow \mathbb{R}^{2}$$
+
+where
+
+$$f(x_1,x_2) = \left( \begin{array}{c} x_1+x_2 \\ x_1-x_2 \end{array} \right)$$
+
+:::
+
+:::note Note
+
+Note that $f(x)=Ax$, where
+
+$$A=\begin{bmatrix} \\ 1&1 \\ 1&-1 \\ \end{bmatrix}$$
+
+:::
+
+:::info Example
+
+Let
+
+$$f: \mathbb{R}^{3} \rightarrow \mathbb{R}^{4}$$
+
+be defined by
+
+$$f(x) = \left( \begin{array}{c} x_1+x_2 \\ x_1-x_3 \\ y-z \\ x_1+x_2+x_3 \end{array} \right)$$
+
+:::
+
+:::Note note
+
+$$f(x) = Ax$$ where $$A = \begin{bmatrix} 1 & 1 & 0\\ 1 & 0 & -1\\ 0 & 1 & -1\\ 1 & 1 & 1 \end{bmatrix}$$
+
+:::
+
+:::info Example
+
+These multi-dimensional functions do not have to be linear, for example the function $f:\mathbb{R}^2\to\mathbb{R}^2$
+
+$$f(x) = \left( \begin{array}{c} x_1x_2 \\ x_1^{2}+x_2^{2} \end{array} \right),$$
+
+is obviously not linear.
+
+:::
+
+## The Gradient
+
+Suppose the real valued function $f:\mathbb{R}^m \rightarrow \mathbb{R}$ is differentiable in each coordinate.
+Then the gradient of $f$, denoted $\nabla f$ is given by
+
+$$\nabla f(x)=\begin{pmatrix}\frac{\partial f}{\partial x_1},&\dots &,\frac{\partial f}{\partial x_1}\end{pmatrix}$$
+
+### Details
+
+:::note Definition
+
+Suppose the real valued function $f:\mathbb{R}^m \rightarrow \mathbb{R}$ is differentiable in each coordinate.
+Then the **gradient** of $f$, denoted $\nabla f$ is given by
+
+$$\nabla f(x)= \begin{pmatrix} \frac{\partial f}{\partial x_1},&\dots &,\frac{\partial f}{\partial x_1}\end{pmatrix}$$
+
+where each partial derivative $\frac{\partial f}{\partial x_i}$ is computed by differentiating $f$ with respect to that variable, regarding the others as fixed.
+
+:::
+
+### Examples
+
+:::info Example
+
+Let
+
+$$f(\underline{x})= x^2+y^2+2xy.$$
+
+Then the partial derivatives of $f$ are
+
+$$\frac{\partial f}{\partial x}=2x+2y$$
+
+and
+
+$$\frac{\partial f}{\partial y}=2y+2x$$
+
+and the gradient of $f$ is therefore
+
+$$\nabla f =\begin{pmatrix}2x+2y, & 2y+2x\end{pmatrix}$$
+
+:::
+
+:::info Example
+
+Let
+
+$$f(\underline{x})=x_1-x_2$$
+
+The gradient of $f$ is
+
+$$\nabla f= \begin{pmatrix}1, & -1\end{pmatrix}$$
+
+:::
+
+## The Jacobian
+
+Now consider a function $f:\mathbb{R}^m\to\mathbb{R}^n$.
+Write $f_i$ for the $i^{th}$ coordinate of $f$, so we can write $f(x)=(f_1(x),f_2(x),\ldots,f_n(x))$, where $x\in\mathbb{R}^m$.
+If each coordinate function $f_i$ is differentiable in each variable we can form the *Jacobian matrix* of $f$:
+
+$$\begin{pmatrix}\nabla f_1\\ \vdots\\ \nabla f_n \end{pmatrix}$$
+
+### Details
+
+Now consider a function $f:\mathbb{R}^m\to\mathbb{R}^n$.
+Write $f_i$ for the $i^{th}$ coordinate of $f$, so we can write $f(x)=(f_1(x),f_2(x),\ldots,f_n(x))$, where $x\in\mathbb{R}^m$.
+If each coordinate function $f_i$ is differentiable in each variable we can form the *Jacobian matrix* of $f$:
+
+$$\begin{pmatrix}\nabla f_1\\ \vdots\\ \nabla f_n \end{pmatrix}$$
+
+In this matrix, the element in the $i^{th}$ row and $j^{th}$ column is $\frac{\partial f_i}{\partial x_j}$.
+
+### Examples
+
+:::info Example
+
+For the function
+
+$$f(x,y)=\begin{pmatrix} x^2 +y \\ x y \\ x \end{pmatrix}= \begin{pmatrix} f_1(x,y) \\ f_2(x,y) \\ f_3(x,y) \end{pmatrix}$$
+
+the Jacobian matrix of $f$ is the matrix
+
+$$J= \begin{bmatrix} \nabla f_1 \\ \nabla f_2 \\ \nabla f_3 \end{bmatrix}= \begin{bmatrix} 2x & 2y \\ y & x \\ 1 & 0 \end{bmatrix}$$
+
+:::
+
+## Univariate Integration By Substitution
+
+If $f$ is a continuous function and $g$ is strictly increasing and differentiable then,
+
+$$\int_{g(a)}^{g(b)} f(x)dx = \int_a^b f(g(t))g^\prime (t)dt.$$
+
+### Details
+
+If $f$ is a continuous function and $g$ is strictly increasing and differentiable then,
+
+$$\int_{g(a)}^{g(b)} f(x)dx = \int_a^b f(g(t))g^\prime (t)dt$$
+
+It follows that if $X$ is a continuous random variable with density $f$
+
+and $Y = h(X)$ is a function of $X$ that has the inverse $g=h^{-1}$, so $X = g(Y)$, then the density of $Y$ is given by,
+
+$$f_Y(y) = f (g(y)) g^\prime (y)$$
+
+This is a consequence of
+
+$$P [Y \leq b] = P [g(Y) \leq g(b)] = P [X \leq g(b)] = \int_{- \infty} ^{g(b)}f(x)dx = \int_{- \infty} ^b f (g(y))g^\prime (y)dy$$
+
+## Multivariate Integration By Substitution
+
+Suppose $f$ is a continuous function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ and $g: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is a one-to-one function with continuous partial derivatives.
+Then if $U \subseteq \mathbb{R}^n$ is a subset,
+
+$$\int_{g(U)} f(\mathbf {x})d\mathbf {x} = \int_{U}({g}(\mathbf {y}))|J|d\mathbf {y}$$
+
+where $J$ is the Jacobian matrix and $|J|$ is the absolute value of it's determinant.
+
+$$J= \left|\begin{bmatrix} \frac{\partial g_1}{\partial y_1} & \frac{\partial g_1}{\partial y_2} & \cdots &\frac{\partial g_1}{\partial y_n} \\ \vdots & \vdots & \cdots & \vdots \\ \frac{\partial g_n}{\partial y_1} & \frac{\partial g_n}{\partial y_2} & \cdots & \frac{\partial g_n}{\partial y_n} \end{bmatrix}\right| = \left|\begin{bmatrix} \nabla g_1 \\ \vdots \\ \nabla g_n \end{bmatrix}\right|$$
+
+### Details
+
+Suppose $f$ is a continuous function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ and $g: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is a one-to-one function with continuous partial derivatives.
+Then if $U \subseteq \mathbb{R}^n$ is a subset,
+
+$$\int_{g(U)} f(\mathbf {x})d\mathbf {x} = \int_{U}({g}(\mathbf {y}))|J|d\mathbf {y}$$
+
+where $J$ is the Jacobian determinant and \|J\| is its absolute value.
+
+$$J= \left|\begin{bmatrix} \frac{\partial g_1}{\partial y_1} & \frac{\partial g_1}{\partial y_2} & \cdots &\frac{\partial g_1}{\partial y_n} \\ \vdots & \vdots & \cdots & \vdots \\ \frac{\partial g_n}{\partial y_1} & \frac{\partial g_n}{\partial y_2} & \cdots & \frac{\partial g_n}{\partial y_n} \end{bmatrix}\right| = \left|\begin{bmatrix} \nabla g_1 \\ \vdots \\ \nabla g_n \end{bmatrix}\right|$$
+
+Similar calculations as in 28.4 give us that if $X$ is a continuous multivariate random variable, $X = (X_1, \ldots, X_n)^\prime$ with density $f$ and $\mathbf{Y} = \mathbf{h} (\mathbf{X})$, where $\mathbf{h}$ is one-to-one with inverse $\mathbf g= \mathbf{h}^{-1}$.
+So, $\mathbf{X} = g(\mathbf{Y})$, then the density of $\mathbf{Y}$ is given by;
+
+$$f_Y(\mathbf y) = f (g(\mathbf y)) |J|$$
+
+### Examples
+
+:::info Example
+
+If $\mathbf{Y} = A \mathbf X$ where $A$ is an $n \times n$ matrix with $\det(A)\neq0$ and $X = (X_1, \ldots, X_n)^\prime$ are independent and identically distributed random variables, then we have the following results.
+
+The joint density of $X_1 \cdots X_n$ is the product of the individual (marginal) densities,
+
+$$f_X(\mathbf x)= f(x_1) f(x_2) \cdots f(x_n)$$
+
+The matrix of partial derivatives corresponds to $\frac{\partial g}{\partial y}$ where $\mathbf X = \mathbf g(\mathbf{Y})$, i.e. these are the derivatives of the transformation: $\mathbf X = g (\mathbf{Y}) = A^{-1}\mathbf{Y}$, or $\mathbf X = B \mathbf{Y}$ where $B = A^{-1}$
+
+But if $\mathbf X = B \mathbf{Y}$, then
+
+$$X_i = b_{i1}y_1 + b_{i2}y_2 + \cdots b_{ij}y_j\cdots b_{in}y_n$$
+
+So, $\frac{\partial x_i}{\partial y_j} = b_{ij}$ and thus,
+
+$$J =\left|\frac{\partial d\mathbf x}{\partial d\mathbf y}\right| = |B| = |A^{-1}| = \frac {1}{|A|}$$
+
+The density of $\mathbf{Y}$ is therefore;
+
+$$f_Y(\mathbf{y}) = f_X(g(\mathbf{y})) |J| = f_X(A^{-1}\mathbf{y}) |A^{-1}|$$
+
+:::