3.coles_thermodynamic_formalism.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# \"Thermodynamic AI\" - Coles Formalism\n",
    "\n",
    "Patrick Coles of Normal Computing, arrived at a different formalism that generalizes the theory of probabilistic computing. Its description is a more mathematically rigorous approach, inspired by Coles' own background in Quantum Computing. Coles argues for reimagining p-bits (he prefers the term s-bits) as Continuous-Time Markov Chains (CTMCs) and extends the idea, more generally, to white noise processes that can take on continuous values in some range in what are called s-modes.\n",
    "\n",
    "We'll begin with the foundational ideas of CTMCs and Weiner processes, and then move on to Coles' formalism.\n",
    "\n",
    "### Continuous-Time Markov Chains (s-bits)\n",
    "\n",
    "The CTMC can be thought of as follows, take some discrete state space $S$ and a continuous time space $T$. The CTMC is a stochastic process that moves from state to state in $S$ at random times in $T$. The probability of moving from state $i$ to state $j$ in time $t$ is given by the transition rate $q_{ij}(t)$.\n",
    "\n",
    "For example imagine we had a discrete state space with three states $\\{1,2,3\\}$ with a $Q$-matrix given by:\n",
    "\n",
    "$$\n",
    "Q = \\begin{bmatrix}\n",
    "-1 & 1 & 0 \\\\\n",
    "1 & -2 & 1 \\\\\n",
    "0 & 1 & -1\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "The way the $Q$-matrix now works is that if we're in state $i$, then we wait until some holding time $E_i\\sim \\text{Exp}(\\|Q_{ii}\\|)$, where $\\|Q_{ii}\\|$ is the absolute value of the diagonal element of the $Q$-matrix. Then we move to state $j$ with probability $q_{ij} = Q_{ij}/\\|Q_{ii}\\|$ - a lot like the normal action of a Markov chain.\n",
    "\n",
    "In the case of Coles' formalism, s-bits are in a 2-state space $\\{0,1\\}$ and the $Q$-matrix is given by:\n",
    "\n",
    "$$\n",
    "Q = \\begin{bmatrix}\n",
    "1-\\lambda_0(t) & \\lambda_0(t) \\\\\n",
    "\\lambda_1(t) & 1-\\lambda_1(t)\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "where $\\lambda_0(t)$ and $\\lambda_1(t)$ are the transition rates from state $0$ to state $1$ and vice versa. The holding times are then given by $E_0\\sim \\text{Exp}(1-\\lambda_0(t))$ and $E_1\\sim \\text{Exp}(1-\\lambda_1(t))$. And transition probabilities are given by $q_{01} = \\lambda_0(t)/(1-\\lambda_0(t))$ and $q_{10} = \\lambda_1(t)/(1-\\lambda_1(t))$.\n",
    "\n",
    "Cole's own choice for this formalism is that he argues that a good analog source of noise is shot-noise which is a Poisson process, meaning exponentially distributed timing between events. These then become clockless flip signals for s-bits which allow for a programmable RNG. This is probably a canny move from Coles to avoid the exotic nanodevices or an explosion of amplifiers in his designs.\n",
    "\n",
    "### Weiner Processes (s-modes)\n",
    "\n",
    "The choice of this arises from the other common source of analog noise: Johnson-Nyquist noise (aka. thermal noise) and theoretically white noise. To understand how Coles' thinks about these, we have use stochastic differential equations. Johnson-Nyquist noise is generally defined as:\n",
    "\n",
    "$$\n",
    "V(t) = \\sqrt{4k_BTR} \\cdot\\xi(t)\n",
    "$$\n",
    "\n",
    "Where $k_B$ is the Boltzmann constant, $T$ is temperature and $R$ is the resistance of circuit we're measuring thermal noise over. The last term is a white noise process $\\xi(t)$ which is best understood as a stationary process with a constant (power) spectral density (ie. your fourier transform tends to a flat line).\n",
    "\n",
    "For mathematical *reasons* the time integral of Johnson-Nyquist noise yields a Weiner process, aka. Brownian motion - this is what Coles' decides is his s-mode that gives him some crisp stochastic differential equations to solve, manipulate and simulate.\n",
    "\n",
    "And boy, how straightforward they are, Weiner processes are the cornerstone of Ito calculus and Weiner processes are very easy to simulate, namely they always start $(x,t) = (0,0)$ and they evolve with the rule, given a change in time $\\Delta t= t_2-t_1$ the change in position is given by $\\Delta x \\sim \\mathcal{N}(0,\\Delta t)$.\n",
    "\n",
    "But they have a lot of other weird properties, especially as a voltage, such as just wonder off to infinity in either direction which makes them a questionable choice... but man is that math clean.\n",
    "\n",
    "### Comparison to p-bits\n",
    "\n",
    "I've let Coles know in [a tweet]() that I think that what he calls an \"s-bit\" which he thinks is the \"time integral of a p-bit\"... iiis really just a p-bit. Especially if you look at the device physics of proposed p-bits. While I'm sure a former PI of QC at Los Alamos labs doesn't care what I have to say on the matter, I do!\n",
    "\n",
    "s-modes are totally legit and new tho, and I'd never heard of a p-mode until he made it up, so dope, dope. Love you fr P. Coley <3"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}