examples/notebooks/spatial-clustering.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Network Clustering "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we show how pypsa can deal with spatial clustering of networks. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "import cartopy.crs as ccrs\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import pypsa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = pypsa.examples.scigrid_de()\n",
    "n.calculate_dependent_values()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The important information that pypsa needs for spatial clustering is in the `busmap`. It contains the mapping of which buses should be grouped together, similar to the groupby groups as we know it from pandas.\n",
    "\n",
    "You can either calculate a `busmap` from the provided clustering algorithms or you can create/use your own busmap."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cluster by custom busmap"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start with creating our own. \n",
    "In the following, we group all buses together which belong to the same operator. Buses which do not have a specific operator just stay on its own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = n.buses.operator.apply(lambda x: re.split(\" |,|;\", x)[0])\n",
    "busmap = groups.where(groups != \"\", n.buses.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The clustering routine will raise an error if values in non-standard columns are not the same when combined to a common cluster. Therefore, we adjust the columns of the components and delete problematic non-standard values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n.lines = n.lines.reindex(columns=n.components[\"Line\"][\"attrs\"].index[1:])\n",
    "n.lines[\"type\"] = np.nan\n",
    "n.buses = n.buses.reindex(columns=n.components[\"Bus\"][\"attrs\"].index[1:])\n",
    "n.buses[\"frequency\"] = 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we cluster the network based on the busmap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "C = n.cluster.get_clustering_from_busmap(busmap)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`C` is a Clustering object which contains all important information.\n",
    "Among others, the new network is now stored in that Clustering object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nc = C.network"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a look at the original and the clustered topology"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, (ax, ax1) = plt.subplots(\n",
    "    1, 2, subplot_kw={\"projection\": ccrs.EqualEarth()}, figsize=(12, 12)\n",
    ")\n",
    "plot_kwrgs = dict(bus_sizes=1e-3, line_widths=0.5)\n",
    "n.plot(ax=ax, title=\"original\", **plot_kwrgs)\n",
    "nc.plot(ax=ax1, title=\"clustered by operator\", **plot_kwrgs)\n",
    "fig.tight_layout()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks a bit messy as over 120 buses do not have assigned operators."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clustering by busmap created from K-means"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now make a clustering based on the kmeans algorithm.\n",
    "Therefore we calculate the `busmap` from a non-weighted kmeans clustering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weighting = pd.Series(1, n.buses.index)\n",
    "busmap2 = n.cluster.busmap_by_kmeans(bus_weightings=weighting, n_clusters=50)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use this new kmeans-based `busmap` to create a new clustered method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nc2 = n.cluster.cluster_by_busmap(busmap2)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, let's plot the networks to compare:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, (ax, ax1) = plt.subplots(\n",
    "    1, 2, subplot_kw={\"projection\": ccrs.EqualEarth()}, figsize=(12, 12)\n",
    ")\n",
    "plot_kwrgs = dict(bus_sizes=1e-3, line_widths=0.5)\n",
    "n.plot(ax=ax, title=\"original\", **plot_kwrgs)\n",
    "nc2.plot(ax=ax1, title=\"clustered by kmeans\", **plot_kwrgs)\n",
    "fig.tight_layout()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are other clustering algorithms in the pipeline of pypsa as the hierarchical\n",
    "clustering which performs better than the kmeans. Also the `get_clustering_from_busmap` function supports various arguments on how components in the network should be aggregated. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "",
   "language": "python",
   "name": ""
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}