diff --git a/examples/AIMMDB_Tiled_Tutorial.ipynb b/examples/AIMMDB_Tiled_Tutorial.ipynb new file mode 100644 index 0000000..a3a2cf5 --- /dev/null +++ b/examples/AIMMDB_Tiled_Tutorial.ipynb @@ -0,0 +1,484 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "3XEk4VVi483l" + }, + "source": [ + "## Table of Contents\n", + "* 1 - [Access data from a web browser](#section-1)\n", + "* 2 - [Access data using Tiled's Python client](#section-2)\n", + " * 2.1 - [Setup kernel and required dependencies](#section-21)\n", + " * 2.2 - [Read and download data](#section-22)\n", + " * 2.3 - [Visualization and data analysis](#section-23)\n", + " * 2.4 - [Log into an Authenticated Tiled Server](#section-24)\n", + "\n", + "Note that the output of this notebook was generated using [Google Colab](https://colab.research.google.com/).\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "t2YtaEtj6w35" + }, + "source": [ + "## 1 - Access data from a web browser\n", + "\n", + "Click the first tab on https://aimm.lbl.gov/" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "C0aLBAhQ7qr2" + }, + "source": [ + "## 2 - Access data using Tiled's Python client\n", + "### 2.1 - Setup kernel and required dependencies\n", + "**Tiled** is a data access service for data-aware portals and data science tools. Tiled has a Python client and integrates naturally with Python data science libraries, but nothing about the service is Python-specific; it also works from a web browser or any Internet-connected program.\n", + "\n", + "You can find a detailed **Tiled** document here: https://blueskyproject.io/tiled/tutorials/installation.html" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "611FyYOCruGJ", + "outputId": "8a484589-7e67-4455-b005-411ae02a32af" + }, + "outputs": [], + "source": [ + "# Install Tiled from PyPI using pip\n", + "!pip install \"tiled[client]\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "jLZMR8gjODEP" + }, + "source": [ + "### 2.2 - Read and download data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "BdoABDMurx5L" + }, + "outputs": [], + "source": [ + "# connect with client\n", + "from tiled.client import from_uri\n", + "\n", + "client = from_uri(\"https://aimm.lbl.gov\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "KSfWXEsWr9Qz", + "outputId": "503a80c1-46fe-4e09-9f7a-2f79c798b422" + }, + "outputs": [], + "source": [ + "client" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "liNBeFEV_Dow" + }, + "source": [ + "`client` holds a nested structure of data. Conceptually, it corresponds well to a directory of files or hierarchical structure like an HDF5 file or XML file.\n", + "\n", + "Tiled provides a utility for visualizing a nested structure. Each (sub)tree displays the names of a couple of its entries—up to however many fit on one line.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pAjnGK5m_C66", + "outputId": "4064d3d3-885d-4cbf-bd3e-f3a05a1859f3" + }, + "outputs": [], + "source": [ + "from tiled.utils import tree\n", + "tree(client)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9dG8jxIb_ZkQ" + }, + "source": [ + "`Containers` act like (nested) mappings in Python. All the (read-only) methods that work on Python dictionaries work on Containers. We can lookup a specific value by its key." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "79f0pNXpz1vx", + "outputId": "4bb64947-8608-4f95-c37d-73361d790d2e" + }, + "outputs": [], + "source": [ + "client" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qLkXvf79_gY9" + }, + "source": [ + "list all the keys." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "0A5p_E2Stj6Y", + "outputId": "d809d623-82e1-4306-f16f-7e8ca53319a0" + }, + "outputs": [], + "source": [ + "list(client)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ZlfARXNHsAsm", + "outputId": "d6482c04-1f61-4b1a-f4f9-190dffbb05f3" + }, + "outputs": [], + "source": [ + "client['newville']" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "jnH6teuwsCuz", + "outputId": "299b3ce3-13b9-45c8-83b4-c1d8019ae4d2" + }, + "outputs": [], + "source": [ + "client['newville']['2DKJLMZPi8s']" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9KIsSXwGNnyu" + }, + "source": [ + "To read data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 424 + }, + "id": "7INhruoMvVJX", + "outputId": "c7293d8b-6b0c-4c2a-ed6b-ab5e2d292d48" + }, + "outputs": [], + "source": [ + "client['newville']['2DKJLMZPi8s'].read()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "tj_plDvVNvTs" + }, + "source": [ + "To read slices of data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 424 + }, + "id": "LD0ycDlcNu0-", + "outputId": "aef08994-7751-4274-eb48-3a40d8b99197" + }, + "outputs": [], + "source": [ + "client['newville']['2DKJLMZPi8s'].read(['energy', 'i0'])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SbitOnXkOYds" + }, + "source": [ + "The Tiled server can encode its structures in various formats for exporting data. These are just a couple of the supported formats:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "sCnQ-wLpOG6w" + }, + "outputs": [], + "source": [ + "# Table\n", + "client['newville']['2DKJLMZPi8s'].export(\"table.xlsx\") # Excel\n", + "client['newville']['2DKJLMZPi8s'].export(\"table.csv\") # CSV\n", + "\n", + "# Array\n", + "client['newville']['2DKJLMZPi8s'].export(\"numbers.csv\") # CSV\n", + "# client['newville']['2DKJLMZPi8s'].export(\"image.png\") # PNG image\n", + "# client['newville']['2DKJLMZPi8s'].export(\"image.tiff\") # TIFF image" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xSxpVbrAO6Ph" + }, + "source": [ + "If running this notebook on Google Colab, click the folder sign on the left panel and you will see the exported files, right click to download to your local machine." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "165CoOVROG8n" + }, + "outputs": [], + "source": [ + "# Export just some of the columns...\n", + "# client['newville']['2DKJLMZPi8s'].export(\"table.csv\", columns=[\"energy\", \"i0\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "1QgVv9u6Q5Wb" + }, + "source": [ + "### 2.3 - Visualization and data analysis" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "Nt_cqdqFwcZu", + "outputId": "592e2718-e627-4a88-fc85-7ff67fee9738" + }, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "energy_data = client['newville']['2DKJLMZPi8s'].read(['energy'])\n", + "i0_data = client['newville']['2DKJLMZPi8s'].read(['i0'])\n", + "\n", + "plt.plot(energy_data, i0_data)\n", + "plt.xlabel('Energy')\n", + "plt.ylabel('i0')\n", + "plt.title('Energy vs i0 Plot')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "RgiE5P5vRQ_h", + "outputId": "ee93822c-4d32-4f67-ce4e-a08b1e1ed15b" + }, + "outputs": [], + "source": [ + "plt.hist(energy_data, bins=50, color='blue', alpha=0.7)\n", + "plt.title('Energy Histogram')\n", + "plt.xlabel('Values')\n", + "plt.ylabel('Frequency')\n", + "plt.grid(True)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CuIak7-zRvzZ" + }, + "source": [ + "\n", + "## 2.4 Log into an Authenticated Tiled Server\n", + "So that you can access private dataset" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "qk7sUInyto1u", + "outputId": "7cb97f0a-c459-44b4-e2b7-c6a89dd60855" + }, + "outputs": [], + "source": [ + "client.login()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "NrQCq4uLuC6e", + "outputId": "6c5827c9-8a78-40fc-9fa8-eefa5228a5b5" + }, + "outputs": [], + "source": [ + "client" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rOwzNdh9uNRj", + "outputId": "8a619ebc-322f-4f46-f46d-ed24e1099b75" + }, + "outputs": [], + "source": [ + "list(client)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "tjdHaFS8uRG5" + }, + "outputs": [], + "source": [ + "wanli_dataset = client['aimm_core']" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 424 + }, + "id": "BlEIzj8IuV1o", + "outputId": "4180f230-4aef-48df-a578-631cb876de76" + }, + "outputs": [], + "source": [ + "wanli_dataset['4yk5nB7RvRN'].read()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qd6XJtkSSbQ4" + }, + "source": [ + "Now log out via:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1PX79QRNul_s" + }, + "outputs": [], + "source": [ + "client.logout()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "plULB7QOScjJ" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}