its_train_eval_predict.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "6dbe05cb-93de-4407-bf6a-b90979124b5f",
      "metadata": {
        "id": "6dbe05cb-93de-4407-bf6a-b90979124b5f"
      },
      "source": [
        "## ‘What Destroys a Person When that Person Appears to Be Destroying Himself?’"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "403f7ee9-1b40-490c-baab-f8feb43ab40d",
      "metadata": {
        "id": "403f7ee9-1b40-490c-baab-f8feb43ab40d"
      },
      "source": [
        "_Fine tunes BERT, RoBERTa, DistilBERT pretrained LMs to classify ITS targets, structures and merges analytic sample linking posts with direct-response comments for fine-tuned LM inference and downstream reflexive thematic analysis. Applies CausalNLP tools to detect post &rarr; comment ruptures in cumulative elicited support conditioned on ITS-labeled expressions and additional post author disclosures._"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6d671a7a-ee4a-42b1-874f-e22a6ee7604e",
      "metadata": {
        "id": "6d671a7a-ee4a-42b1-874f-e22a6ee7604e"
      },
      "source": [
        "> its_train_eval_pred.ipynb<br>\n",
        "> Simone J. Skeen (08-14-2024)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "1. [Prepare](#scrollTo=VfO9Lvsq5pOQ)\n",
        "2. [Preprocess](#scrollTo=st5A4Pp5MFtZ)\n",
        "3. [Train-Test: BERT, RoBERTa, DistilBERT](#scrollTo=a4f0b99a-a3cc-493a-b74e-0af2973af2eb)\n",
        "4. [Train-Test-Tune: model x sub/construct-specific](#scrollTo=xxM1ebUSIvA4)\n",
        "5. [Predict](#scrollTo=6ALab8tymeX6)\n",
        "6. [Causal inference](#scrollTo=8-MPUSq8J-Vv)"
      ],
      "metadata": {
        "id": "fqeZESuik1qP"
      },
      "id": "fqeZESuik1qP"
    },
    {
      "cell_type": "markdown",
      "id": "VfO9Lvsq5pOQ",
      "metadata": {
        "id": "VfO9Lvsq5pOQ"
      },
      "source": [
        "### 1. Prepare\n",
        "Installs, imports, and downloads requisite models and packages. Organizes RAP-consistent directory structure.\n",
        "***"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "WwWA5bRlpN-j",
      "metadata": {
        "id": "WwWA5bRlpN-j"
      },
      "source": [
        "**Install**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "JwyBB3gX5oCo",
      "metadata": {
        "collapsed": true,
        "id": "JwyBB3gX5oCo"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "\n",
        "%pip install causalnlp\n",
        "%pip install contractions\n",
        "%pip install unidecode\n",
        "%pip install wandb\n",
        "\n",
        "#%pip uninstall -y pyarrow datasets\n",
        "#%pip install pyarrow datasets\n",
        "\n",
        "!pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-3.0.0/en_core_web_lg-3.0.0.tar.gz"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Ec6yfzjqpWDf",
      "metadata": {
        "id": "Ec6yfzjqpWDf"
      },
      "source": [
        "**Import**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "849d095e-174b-499f-b164-d36e37dcc6c3",
      "metadata": {
        "id": "849d095e-174b-499f-b164-d36e37dcc6c3"
      },
      "outputs": [],
      "source": [
        "import contractions\n",
        "import en_core_web_lg\n",
        "import gzip\n",
        "import json\n",
        "import logging\n",
        "import numpy as np\n",
        "import os\n",
        "import pandas as pd\n",
        "import re\n",
        "import sklearn\n",
        "import spacy\n",
        "import string\n",
        "import torch\n",
        "import wandb.sdk\n",
        "import warnings\n",
        "\n",
        "from causalnlp import Autocoder\n",
        "from causalnlp import Autocoder, CausalInferenceModel\n",
        "from google.colab import drive\n",
        "from lightgbm import LGBMClassifier\n",
        "from sklearn.feature_extraction import text\n",
        "from sklearn.linear_model import LogisticRegression, LinearRegression\n",
        "from sklearn.model_selection import KFold, ParameterGrid, StratifiedKFold, train_test_split\n",
        "from sklearn.metrics import classification_report, f1_score, matthews_corrcoef, average_precision_score\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "from sklearn.utils.multiclass import type_of_target\n",
        "from textblob import TextBlob\n",
        "from torch.optim import AdamW\n",
        "from torch.nn import CrossEntropyLoss\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from tqdm import tqdm\n",
        "from transformers import BertTokenizer, BertForSequenceClassification, RobertaTokenizer, RobertaForSequenceClassification, DistilBertTokenizer, DistilBertForSequenceClassification\n",
        "from unidecode import unidecode\n",
        "\n",
        "spacy.cli.download('en_core_web_lg')\n",
        "\n",
        "from IPython.core.interactiveshell import InteractiveShell\n",
        "InteractiveShell.ast_node_interactivity = 'all'\n",
        "\n",
        "pd.set_option(\n",
        "              'display.max_columns',\n",
        "              None,\n",
        "              )\n",
        "\n",
        "pd.set_option(\n",
        "              'display.max_rows',\n",
        "              None,\n",
        "              )\n",
        "\n",
        "warnings.simplefilter(\n",
        "                      action = 'ignore',\n",
        "                      category = FutureWarning,\n",
        "                      )\n",
        "\n",
        "#!python -m prodigy stats"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "zlfrD3_Fpb7N",
      "metadata": {
        "id": "zlfrD3_Fpb7N"
      },
      "source": [
        "**Mount gdrive**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "NvHrz3hvr-tW",
      "metadata": {
        "id": "NvHrz3hvr-tW",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "drive.mount(\n",
        "            '/content/gdrive/',\n",
        "            force_remount = True,\n",
        "            )"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bLQVU7b9Tf2C",
      "metadata": {
        "id": "bLQVU7b9Tf2C"
      },
      "source": [
        "**Structure directories**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "WdQyik1uTfPI",
      "metadata": {
        "id": "WdQyik1uTfPI"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality\n",
        "#%cd /content/gdrive/My Drive/#<my_project_folder>\n",
        "\n",
        "#%mkdir intrapolitical_suicidality"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "y_f15JlvWpMZ",
      "metadata": {
        "collapsed": true,
        "id": "y_f15JlvWpMZ"
      },
      "outputs": [],
      "source": [
        "#%cd intrapolitical_suicidality\n",
        "#%mkdir inputs outputs code temp"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "21NChjxYI3TO",
      "metadata": {
        "id": "21NChjxYI3TO"
      },
      "outputs": [],
      "source": [
        "#%cd inputs\n",
        "#%mkdir archives data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d890afd5Iu2J",
      "metadata": {
        "id": "d890afd5Iu2J"
      },
      "outputs": [],
      "source": [
        "#%cd outputs\n",
        "#%mkdir models tables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "eV5-Ge5FFa8w",
      "metadata": {
        "id": "eV5-Ge5FFa8w"
      },
      "outputs": [],
      "source": [
        "### dir structure TKTK"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "st5A4Pp5MFtZ",
      "metadata": {
        "id": "st5A4Pp5MFtZ"
      },
      "source": [
        "### 2. Preprocess\n",
        "Cleans annotated dataset $\\mathcal{d}$<sub>annotated</sub>, builds rationale-augmented $\\mathcal{d}$<sub>augmented</sub>. Builds and NER-anonymizes analytic sample $\\mathcal{D}$<sub>inference</sub>, subsamples $\\mathcal{D}$<sub>reflexive</sub>.\n",
        "***"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Yo-CQl2lJW6Y",
      "metadata": {
        "id": "Yo-CQl2lJW6Y"
      },
      "source": [
        "#### Clean: annotated set $\\mathcal{d}$<sub>annotated</sub>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ZHD9hN-P6va2",
      "metadata": {
        "collapsed": true,
        "id": "ZHD9hN-P6va2"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "%cd inputs/data\n",
        "\n",
        "d = pd.read_excel('d_annotated.xlsx')\n",
        "\n",
        "d.dtypes\n",
        "d.shape\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "15WVN78h7BSF",
      "metadata": {
        "id": "15WVN78h7BSF"
      },
      "source": [
        "**Reformat**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "38e58bcb",
      "metadata": {
        "collapsed": true,
        "id": "38e58bcb",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "d.set_index(\n",
        "            'index',\n",
        "            inplace = True,\n",
        "            )\n",
        "\n",
        "# Encode multinomial target\n",
        "\n",
        "multi = [\n",
        "         (d['pb'] == 0) & (d['tb'] == 0),\n",
        "         (d['pb'] == 1) & (d['tb'] == 0),\n",
        "         (d['pb'] == 0) & (d['tb'] == 1),\n",
        "         (d['pb'] == 1) & (d['tb'] == 1),\n",
        "         ]\n",
        "\n",
        "label_encode = [\n",
        "                0,\n",
        "                1,\n",
        "                2,\n",
        "                3,\n",
        "                ]\n",
        "\n",
        "d['its'] = np.select(\n",
        "                     multi,\n",
        "                     label_encode,\n",
        "                     )\n",
        "\n",
        "# Append PB + TB rationales\n",
        "\n",
        "d['pb_rtnl'] = d['pb_rtnl'].astype(str)\n",
        "d['tb_rtnl'] = d['tb_rtnl'].astype(str)\n",
        "\n",
        "d['its_rtnl'] = d['pb_rtnl'] + ' ' + d['tb_rtnl']\n",
        "\n",
        "# Replace with \".\" to denote no rationale\n",
        "\n",
        "rationales = [\n",
        "              'pb_rtnl',\n",
        "              'tb_rtnl',\n",
        "              ]\n",
        "\n",
        "for rationale in rationales:\n",
        "    d[rationale] = d[rationale].str.replace(r'0', '.', regex = True)\n",
        "\n",
        "d['its_rtnl'] = d['its_rtnl'].str.replace(r'0 0', '.', regex = True)\n",
        "\n",
        "d.dtypes\n",
        "d.shape\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0UO3J9y67FdP",
      "metadata": {
        "id": "0UO3J9y67FdP"
      },
      "source": [
        "#### Build: augmented set $\\mathcal{d}$<sub>augmented</sub>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "FXGRc7yFbTSh",
      "metadata": {
        "id": "FXGRc7yFbTSh"
      },
      "outputs": [],
      "source": [
        "%cd ../../code"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ef3f257e",
      "metadata": {
        "collapsed": true,
        "id": "ef3f257e"
      },
      "outputs": [],
      "source": [
        "%%writefile augment.py\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "def augment_training_data_with_rationales(row):\n",
        "    \"\"\"\n",
        "    Identifies all pos(1) ITS rows, duplicates as new row below, replaces new row 'text' cell with appended PB-TB 'rationale' cell\n",
        "    \"\"\"\n",
        "    if row['its'] > 0:\n",
        "        new_row = row.copy()\n",
        "        new_row['text'] = row['its_rtnl']\n",
        "        return pd.concat([row, new_row], axis = 1)\n",
        "    else:\n",
        "        return row\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "wUsmVp9lZCYi",
      "metadata": {
        "collapsed": true,
        "id": "wUsmVp9lZCYi"
      },
      "outputs": [],
      "source": [
        "from augment import augment_training_data_with_rationales\n",
        "\n",
        "#import pandas as pd\n",
        "\n",
        "d = d.apply(\n",
        "            augment_training_data_with_rationales,\n",
        "            axis = 1,\n",
        "            )\n",
        "\n",
        "d_aug = pd.concat(\n",
        "                  d.tolist(),\n",
        "                  axis = 1,\n",
        "                  ).T\n",
        "\n",
        "# Reset index\n",
        "\n",
        "d_aug.reset_index(\n",
        "                  drop = True,\n",
        "                  inplace = True,\n",
        "                  )\n",
        "\n",
        "# Verify...\n",
        "\n",
        "d_aug.head(3)\n",
        "d_aug.shape\n",
        "d_aug['its'].value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "l30WiopV7Mk5",
      "metadata": {
        "id": "l30WiopV7Mk5"
      },
      "source": [
        "**Clean**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d4e15595",
      "metadata": {
        "collapsed": true,
        "id": "d4e15595"
      },
      "outputs": [],
      "source": [
        "# Clear empty cells\n",
        "\n",
        "d_aug = d_aug[d_aug.text != ' ']\n",
        "\n",
        "# Clear '0' cells\n",
        "\n",
        "d_aug = d_aug[d_aug['text'] != '0']\n",
        "\n",
        "# Clear empty + '0' by visually confirmed p_id (failsafe)\n",
        "\n",
        "# Define the list of p_id values to drop\n",
        "del_id = [\n",
        "          '7m1105',\n",
        "          'fsncni',\n",
        "          '10rv1kt',\n",
        "          'le5egj',\n",
        "          'onpzzm',\n",
        "          'xkw7wi',\n",
        "          'pdearh',\n",
        "          'hkqtcw',\n",
        "          'bq8omz',\n",
        "          'f9mf2g',\n",
        "          'gf4ukm',\n",
        "          '9jpshz',\n",
        "          'x7l9ck',\n",
        "          '98oo27',\n",
        "          '373hrz',\n",
        "          '38au87',\n",
        "          '9wxbvv',\n",
        "          'iislhk',\n",
        "          'ypa71a',\n",
        "          'zd31tr',\n",
        "          '2pulw8',\n",
        "          '1yjsba',\n",
        "          'ukw36h',\n",
        "          '2699w9',\n",
        "          '8q029b',\n",
        "          'skgzms',\n",
        "         ]\n",
        "\n",
        "# Drop rows with specified p_id values\n",
        "\n",
        "d_aug = d_aug[~d_aug['p_id'].isin(del_id)]\n",
        "\n",
        "# Excise deprecated '[REDACTED]' NER tags\n",
        "\n",
        "d_aug['text'] = d_aug['text'].str.replace(r'\\[REDACTED\\]', ' ', regex = True)\n",
        "\n",
        "# Confirm...\n",
        "\n",
        "#rdct_rows = d_aug[d_aug['text'].str.contains(r'\\[REDACTED\\]', regex = True)]\n",
        "#print(rdct_rows)\n",
        "\n",
        "# Excise encoding artifacts\n",
        "\n",
        "artifacts = [\n",
        "             '0',\n",
        "             'Ã,',\n",
        "             'Â…Ã',\n",
        "             'Â',\n",
        "             '€',\n",
        "             '¢',\n",
        "             'ƒ',\n",
        "             '¬',\n",
        "             '£',\n",
        "             ]\n",
        "\n",
        "for artifact in artifacts:\n",
        "    d_aug['text'] = d_aug['text'].str.replace(\n",
        "                                              artifact,\n",
        "                                              ' ',\n",
        "                                              regex = True,\n",
        "                                              )\n",
        "\n",
        "d_aug.shape\n",
        "d_aug.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "60d774a1-5f17-41b2-b033-1b2aa50a6c80",
      "metadata": {
        "collapsed": true,
        "id": "60d774a1-5f17-41b2-b033-1b2aa50a6c80"
      },
      "outputs": [],
      "source": [
        "%%writefile -a augment.py\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "def dummy_code_augmented_rows(df):\n",
        "    \"\"\"\n",
        "    Identifies all rationale-augmented rows in df, dummy codes for deletion prior to evaluation.\n",
        "    \"\"\"\n",
        "\n",
        "    df = df.reset_index(drop = True)\n",
        "\n",
        "    df['aug'] = 0\n",
        "\n",
        "    for i in range(1, len(df)):\n",
        "        if df.at[i, 'its_rtnl'] != '.' and df.at[i, 'its_rtnl'] == df.at[i-1, 'its_rtnl']:\n",
        "            df.at[i, 'aug'] = 1\n",
        "\n",
        "    return df"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "THtd8qPmVU59",
      "metadata": {
        "collapsed": true,
        "id": "THtd8qPmVU59"
      },
      "outputs": [],
      "source": [
        "from augment import dummy_code_augmented_rows\n",
        "\n",
        "d_aug = dummy_code_augmented_rows(d_aug)\n",
        "\n",
        "d_aug[[\n",
        "       'aug',\n",
        "       'hate',\n",
        "       'libl',\n",
        "       'lone',\n",
        "       'recp',\n",
        "       'pb',\n",
        "       'tb',\n",
        "       ]].apply(pd.Series.value_counts)\n",
        "\n",
        "d_aug.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "mJ3DsC2VP5Eo",
      "metadata": {
        "id": "mJ3DsC2VP5Eo"
      },
      "outputs": [],
      "source": [
        "%cd ../inputs/data\n",
        "\n",
        "d_aug.to_excel('d_augmented.xlsx')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "lnlSvakUROx4",
      "metadata": {
        "id": "lnlSvakUROx4"
      },
      "source": [
        "#### Build: prediction set $\\mathcal{D}$<sub>inference</sub>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "GJWKpACZUnrk",
      "metadata": {
        "collapsed": true,
        "id": "GJWKpACZUnrk"
      },
      "outputs": [],
      "source": [
        "archives_path = '/content/gdrive/MyDrive/Colab/intrapolitical_suicidality/inputs/archives/'\n",
        "\n",
        "dir_p = f'{archives_path}sw_complete-posts.json.gz'\n",
        "\n",
        "dir_c = f'{archives_path}sw_complete-comments.json.gz'\n",
        "\n",
        "\n",
        "# d_p = posts\n",
        "\n",
        "d_p = pd.DataFrame(json.loads(l) for l in gzip.open(\n",
        "                                                    dir_p,\n",
        "                                                    'rb',\n",
        "                                                    )\n",
        "                   )\n",
        "\n",
        "d_p = d_p.drop_duplicates(subset = 'id')\n",
        "\n",
        "d_p['date'] = pd.to_datetime(\n",
        "                             d_p.created_utc,\n",
        "                             unit = 's',\n",
        "                             )\n",
        "\n",
        "d_p.set_index(\n",
        "              'date',\n",
        "              drop = False,\n",
        "              inplace = True,\n",
        "              )\n",
        "\n",
        "d_p = d_p.loc[(d_p['date'] >= '2018-01-01') & (d_p['date'] < '2023-01-01')]\n",
        "\n",
        "d_p = d_p[~d_p['selftext'].isin(['[deleted]', '[removed]'])]\n",
        "\n",
        "\n",
        "# d_c = comments\n",
        "\n",
        "d_c = pd.DataFrame(json.loads(l) for l in gzip.open(\n",
        "                                                    dir_c,\n",
        "                                                    'rb',\n",
        "                                                    )\n",
        "                   )\n",
        "\n",
        "d_c = d_c.drop_duplicates(subset = 'id')\n",
        "\n",
        "d_c['date'] = pd.to_datetime(\n",
        "                             d_c.created_utc,\n",
        "                             unit = 's',\n",
        "                             )\n",
        "\n",
        "d_c.set_index(\n",
        "              'date',\n",
        "              drop = False,\n",
        "              inplace = True,\n",
        "              )\n",
        "\n",
        "d_c = d_c.loc[(d_c['date'] >= '2018-01-01') & (d_c['date'] < '2023-01-01')]\n",
        "\n",
        "d_c = d_c[~d_c['body'].isin(['[deleted]', '[removed]'])]\n",
        "\n",
        "\n",
        "d_p.shape\n",
        "d_c.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Delete comment-level prefix**"
      ],
      "metadata": {
        "id": "d9D_LLLcbYH1"
      },
      "id": "d9D_LLLcbYH1"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "edMUyQLr4WGp",
      "metadata": {
        "collapsed": true,
        "id": "edMUyQLr4WGp"
      },
      "outputs": [],
      "source": [
        "d_c['link_id'] = d_c['link_id'].str.replace('t3_', '')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Disambiguate 'date', 'id'**"
      ],
      "metadata": {
        "id": "VVNDg_LVbduV"
      },
      "id": "VVNDg_LVbduV"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ASkAlH-84tji",
      "metadata": {
        "id": "ASkAlH-84tji"
      },
      "outputs": [],
      "source": [
        "d_c.rename(\n",
        "           columns = {'date': 'c_date'},\n",
        "           inplace = True,\n",
        "           )\n",
        "d_c.rename(\n",
        "           columns = {'id': 'c_id'},\n",
        "           inplace = True,\n",
        "           )"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Harmonize 1:_n_ merge variable**"
      ],
      "metadata": {
        "id": "UH-smqs2bjp_"
      },
      "id": "UH-smqs2bjp_"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f4hfSfcz4tnZ",
      "metadata": {
        "collapsed": true,
        "id": "f4hfSfcz4tnZ"
      },
      "outputs": [],
      "source": [
        "d_c.rename(\n",
        "           columns = {'link_id': 'id'},\n",
        "           inplace = True,\n",
        "           )\n",
        "#d_c.columns"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "i3IfngCd4tvI",
      "metadata": {
        "collapsed": true,
        "id": "i3IfngCd4tvI"
      },
      "outputs": [],
      "source": [
        "d = d_p.merge(\n",
        "              d_c,\n",
        "              how = 'left',\n",
        "              on = 'id',\n",
        "              suffixes = ('_post', '_comment')\n",
        "              )\n",
        "\n",
        "d.shape\n",
        "#d.columns\n",
        "#d.head(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "kneiePh84t--",
      "metadata": {
        "collapsed": true,
        "id": "kneiePh84t--"
      },
      "outputs": [],
      "source": [
        "# Housekeeping\n",
        "\n",
        "d = d[[\n",
        "       'author_post',\n",
        "       'date',\n",
        "       'id',\n",
        "       'num_comments',\n",
        "       'selftext',\n",
        "       'subreddit_post',\n",
        "       'title',\n",
        "       'author_comment',\n",
        "       'subreddit_comment',\n",
        "       'body',\n",
        "       ]].copy()\n",
        "\n",
        "d.rename(\n",
        "         columns = {\n",
        "                    'author_post' : 'p_au',\n",
        "                    'date' : 'p_date',\n",
        "                    'num_comments' : 'n_cmnt',\n",
        "                    'selftext' : 'text',\n",
        "                    'subreddit_post' : 'p_sbrt',\n",
        "                    'title' : 'p_titl',\n",
        "                    'comment_date' : 'c_date',\n",
        "                    'author_comment' : 'c_au',\n",
        "                    'subreddit_comment' : 'c_sbrt',\n",
        "                    'body' : 'c_text',\n",
        "                    }, inplace = True,\n",
        "           )\n",
        "\n",
        "\n",
        "# Pseudo-word token for repeat rows\n",
        "\n",
        "d['dupl'] = d.duplicated(\n",
        "                        subset = 'text',\n",
        "                        keep = 'first',\n",
        "                        )\n",
        "\n",
        "d.loc[d['dupl'], 'text'] = '<|RPT|>'\n",
        "d = d.drop(columns = ['dupl'])\n",
        "\n",
        "d['dupl'] = d.duplicated(\n",
        "                         subset = 'p_titl',\n",
        "                         keep = 'first',\n",
        "                         )\n",
        "\n",
        "d.loc[d['dupl'], 'p_titl'] = '<|RPT|>'\n",
        "d = d.drop(columns = ['dupl'])\n",
        "\n",
        "\n",
        "# Post-comment block var\n",
        "\n",
        "# New block\n",
        "d['new_block'] = d['text'] != '<|RPT|>'\n",
        "\n",
        "# _Sum_ blocks for blockid\n",
        "d['block'] = (d['new_block'].cumsum() + 1).where(d['new_block'], 0).astype(int)\n",
        "\n",
        "# Forward fill\n",
        "d['block'] = d['block'].replace(0, method = 'ffill').astype(int)\n",
        "\n",
        "        ### SJS 12/30: preserve new_block var for sense-check (for now)\n",
        "\n",
        "# Drop 'new_block'\n",
        "#post_comments = post_comments.drop(columns=['new_block'])\n",
        "\n",
        "# Dummy for p_author replying to comments\n",
        "\n",
        "d['p_au_reply'] = np.where(d['p_au'] == d['c_au'], 1, 0)\n",
        "\n",
        "d.shape\n",
        "d.head(20)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "mEVZVTVEXnVg",
      "metadata": {
        "id": "mEVZVTVEXnVg"
      },
      "source": [
        "**Subsample and export: $\\mathcal{D}$<sub>reflexive</sub>**"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%pwd"
      ],
      "metadata": {
        "id": "wxncUlKbcPl1"
      },
      "id": "wxncUlKbcPl1",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "A89jUEfV4uNt",
      "metadata": {
        "collapsed": true,
        "id": "A89jUEfV4uNt"
      },
      "outputs": [],
      "source": [
        "# Subsample\n",
        "\n",
        "sampled_blocks = d['block'].sample(\n",
        "                                  n = 1000,\n",
        "                                  replace = False,\n",
        "                                  random_state = 56,\n",
        "                                  )\n",
        "\n",
        "# Filter\n",
        "\n",
        "d_reflexive = d[d['block'].isin(sampled_blocks)]\n",
        "d_reflexive.shape\n",
        "#d_reflexive.head(30)\n",
        "\n",
        "d_reflexive.to_excel('d_reflexive.xlsx')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "w4yHMM_V3hVM",
      "metadata": {
        "id": "w4yHMM_V3hVM"
      },
      "source": [
        "**Restrict (2021-2022) and export: $\\mathcal{D}$<sub>inference</sub>**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "gkeK2ixK3uC-",
      "metadata": {
        "id": "gkeK2ixK3uC-"
      },
      "outputs": [],
      "source": [
        "d_inference = d.loc[(d['p_date'] >= '2021-01-01') & (d['p_date'] < '2023-01-01')]\n",
        "\n",
        "d_inference.head(3)\n",
        "d_inference.tail(3)\n",
        "d_inference.shape\n",
        "\n",
        "d_inference.to_csv('d_inference.csv')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a4f0b99a-a3cc-493a-b74e-0af2973af2eb",
      "metadata": {
        "id": "a4f0b99a-a3cc-493a-b74e-0af2973af2eb"
      },
      "source": [
        "### 3. Train-Test: BERT, RoBERTa, DistilBERT\n",
        "Trains baseline BERT, RoBERTa, and DistilBERT, using rationale-augmented data $\\mathcal{d}$<sub>augmented</sub>, iterating over intrapolitical theory targets. <br>Evaluates using de-augmented data. Outputs model x target _F_<sub>1</sub> (macro) performance scores.\n",
        "***"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "OQ30LOaV7w_X",
      "metadata": {
        "id": "OQ30LOaV7w_X"
      },
      "outputs": [],
      "source": [
        "d_augmented = pd.read_excel('d_augmented.xlsx')\n",
        "\n",
        "d_augmented = d_augmented[[\n",
        "                           'text',\n",
        "                           'aug',\n",
        "                           'hate',\n",
        "                           'libl',\n",
        "                           'lone',\n",
        "                           'recp',\n",
        "                           'pb',\n",
        "                           'tb',\n",
        "                           ]].copy()\n",
        "\n",
        "d_augmented[[\n",
        "             'aug',\n",
        "             'hate',\n",
        "             'libl',\n",
        "             'lone',\n",
        "             'recp',\n",
        "             'pb',\n",
        "             'tb',\n",
        "             ]].apply(pd.Series.value_counts)\n",
        "\n",
        "d_augmented.shape\n",
        "d_augmented.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bnxsQhRxTPs1",
      "metadata": {
        "id": "bnxsQhRxTPs1"
      },
      "outputs": [],
      "source": [
        "%cd ../../outputs/tables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "79af0ffe-3ecc-4978-a28f-830b6239b73c",
      "metadata": {
        "collapsed": true,
        "id": "79af0ffe-3ecc-4978-a28f-830b6239b73c"
      },
      "outputs": [],
      "source": [
        "#%%capture\n",
        "\n",
        "targets_and_class_weights = {\n",
        "                             'hate': [\n",
        "                                      0.6416, ### neg_0 class_weight\n",
        "                                      2.2646, ### pos_1 class_weight\n",
        "                                      ],\n",
        "                             'libl': [\n",
        "                                      0.5207,\n",
        "                                      12.5688,\n",
        "                                      ],\n",
        "                             'lone': [\n",
        "                                      0.6911,\n",
        "                                      1.8084,\n",
        "                                      ],\n",
        "                             'recp': [\n",
        "                                      0.5314,\n",
        "                                      8.4496,\n",
        "                                      ],\n",
        "                               'pb': [\n",
        "                                      0.6735,\n",
        "                                      1.9411,\n",
        "                                      ],\n",
        "                               'tb': [\n",
        "                                      0.7281,\n",
        "                                      1.5960,\n",
        "                                      ],\n",
        "                              }\n",
        "\n",
        "\n",
        "models = {\n",
        "          'bert': (\n",
        "                   BertForSequenceClassification,\n",
        "                   BertTokenizer,\n",
        "                   'bert-base-uncased',\n",
        "                   ),\n",
        "\n",
        "          'roberta': (\n",
        "                      RobertaForSequenceClassification,\n",
        "                      RobertaTokenizer,\n",
        "                      'roberta-base',\n",
        "                      ),\n",
        "\n",
        "          'distilbert': (\n",
        "                         DistilBertForSequenceClassification,\n",
        "                         DistilBertTokenizer,\n",
        "                         'distilbert-base-uncased',\n",
        "                         )\n",
        "          }\n",
        "\n",
        "\n",
        "k_fold = StratifiedKFold(\n",
        "                         n_splits = 5,\n",
        "                         shuffle = True,\n",
        "                         random_state = 56,\n",
        "                         )\n",
        "\n",
        "results = []\n",
        "\n",
        "# Check CUDA\n",
        "\n",
        "print(\"CUDA: \", torch.cuda.is_available())\n",
        "use_cuda = torch.cuda.is_available()\n",
        "\n",
        "# Best F_1 tracking\n",
        "\n",
        "best_f1_scores = {target: {'score': 0, 'model': None} for target in targets_and_class_weights}\n",
        "\n",
        "# Training loop: target x model\n",
        "\n",
        "for target, class_weights in targets_and_class_weights.items():\n",
        "\n",
        "    print(\"\\n======================================================================================\")\n",
        "    print(f\"Label: {target}\")\n",
        "    print(\"======================================================================================\")\n",
        "\n",
        "    for model_name, (\n",
        "                     model_class,\n",
        "                     tokenizer_class,\n",
        "                     pretrained_model_name,\n",
        "                     ) in models.items():\n",
        "\n",
        "        print(f\"\\nFine-tuning {model_name} for {target}\")\n",
        "        print(\"--------------------------------------------------------------------------------------\")\n",
        "\n",
        "        fold_results = {\n",
        "                        'target': target,\n",
        "                        'model': model_name,\n",
        "                        }\n",
        "        metrics = {\n",
        "                   'f1_macro': [],\n",
        "                   'mcc': [],\n",
        "                   'auprc': [],\n",
        "                   }\n",
        "\n",
        "\n",
        "        # Extract text, targets\n",
        "\n",
        "        X = d_augmented['text'].values\n",
        "        y = d_augmented[target].values\n",
        "\n",
        "        # Determine target type\n",
        "\n",
        "        target_type = 'binary' if len(np.unique(y)) <= 2 else 'multiclass'\n",
        "\n",
        "        # Convert target (as needed)\n",
        "\n",
        "        if target_type == 'binary':\n",
        "            le = LabelEncoder()\n",
        "            y = le.fit_transform(y)\n",
        "\n",
        "        # Iterate over 5-fold cv\n",
        "\n",
        "        for fold_idx, (train_idx, valid_idx) in enumerate(k_fold.split(X, y)):\n",
        "            X_train, y_train = X[train_idx], y[train_idx]\n",
        "            X_valid, y_valid = X[valid_idx], y[valid_idx]\n",
        "\n",
        "            # Parse by 'aug'\n",
        "\n",
        "            train_aug_mask = d_augmented.iloc[train_idx]['aug'] == 1\n",
        "            valid_no_aug_mask = d_augmented.iloc[valid_idx]['aug'] != 1\n",
        "\n",
        "            X_train_aug = X_train[train_aug_mask]\n",
        "            y_train_aug = y_train[train_aug_mask]\n",
        "            X_valid_no_aug = X_valid[valid_no_aug_mask]\n",
        "            y_valid_no_aug = y_valid[valid_no_aug_mask]\n",
        "\n",
        "            # Tokenize\n",
        "\n",
        "            tokenizer = tokenizer_class.from_pretrained(pretrained_model_name)\n",
        "\n",
        "            encoded_train_aug = tokenizer(\n",
        "                                          X_train_aug.tolist(),\n",
        "                                          padding = True,\n",
        "                                          truncation = True,\n",
        "                                          return_tensors = 'pt',\n",
        "                                          )\n",
        "\n",
        "            encoded_valid_no_aug = tokenizer(\n",
        "                                             X_valid_no_aug.tolist(),\n",
        "                                             padding = True,\n",
        "                                             truncation = True,\n",
        "                                             return_tensors = 'pt',\n",
        "                                             )\n",
        "\n",
        "            train_dataset_aug = TensorDataset(\n",
        "                                              encoded_train_aug['input_ids'],\n",
        "                                              encoded_train_aug['attention_mask'],\n",
        "                                              torch.tensor(y_train_aug)\n",
        "                                              )\n",
        "\n",
        "            valid_dataset_no_aug = TensorDataset(\n",
        "                                                 encoded_valid_no_aug['input_ids'],\n",
        "                                                 encoded_valid_no_aug['attention_mask'],\n",
        "                                                 torch.tensor(y_valid_no_aug)\n",
        "                                                 )\n",
        "\n",
        "            train_loader_aug = DataLoader(\n",
        "                                          train_dataset_aug,\n",
        "                                          batch_size = 4,\n",
        "                                          shuffle = True,\n",
        "                                          )\n",
        "\n",
        "            valid_loader_no_aug = DataLoader(\n",
        "                                             valid_dataset_no_aug,\n",
        "                                             batch_size = 4,\n",
        "                                             shuffle = False,\n",
        "                                             )\n",
        "\n",
        "            # Instantiate model\n",
        "\n",
        "            model = model_class.from_pretrained(pretrained_model_name)\n",
        "\n",
        "            # Migrate to CUDA (if available)\n",
        "\n",
        "            if use_cuda:\n",
        "                model = model.cuda()\n",
        "\n",
        "            # Fine-tune model on training data with 'aug' rows\n",
        "\n",
        "            optimizer = torch.optim.AdamW(\n",
        "                                          model.parameters(),\n",
        "                                          lr = 2e-5,\n",
        "                                          )\n",
        "\n",
        "            # Training with 'aug'\n",
        "\n",
        "            model.train()\n",
        "\n",
        "            # Define the loss criterion with class weights\n",
        "\n",
        "            criterion = CrossEntropyLoss(\n",
        "                                         weight = torch.tensor(\n",
        "                                                               class_weights,\n",
        "                                                               dtype = torch.float,\n",
        "                                                               ).cuda() if use_cuda else torch.tensor(\n",
        "                                                                                                      class_weights,\n",
        "                                                                                                      dtype = torch.float,\n",
        "                                                                                                      )\n",
        "                                         )\n",
        "\n",
        "            for epoch in range(2):  ### range(n) = n epochs\n",
        "                for i, batch in enumerate(train_loader_aug):\n",
        "                    input_ids, attention_mask, labels = batch\n",
        "                    labels = labels.long()\n",
        "                    if use_cuda:\n",
        "                        input_ids, attention_mask, labels = input_ids.cuda(), attention_mask.cuda(), labels.cuda()\n",
        "                    optimizer.zero_grad()\n",
        "                    outputs = model(\n",
        "                                    input_ids,\n",
        "                                    attention_mask = attention_mask,\n",
        "                                    )\n",
        "                    logits = outputs.logits\n",
        "                    loss = criterion(\n",
        "                                     logits,\n",
        "                                     labels,\n",
        "                                     )\n",
        "                    loss.backward()\n",
        "                    optimizer.step()\n",
        "\n",
        "            # Evaluate model on validation data without 'aug'\n",
        "\n",
        "            model.eval()\n",
        "            all_predictions = []\n",
        "            all_true_labels = []\n",
        "            with torch.no_grad():\n",
        "                for batch in valid_loader_no_aug:\n",
        "                    input_ids, attention_mask, labels = batch\n",
        "                    if use_cuda:  # Added this line\n",
        "                        input_ids, attention_mask, labels = input_ids.cuda(), attention_mask.cuda(), labels.cuda()  # Added this line\n",
        "                    outputs = model(\n",
        "                                    input_ids,\n",
        "                                    attention_mask = attention_mask,\n",
        "                                    )\n",
        "                    logits = outputs.logits\n",
        "                    predictions = torch.argmax(\n",
        "                                               logits,\n",
        "                                               dim = 1,\n",
        "                                               ).tolist()\n",
        "                    all_predictions.extend(predictions)\n",
        "                    all_true_labels.extend(labels.tolist())\n",
        "\n",
        "            # Calculate evaluation metrics without 'aug'\n",
        "\n",
        "            f1_macro = f1_score(\n",
        "                                all_true_labels,\n",
        "                                all_predictions,\n",
        "                                average = 'macro',\n",
        "                                )\n",
        "\n",
        "            mcc = matthews_corrcoef(\n",
        "                                    all_true_labels,\n",
        "                                    all_predictions,\n",
        "                                    )\n",
        "\n",
        "            auprc = average_precision_score(\n",
        "                                            all_true_labels,\n",
        "                                            all_predictions,\n",
        "                                            average='macro',\n",
        "                                            )\n",
        "\n",
        "            metrics['f1_macro'].append(f1_macro)\n",
        "            metrics['mcc'].append(mcc)\n",
        "            metrics['auprc'].append(auprc)\n",
        "\n",
        "        # Compute average metrics for the fold\n",
        "\n",
        "        fold_results.update({metric: np.mean(values) for metric, values in metrics.items()})\n",
        "        fold_results.update({metric + '_std': np.std(values) for metric, values in metrics.items()})\n",
        "\n",
        "        # Append fold results to overall results\n",
        "\n",
        "        results.append(fold_results)\n",
        "\n",
        "        # Update best F_1 score\n",
        "\n",
        "        if np.mean(metrics['f1_macro']) > best_f1_scores[target]['score']:\n",
        "            best_f1_scores[target]['score'] = np.mean(metrics['f1_macro'])\n",
        "            best_f1_scores[target]['model'] = model_name\n",
        "\n",
        "# Display and export results\n",
        "\n",
        "print(\"\\n--------------------------------------------------------------------------------------\")\n",
        "print(f\"Summary\")\n",
        "print(\"--------------------------------------------------------------------------------------\")\n",
        "\n",
        "for target, info in best_f1_scores.items():\n",
        "    print(f\"Best F1 (macro) for {target}: {info['score']} achieved by {info['model']}\")\n",
        "\n",
        "d_perf = pd.DataFrame(results)\n",
        "#print(d_perf)\n",
        "d_perf.head(24)\n",
        "\n",
        "d_perf.to_excel('d_bl_performance.xlsx')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "xxM1ebUSIvA4",
      "metadata": {
        "id": "xxM1ebUSIvA4"
      },
      "source": [
        "### 4. Train-Test-Tune: model x sub/construct-specific\n",
        "Builds stratified train-test sets, searches hyperparam space to optimize highest-performing sub/construct x pretrained model configs.\n",
        "***"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "H2o-IZ3U6DH6",
      "metadata": {
        "id": "H2o-IZ3U6DH6"
      },
      "outputs": [],
      "source": [
        "%cd ../../code"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "HSbCEWIh7PPw",
      "metadata": {
        "id": "HSbCEWIh7PPw"
      },
      "source": [
        "#### Build train-test sets: $\\mathcal{d}$<sub>train</sub> (rationale-augmented), $\\mathcal{d}$<sub>test</sub>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**_stratified_train_test_split_with_rationales_**"
      ],
      "metadata": {
        "id": "heqwHjFffXbm"
      },
      "id": "heqwHjFffXbm"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "oAqnb49Mzrcx",
      "metadata": {
        "id": "oAqnb49Mzrcx"
      },
      "outputs": [],
      "source": [
        "%%writefile train.py\n",
        "\n",
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "def stratified_train_test_split_with_rationales(df, test_size = 0.2, random_state = 56):\n",
        "    \"\"\"\n",
        "    Splits df into target-stratified train and test sets, partitions 'rationales (aug = 1) to train set\n",
        "    \"\"\"\n",
        "\n",
        "    aug_rows = df[df['aug'] == 1]\n",
        "    non_aug_rows = df[df['aug'] != 1]\n",
        "\n",
        "    train_non_aug, test_non_aug = train_test_split(\n",
        "                                                   non_aug_rows,\n",
        "                                                   test_size = test_size,\n",
        "                                                   stratify = non_aug_rows['targets'],\n",
        "                                                   random_state = random_state,\n",
        "                                                   )\n",
        "\n",
        "    d_train = pd.concat([\n",
        "                         train_non_aug,\n",
        "                         aug_rows,\n",
        "                         ])\n",
        "\n",
        "    print(\"--------------------------------------------------------------------------------------\")\n",
        "    print(f\"d_train: Augmented training data\")\n",
        "    print(\"--------------------------------------------------------------------------------------\")\n",
        "\n",
        "    d_train = d_train.sample(\n",
        "                             frac = 1,\n",
        "                             random_state = random_state,\n",
        "                             ).reset_index(drop = True)\n",
        "\n",
        "    print(d_train.shape)\n",
        "    print(d_train['aug'].value_counts(normalize = True))\n",
        "    print(d_train['targets'].value_counts(normalize = True))\n",
        "    print(d_train.head(6))\n",
        "\n",
        "    d_test = test_non_aug\n",
        "    print(\"\\n--------------------------------------------------------------------------------------\")\n",
        "    print(f\"d_test: De-augmented evaluation data\")\n",
        "    print(\"--------------------------------------------------------------------------------------\")\n",
        "    print(d_test.shape)\n",
        "    print(d_test['aug'].value_counts(normalize = True))\n",
        "    print(d_test['targets'].value_counts(normalize = True))\n",
        "    print(d_test.head(6))\n",
        "\n",
        "    print(\"\\n--------------------------------------------------------------------------------------\")\n",
        "    print(\"Confirming 'aug' deletion\")\n",
        "    print(\"--------------------------------------------------------------------------------------\")\n",
        "    d_train = d_train.drop('aug', axis = 1)\n",
        "    print(d_train.shape)\n",
        "    print(d_train.head(1))\n",
        "\n",
        "    d_test = d_test.drop('aug', axis = 1)\n",
        "    print(d_test.shape)\n",
        "    print(d_test.head(1))\n",
        "\n",
        "    return d_train, d_test"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "vESoG570JAQg",
      "metadata": {
        "id": "vESoG570JAQg"
      },
      "source": [
        "#### Self-hatred: 'hate'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c78701a6-58ec-476e-923e-96ec28b6eb3b",
      "metadata": {
        "id": "c78701a6-58ec-476e-923e-96ec28b6eb3b"
      },
      "outputs": [],
      "source": [
        "# Streamline\n",
        "\n",
        "d = d_augmented[[\n",
        "                 'text',\n",
        "                 'hate',\n",
        "                 'aug',\n",
        "                 ]].copy()\n",
        "\n",
        "d.columns = [\n",
        "             'text',\n",
        "             'targets',\n",
        "             'aug',\n",
        "              ]\n",
        "\n",
        "d.dtypes\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "PC-DoNKTrJiN",
      "metadata": {
        "id": "PC-DoNKTrJiN"
      },
      "outputs": [],
      "source": [
        "from train import stratified_train_test_split_with_rationales\n",
        "\n",
        "d_train, d_test = stratified_train_test_split_with_rationales(d)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "frE9rT0yOgKy",
      "metadata": {
        "id": "frE9rT0yOgKy"
      },
      "outputs": [],
      "source": [
        "%cd ../temp\n",
        "\n",
        "d_train.to_excel('d_train.xlsx')\n",
        "d_test.to_excel('d_test.xlsx')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8p20lYax6V8u",
      "metadata": {
        "id": "8p20lYax6V8u"
      },
      "source": [
        "**Training, hyperparam tuning**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "iaYfaNb0G22_",
      "metadata": {
        "id": "iaYfaNb0G22_"
      },
      "outputs": [],
      "source": [
        "# Define tokenizer, model class, pretrained model\n",
        "\n",
        "tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')\n",
        "model_class = DistilBertForSequenceClassification\n",
        "pretrained_model_name = 'distilbert-base-uncased'\n",
        "\n",
        "# Define label\n",
        "\n",
        "target = 'hate'\n",
        "\n",
        "# Define class weights\n",
        "\n",
        "class_weights = torch.tensor(\n",
        "                             [\n",
        "                              0.6416,  # neg_0\n",
        "                              2.2646,  # pos_1\n",
        "                              ], dtype=torch.float,\n",
        "                              )\n",
        "\n",
        "# Define save_path\n",
        "\n",
        "%cd ../outputs/models\n",
        "%mkdir hate_distilbert\n",
        "\n",
        "save_path = '/content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models/hate_distilbert'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Y9HZ4HdVFY7g",
      "metadata": {
        "id": "Y9HZ4HdVFY7g"
      },
      "outputs": [],
      "source": [
        "#%cd ../../code"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**_tune_and_optimize_model_hyperparams_**"
      ],
      "metadata": {
        "id": "M4dYSw-mfeJk"
      },
      "id": "M4dYSw-mfeJk"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "TvTRbDh2SXeh",
      "metadata": {
        "id": "TvTRbDh2SXeh"
      },
      "outputs": [],
      "source": [
        "%%writefile -a train.py\n",
        "\n",
        "import torch\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from torch.nn import CrossEntropyLoss\n",
        "from transformers import BertForSequenceClassification, AdamW\n",
        "from sklearn.metrics import f1_score\n",
        "from sklearn.model_selection import ParameterGrid\n",
        "from tqdm import tqdm\n",
        "\n",
        "# Define Fx\n",
        "\n",
        "def tune_and_optimize_model_hyperparams(tokenizer, model_class, pretrained_model_name, d_train, d_test, target, class_weights, save_path):\n",
        "    \"\"\"\n",
        "    Tune and optimize model hyperparameters.\n",
        "\n",
        "    Parameters:\n",
        "    tokenizer: Pre-trained tokenizer.\n",
        "    model_class: Pre-trained model class.\n",
        "    pretrained_model_name: Name of the pre-trained model.\n",
        "    d_train: Training dataset.\n",
        "    d_test: Test dataset.\n",
        "    target: Target variable for classification.\n",
        "    class_weights: Weights for each class.\n",
        "    save_path: Path to save the best model.\n",
        "\n",
        "    Returns:\n",
        "    d_test: Test dataset with predictions and probabilities.\n",
        "    \"\"\"\n",
        "\n",
        "    # Check CUDA\n",
        "\n",
        "    use_cuda = torch.cuda.is_available()\n",
        "    print(\"CUDA: \", use_cuda)\n",
        "\n",
        "    print(\"======================================================================================\")\n",
        "    print(f\"Optimizing: {pretrained_model_name}\\nLabel: {target}\")\n",
        "    print(\"======================================================================================\")\n",
        "\n",
        "    # Define hyperparam grid\n",
        "\n",
        "    param_grid = {\n",
        "        'per_gpu_batch_size': [8, 16],\n",
        "        'weight_decay': [0, 0.3],\n",
        "        'learning_rate': [2e-5, 3e-5, 5e-5],\n",
        "        'warmup_steps': [0, 500],\n",
        "        'num_epochs': [2, 3]\n",
        "    }\n",
        "\n",
        "    # Class weights\n",
        "\n",
        "    if use_cuda:\n",
        "        class_weights = class_weights.cuda()\n",
        "\n",
        "    # Tokenize\n",
        "\n",
        "    encoded_train = tokenizer(\n",
        "        d_train['text'].tolist(),\n",
        "        padding=True,\n",
        "        truncation=True,\n",
        "        return_tensors='pt'\n",
        "    )\n",
        "    encoded_test = tokenizer(\n",
        "        d_test['text'].tolist(),\n",
        "        padding=True,\n",
        "        truncation=True,\n",
        "        return_tensors='pt'\n",
        "    )\n",
        "\n",
        "    train_labels = torch.tensor(d_train['targets'].values)\n",
        "    test_labels = torch.tensor(d_test['targets'].values)\n",
        "\n",
        "    train_dataset = TensorDataset(\n",
        "        encoded_train['input_ids'],\n",
        "        encoded_train['attention_mask'],\n",
        "        train_labels\n",
        "    )\n",
        "    test_dataset = TensorDataset(\n",
        "        encoded_test['input_ids'],\n",
        "        encoded_test['attention_mask'],\n",
        "        test_labels\n",
        "    )\n",
        "\n",
        "    # Initialize tracking variables\n",
        "\n",
        "    best_f1_macro = 0\n",
        "    best_params = None\n",
        "    best_model_state = None\n",
        "    best_predictions = []\n",
        "    best_probabilities = []\n",
        "\n",
        "    f1_scores = []\n",
        "\n",
        "    # Gradient accumulation steps\n",
        "\n",
        "    accumulation_steps = 2\n",
        "\n",
        "    # Grid search\n",
        "\n",
        "    for params in ParameterGrid(param_grid):\n",
        "        train_loader = DataLoader(\n",
        "            train_dataset,\n",
        "            batch_size=params['per_gpu_batch_size'],\n",
        "            shuffle=True\n",
        "        )\n",
        "        test_loader = DataLoader(\n",
        "            test_dataset,\n",
        "            batch_size=params['per_gpu_batch_size'],\n",
        "            shuffle=False\n",
        "        )\n",
        "\n",
        "        print(f\"\\nTotal training rows: {len(train_dataset)}\")\n",
        "        print(f\"Total evaluation rows: {len(test_dataset)}\")\n",
        "        print(f\"Training batch size: {params['per_gpu_batch_size']}\")\n",
        "        print(f\"Evaluation batch size: {params['per_gpu_batch_size']}\")\n",
        "        print(f\"Total training batches: {len(train_loader)}\")\n",
        "        print(f\"Total evaluation batches: {len(test_loader)}\")\n",
        "        print(\"\\n\")\n",
        "\n",
        "        model = model_class.from_pretrained(pretrained_model_name)\n",
        "        if use_cuda:\n",
        "            model.cuda()\n",
        "\n",
        "        optimizer = AdamW(\n",
        "            model.parameters(),\n",
        "            lr=params['learning_rate'],\n",
        "            weight_decay=params['weight_decay']\n",
        "        )\n",
        "\n",
        "        criterion = CrossEntropyLoss(weight=class_weights)\n",
        "\n",
        "        # Training loop\n",
        "\n",
        "        for epoch in range(params['num_epochs']):\n",
        "            model.train()\n",
        "            optimizer.zero_grad()\n",
        "            for i, batch in enumerate(tqdm(train_loader, desc=f\"Training Epoch {epoch + 1}/{params['num_epochs']}\", leave=True)):\n",
        "                input_ids, attention_mask, labels = batch\n",
        "                if use_cuda:\n",
        "                    input_ids, attention_mask, labels = input_ids.cuda(), attention_mask.cuda(), labels.cuda()\n",
        "                outputs = model(\n",
        "                    input_ids,\n",
        "                    attention_mask=attention_mask\n",
        "                )\n",
        "                loss = criterion(outputs.logits, labels)\n",
        "                loss = loss / accumulation_steps\n",
        "                loss.backward()\n",
        "                if (i + 1) % accumulation_steps == 0:\n",
        "                    optimizer.step()\n",
        "                    optimizer.zero_grad()\n",
        "\n",
        "        # Eval loop\n",
        "\n",
        "        model.eval()\n",
        "        all_predictions = []\n",
        "        all_true_labels = []\n",
        "        all_probabilities = []\n",
        "        with torch.no_grad():\n",
        "            progress_bar = tqdm(total=len(test_loader), desc=\"Evaluating\", leave=True)\n",
        "            for batch in test_loader:\n",
        "                input_ids, attention_mask, labels = batch\n",
        "                if use_cuda:\n",
        "                    input_ids, attention_mask, labels = input_ids.cuda(), attention_mask.cuda(), labels.cuda()\n",
        "                outputs = model(\n",
        "                    input_ids,\n",
        "                    attention_mask=attention_mask\n",
        "                )\n",
        "                probabilities = torch.softmax(outputs.logits, dim=1)\n",
        "                predictions = torch.argmax(probabilities, dim=1).cpu().tolist()\n",
        "                all_predictions.extend(predictions)\n",
        "                all_true_labels.extend(labels.cpu().tolist())\n",
        "                all_probabilities.extend(probabilities.cpu().tolist())\n",
        "                progress_bar.update(1)\n",
        "            progress_bar.close()\n",
        "\n",
        "        current_f1_macro = f1_score(all_true_labels, all_predictions, average='macro')\n",
        "        f1_scores.append(current_f1_macro)\n",
        "        print(f\"\\nCurrent F1 macro for params {params}: {current_f1_macro}\")\n",
        "\n",
        "        if current_f1_macro > best_f1_macro:\n",
        "            best_f1_macro = current_f1_macro\n",
        "            best_params = params\n",
        "            best_model_state = model.state_dict()\n",
        "            best_predictions = all_predictions\n",
        "            best_probabilities = all_probabilities\n",
        "\n",
        "    if len(best_predictions) == len(d_test):\n",
        "        d_test['predicted_labels'] = best_predictions\n",
        "        d_test['predicted_probabilities'] = best_probabilities\n",
        "    else:\n",
        "        print(\"Error: Length of predictions does not match length of test set\")\n",
        "\n",
        "    # Output test df with predictions and raw probabilities\n",
        "\n",
        "    print(\"--------------------------------------------------------------------------------------\")\n",
        "    print(f\"Summary: {target}\")\n",
        "    print(\"--------------------------------------------------------------------------------------\")\n",
        "\n",
        "    print(d_test.head(6))\n",
        "    d_test.to_excel('d_test_preds.xlsx')\n",
        "\n",
        "    if best_model_state:\n",
        "        model_path = f\"{save_path}/{target}_best_model.bin\"\n",
        "        torch.save(best_model_state, model_path)\n",
        "        print(\"\\nBest model saved with F1 macro:\", best_f1_macro)\n",
        "        print(\"Best hyperparameters:\", best_params)\n",
        "\n",
        "    # Print M (SD) of F_1 scores\n",
        "\n",
        "    f1_mean = sum(f1_scores) / len(f1_scores)\n",
        "    f1_std = (sum((x - f1_mean) ** 2 for x in f1_scores) / len(f1_scores)) ** 0.5\n",
        "    print(f\"Mean F1 macro: {f1_mean}\")\n",
        "    print(f\"Standard deviation of F1 macro: {f1_std}\")\n",
        "\n",
        "    return d_test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "_plt67dcS9eN",
      "metadata": {
        "collapsed": true,
        "id": "_plt67dcS9eN"
      },
      "outputs": [],
      "source": [
        "%cd ../../code\n",
        "\n",
        "from train import tune_and_optimize_model_hyperparams\n",
        "\n",
        "d_test = tune_and_optimize_model_hyperparams(tokenizer, model_class, pretrained_model_name, d_train, d_test, target, class_weights, save_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "gFjfDPj8FBE-",
      "metadata": {
        "id": "gFjfDPj8FBE-"
      },
      "source": [
        "#### Loneliness: 'lone'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b9bf9884-2923-40f8-9194-87e8ab4558cf",
      "metadata": {
        "id": "b9bf9884-2923-40f8-9194-87e8ab4558cf"
      },
      "outputs": [],
      "source": [
        "# Streamline\n",
        "\n",
        "d = d_augmented[[\n",
        "                 'text',\n",
        "                 'lone',\n",
        "                 'aug',\n",
        "                 ]].copy()\n",
        "\n",
        "d.columns = [\n",
        "             'text',\n",
        "             'targets',\n",
        "             'aug',\n",
        "              ]\n",
        "\n",
        "d.dtypes\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "BtC9pqQNRLaH",
      "metadata": {
        "id": "BtC9pqQNRLaH"
      },
      "outputs": [],
      "source": [
        "#from train import stratified_train_test_split_with_rationales\n",
        "\n",
        "d_train, d_test = stratified_train_test_split_with_rationales(d)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "LyfuGUXgcoyS",
      "metadata": {
        "id": "LyfuGUXgcoyS"
      },
      "outputs": [],
      "source": [
        "%cd ../temp\n",
        "\n",
        "d_train.to_excel('d_train.xlsx')\n",
        "d_test.to_excel('d_test.xlsx')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "kILE4dr4RLdo",
      "metadata": {
        "id": "kILE4dr4RLdo"
      },
      "outputs": [],
      "source": [
        "# Define tokenizer, model class, pretrained model\n",
        "\n",
        "tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')\n",
        "model_class = DistilBertForSequenceClassification\n",
        "pretrained_model_name = 'distilbert-base-uncased'\n",
        "\n",
        "# Define label\n",
        "\n",
        "target = 'lone'\n",
        "\n",
        "# Define class weights\n",
        "\n",
        "class_weights = torch.tensor(\n",
        "                             [\n",
        "                              0.6911,  # neg_0\n",
        "                              1.8084,  # pos_1\n",
        "                              ], dtype = torch.float,\n",
        "                             )\n",
        "\n",
        "# Define save_path\n",
        "\n",
        "%cd ../outputs/models\n",
        "%mkdir lone_distilbert\n",
        "\n",
        "save_path = '/content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models/lone_distilbert'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "UdlEd9Wub3NX",
      "metadata": {
        "id": "UdlEd9Wub3NX"
      },
      "outputs": [],
      "source": [
        "%cd lone_distilbert\n",
        "\n",
        "d_test = tune_and_optimize_model_hyperparams(tokenizer, model_class, pretrained_model_name, d_train, d_test, target, class_weights, save_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "FFUab1oIfqTe",
      "metadata": {
        "id": "FFUab1oIfqTe"
      },
      "source": [
        "#### Thwarted belongingness: 'tb'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cEjr1JG1RLv6",
      "metadata": {
        "id": "cEjr1JG1RLv6"
      },
      "outputs": [],
      "source": [
        "# Streamline\n",
        "\n",
        "d = d_augmented[[\n",
        "                 'text',\n",
        "                 'tb',\n",
        "                 'aug',\n",
        "                 ]].copy()\n",
        "\n",
        "d.columns = [\n",
        "             'text',\n",
        "             'targets',\n",
        "             'aug',\n",
        "              ]\n",
        "\n",
        "d.dtypes\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "yaHq5EJTRLzx",
      "metadata": {
        "id": "yaHq5EJTRLzx"
      },
      "outputs": [],
      "source": [
        "d_train, d_test = stratified_train_test_split_with_rationales(d)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "piwplrIkU0Nh",
      "metadata": {
        "id": "piwplrIkU0Nh"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "GAIOl9JpRL3S",
      "metadata": {
        "id": "GAIOl9JpRL3S"
      },
      "outputs": [],
      "source": [
        "# Define tokenizer, model class, pretrained model\n",
        "\n",
        "tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",
        "model_class = BertForSequenceClassification\n",
        "pretrained_model_name = 'bert-base-uncased'\n",
        "\n",
        "# Define label\n",
        "\n",
        "target = 'tb'\n",
        "\n",
        "# Define class weights\n",
        "\n",
        "class_weights = torch.tensor(\n",
        "                             [\n",
        "                              0.7281,  # neg_0\n",
        "                              1.5960,  # pos_1\n",
        "                              ], dtype = torch.float,\n",
        "                             )\n",
        "\n",
        "# Define save_path\n",
        "\n",
        "#%cd ../outputs/models\n",
        "#%mkdir tb_bert\n",
        "\n",
        "save_path = '/content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models/tb_bert'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "vthQvrh3dXmA",
      "metadata": {
        "id": "vthQvrh3dXmA"
      },
      "outputs": [],
      "source": [
        "%cd tb_bert\n",
        "\n",
        "d_test = tune_and_optimize_model_hyperparams(tokenizer, model_class, pretrained_model_name, d_train, d_test, target, class_weights, save_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "iEIKXGHPxsmh",
      "metadata": {
        "id": "iEIKXGHPxsmh"
      },
      "source": [
        "#### Perceived burdensomeness: 'pb'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "SazKj0gxhPFw",
      "metadata": {
        "id": "SazKj0gxhPFw"
      },
      "outputs": [],
      "source": [
        "# Streamline\n",
        "\n",
        "d = d_augmented[[\n",
        "                 'text',\n",
        "                 'pb',\n",
        "                 'aug',\n",
        "                 ]].copy()\n",
        "\n",
        "d.columns = [\n",
        "             'text',\n",
        "             'targets',\n",
        "             'aug',\n",
        "              ]\n",
        "\n",
        "d.dtypes\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "eByCV7Ptxqh1",
      "metadata": {
        "id": "eByCV7Ptxqh1"
      },
      "outputs": [],
      "source": [
        "from train import stratified_train_test_split_with_rationales\n",
        "\n",
        "d_train, d_test = stratified_train_test_split_with_rationales(d)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7xZJPFMCyD11",
      "metadata": {
        "id": "7xZJPFMCyD11"
      },
      "outputs": [],
      "source": [
        "# Define tokenizer, model class, pretrained model\n",
        "\n",
        "tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",
        "model_class = BertForSequenceClassification\n",
        "pretrained_model_name = 'bert-base-uncased'\n",
        "\n",
        "# Define label\n",
        "\n",
        "target = 'pb'\n",
        "\n",
        "# Define class weights\n",
        "\n",
        "class_weights = torch.tensor(\n",
        "                             [\n",
        "                              0.6735,  # neg_0\n",
        "                              1.9411,  # pos_1\n",
        "                              ], dtype = torch.float,\n",
        "                             )\n",
        "\n",
        "# Define save_path\n",
        "\n",
        "%cd ../outputs/models\n",
        "%mkdir pb_bert\n",
        "\n",
        "save_path = '/content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models/pb_bert'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "IwI3QRZTxqvJ",
      "metadata": {
        "id": "IwI3QRZTxqvJ"
      },
      "outputs": [],
      "source": [
        "%cd pb_bert\n",
        "\n",
        "d_test = tune_and_optimize_model_hyperparams(tokenizer, model_class, pretrained_model_name, d_train, d_test, target, class_weights, save_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6ALab8tymeX6",
      "metadata": {
        "id": "6ALab8tymeX6"
      },
      "source": [
        "### 5. Predict\n",
        "Defines functions to tokenize and label $\\mathcal{D}$<sub>inference</sub>, $\\mathcal{D}$<sub>reflexive</sub> posts with classification models. Predicts labels.\n",
        "***"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "UD4DByMTUVKH",
      "metadata": {
        "id": "UD4DByMTUVKH"
      },
      "source": [
        "**_ner_redact_post_texts_**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "RWcWvA5GZcx3",
      "metadata": {
        "id": "RWcWvA5GZcx3"
      },
      "outputs": [],
      "source": [
        "%cd ../../code"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "APAihJrOUTzi",
      "metadata": {
        "id": "APAihJrOUTzi"
      },
      "outputs": [],
      "source": [
        "%%writefile predict.py\n",
        "\n",
        "import spacy\n",
        "nlp = spacy.load('en_core_web_lg')\n",
        "\n",
        "def ner_redact_post_texts(p_text):\n",
        "    \"\"\"\n",
        "    Redacts all named entities recognized by spaCy EntityRecognizer, replaces with <|PII|> pseudo-word token.\n",
        "    \"\"\"\n",
        "    ne = list(\n",
        "              [\n",
        "               'PERSON',   ### people, including fictional\n",
        "               'NORP',     ### nationalities or religious or political groups\n",
        "               'FAC',      ### buildings, airports, highways, bridges, etc.\n",
        "               'ORG',      ### companies, agencies, institutions, etc.\n",
        "               'GPE',     ### countries, cities, states\n",
        "               'LOC',      ### non-GPE locations, mountain ranges, bodies of water\n",
        "               'PRODUCT',  ### objects, vehicles, foods, etc. (not services)\n",
        "               'EVENT',    ### named hurricanes, battles, wars, sports events, etc.\n",
        "               ]\n",
        "                )\n",
        "\n",
        "    doc = nlp(p_text)\n",
        "    ne_to_remove = []\n",
        "    final_string = str(p_text)\n",
        "    for sent in doc.ents:\n",
        "        if sent.label_ in ne:\n",
        "            ne_to_remove.append(str(sent.text))\n",
        "    for n in range(len(ne_to_remove)):\n",
        "        final_string = final_string.replace(\n",
        "                                            ne_to_remove[n],\n",
        "                                            '<|PII|>',\n",
        "                                            )\n",
        "    return final_string"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "QRpNvwZLUJMF",
      "metadata": {
        "id": "QRpNvwZLUJMF"
      },
      "source": [
        "**_load_model_**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "LrdSLS8Nam7A",
      "metadata": {
        "id": "LrdSLS8Nam7A"
      },
      "outputs": [],
      "source": [
        "%%writefile -a predict.py\n",
        "\n",
        "import torch\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from transformers import DistilBertTokenizer, DistilBertForSequenceClassification, BertTokenizer, BertForSequenceClassification, RobertaTokenizer, RobertaForSequenceClassification\n",
        "from tqdm import tqdm\n",
        "import pandas as pd\n",
        "\n",
        "def load_model(model_path, model_class, pretrained_model_name):\n",
        "    \"\"\"\n",
        "    Loads a pre-trained fine-tined LM from a specified path.\n",
        "    \"\"\"\n",
        "    model = model_class.from_pretrained(pretrained_model_name)\n",
        "    model.load_state_dict(torch.load(model_path))\n",
        "    model.eval()\n",
        "    return model"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**_preprocess_data_**"
      ],
      "metadata": {
        "id": "BZZw9lgPd4rC"
      },
      "id": "BZZw9lgPd4rC"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "kbGizM_Tam0R",
      "metadata": {
        "id": "kbGizM_Tam0R"
      },
      "outputs": [],
      "source": [
        "%%writefile -a predict.py\n",
        "\n",
        "import torch\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from transformers import DistilBertTokenizer, DistilBertForSequenceClassification, BertTokenizer, BertForSequenceClassification, RobertaTokenizer, RobertaForSequenceClassification\n",
        "from tqdm import tqdm\n",
        "import pandas as pd\n",
        "\n",
        "def preprocess_data(tokenizer, texts):\n",
        "    \"\"\"\n",
        "    Tokenizes a list of texts using the specified LM-specific tokenizer.\n",
        "    \"\"\"\n",
        "    encoded_texts = tokenizer(\n",
        "        texts,\n",
        "        padding=True,\n",
        "        truncation=True,\n",
        "        return_tensors='pt'\n",
        "    )\n",
        "    return encoded_texts"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**_predict_**"
      ],
      "metadata": {
        "id": "F-nqVtmpd-22"
      },
      "id": "F-nqVtmpd-22"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "kspFi-G_ampu",
      "metadata": {
        "id": "kspFi-G_ampu"
      },
      "outputs": [],
      "source": [
        "%%writefile -a predict.py\n",
        "\n",
        "import torch\n",
        "from torch.utils.data import DataLoader, TensorDataset\n",
        "from transformers import DistilBertTokenizer, DistilBertForSequenceClassification, BertTokenizer, BertForSequenceClassification, RobertaTokenizer, RobertaForSequenceClassification\n",
        "from tqdm import tqdm\n",
        "import pandas as pd\n",
        "\n",
        "def predict(model, tokenizer, texts, batch_size = 8, use_cuda = True):\n",
        "    \"\"\"\n",
        "    Predicts labels and probabilities for a list of texts using the specified model and tokenizer.\n",
        "    \"\"\"\n",
        "    print(f\"Total number of texts to predict: {len(texts)}\")\n",
        "    encoded_texts = preprocess_data(tokenizer, texts)\n",
        "    dataset = TensorDataset(encoded_texts['input_ids'], encoded_texts['attention_mask'])\n",
        "    data_loader = DataLoader(dataset, batch_size=batch_size, shuffle=False)\n",
        "\n",
        "    print(f\"Batch size: {batch_size}\")\n",
        "    print(f\"Total number of batches: {len(data_loader)}\")\n",
        "\n",
        "    if use_cuda:\n",
        "        model.cuda()\n",
        "\n",
        "    all_predictions = []\n",
        "    all_probabilities = []\n",
        "\n",
        "    with torch.no_grad():\n",
        "        progress_bar = tqdm(total=len(data_loader), desc=\"Predicting\", leave=False)\n",
        "        for batch in data_loader:\n",
        "            input_ids, attention_mask = batch\n",
        "            if use_cuda:\n",
        "                input_ids, attention_mask = input_ids.cuda(), attention_mask.cuda()\n",
        "\n",
        "            outputs = model(input_ids, attention_mask=attention_mask)\n",
        "            probabilities = torch.softmax(outputs.logits, dim=1)\n",
        "            predictions = torch.argmax(probabilities, dim=1).cpu().tolist()\n",
        "            all_predictions.extend(predictions)\n",
        "            all_probabilities.extend(probabilities.cpu().tolist())\n",
        "            progress_bar.update(1)\n",
        "        progress_bar.close()\n",
        "\n",
        "    return all_predictions, all_probabilities"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Qnr7ZfLToJBr",
      "metadata": {
        "collapsed": true,
        "id": "Qnr7ZfLToJBr"
      },
      "outputs": [],
      "source": [
        "%cd ../inputs/data\n",
        "\n",
        "#d_reflexive = pd.read_excel('d_reflexive.xlsx')\n",
        "\n",
        "#d_reflexive.shape\n",
        "#d_reflexive.dtypes\n",
        "#d_reflexive.head(3)\n",
        "\n",
        "d_inference = pd.read_csv('d_inference.csv')\n",
        "\n",
        "d_inference.shape\n",
        "d_inference.dtypes\n",
        "d_inference.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "nw5gnrs8dlhd",
      "metadata": {
        "id": "nw5gnrs8dlhd"
      },
      "outputs": [],
      "source": [
        "%cd ../../code\n",
        "\n",
        "from predict import ner_redact_post_texts, load_model, preprocess_data, predict\n",
        "\n",
        "d_inference = d_inference.drop('Unnamed: 0', axis = 1)\n",
        "d_inference = d_inference.reset_index(drop = True)\n",
        "\n",
        "d_inference['text'] = d_inference['text'].astype(str)\n",
        "\n",
        "d_inference.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "VK3_JDeNXJ6l",
      "metadata": {
        "collapsed": true,
        "id": "VK3_JDeNXJ6l"
      },
      "outputs": [],
      "source": [
        "d_reflexive = d_reflexive.drop('Unnamed: 0', axis = 1)\n",
        "d_reflexive = d_reflexive.reset_index(drop = True)\n",
        "\n",
        "d_reflexive['text'] = d_reflexive['text'].astype(str).apply(lambda i: ner_redact_post_texts(i))\n",
        "\n",
        "d_reflexive.head(3)\n",
        "\n",
        "    ### SJS 8/9: move all this housekeeping shit into the Preprocess section..."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Mw7NBQc5akD-",
      "metadata": {
        "id": "Mw7NBQc5akD-"
      },
      "source": [
        "**'hate'**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Pb02Qp2QkBeL",
      "metadata": {
        "id": "Pb02Qp2QkBeL"
      },
      "outputs": [],
      "source": [
        "%pwd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "H9RDVuvzadsc",
      "metadata": {
        "collapsed": true,
        "id": "H9RDVuvzadsc"
      },
      "outputs": [],
      "source": [
        "#from predict import load_model, predict\n",
        "\n",
        "models_path = '/content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models/'\n",
        "\n",
        "pretrained_model_name = 'distilbert-base-uncased'\n",
        "model_class = DistilBertForSequenceClassification\n",
        "tokenizer_class = DistilBertTokenizer\n",
        "model_path = f'{models_path}hate_distilbert/hate_best_model.bin'\n",
        "\n",
        "target = 'hate'\n",
        "\n",
        "tokenizer = tokenizer_class.from_pretrained(pretrained_model_name)\n",
        "\n",
        "model = load_model(\n",
        "                   model_path,\n",
        "                   model_class,\n",
        "                   pretrained_model_name,\n",
        "                   )\n",
        "\n",
        "#texts = d_reflexive['text'].tolist()\n",
        "texts = d_inference['text'].tolist()\n",
        "#d_inference['text'] = d_inference['text'].astype(str)\n",
        "\n",
        "predictions, probabilities = predict(\n",
        "                                     model,\n",
        "                                     tokenizer,\n",
        "                                     texts,\n",
        "                                     )\n",
        "\n",
        "#d_reflexive[f'{target}_label'] = predictions\n",
        "#d_reflexive[f'{target}_proba'] = probabilities\n",
        "d_inference[f'{target}_label'] = predictions\n",
        "d_inference[f'{target}_proba'] = probabilities\n",
        "\n",
        "#d_reflexive.head(3)\n",
        "d_inference.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "nKK5TACZapWF",
      "metadata": {
        "id": "nKK5TACZapWF"
      },
      "source": [
        "**'lone'**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "u814D1tqadku",
      "metadata": {
        "id": "u814D1tqadku"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/inputs/data\n",
        "\n",
        "pretrained_model_name = 'distilbert-base-uncased'\n",
        "model_class = DistilBertForSequenceClassification\n",
        "tokenizer_class = DistilBertTokenizer\n",
        "model_path = f'{models_path}lone_distilbert/lone_best_model.bin'\n",
        "\n",
        "target = 'lone'\n",
        "\n",
        "tokenizer = tokenizer_class.from_pretrained(pretrained_model_name)\n",
        "\n",
        "model = load_model(\n",
        "                   model_path,\n",
        "                   model_class,\n",
        "                   pretrained_model_name,\n",
        "                   )\n",
        "\n",
        "#texts = d_reflexive['text'].tolist()\n",
        "texts = d_inference['text'].tolist()\n",
        "\n",
        "predictions, probabilities = predict(\n",
        "                                     model,\n",
        "                                     tokenizer,\n",
        "                                     texts,\n",
        "                                     )\n",
        "\n",
        "#d_reflexive[f'{target}_label'] = predictions\n",
        "#d_reflexive[f'{target}_proba'] = probabilities\n",
        "d_inference[f'{target}_label'] = predictions\n",
        "d_inference[f'{target}_proba'] = probabilities\n",
        "\n",
        "#d_reflexive.head(3)\n",
        "d_inference.head(3)\n",
        "\n",
        "d_inference.to_csv('d_inference_hate_lone.csv')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rFd5oIu_atG8",
      "metadata": {
        "id": "rFd5oIu_atG8"
      },
      "source": [
        "**'tb'**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "IY7jcN-Sadbi",
      "metadata": {
        "collapsed": true,
        "id": "IY7jcN-Sadbi"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/inputs/data\n",
        "\n",
        "pretrained_model_name = 'bert-base-uncased'\n",
        "model_class = BertForSequenceClassification\n",
        "tokenizer_class = BertTokenizer\n",
        "model_path = f'{models_path}tb_bert/tb_best_model.bin'\n",
        "\n",
        "target = 'tb'\n",
        "\n",
        "tokenizer = tokenizer_class.from_pretrained(pretrained_model_name)\n",
        "\n",
        "model = load_model(\n",
        "                   model_path,\n",
        "                   model_class,\n",
        "                   pretrained_model_name,\n",
        "                   )\n",
        "\n",
        "#texts = d_reflexive['text'].tolist()\n",
        "texts = d_inference['text'].tolist()\n",
        "\n",
        "predictions, probabilities = predict(\n",
        "                                     model,\n",
        "                                     tokenizer,\n",
        "                                     texts,\n",
        "                                     )\n",
        "\n",
        "#d_reflexive[f'{target}_label'] = predictions\n",
        "#d_reflexive[f'{target}_proba'] = probabilities\n",
        "d_inference[f'{target}_label'] = predictions\n",
        "d_inference[f'{target}_proba'] = probabilities\n",
        "\n",
        "#d_reflexive.head(3)\n",
        "d_inference.head(3)\n",
        "\n",
        "d_inference.to_csv('d_inference_hate_lone_tb.csv')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "-efbZY_MawKK",
      "metadata": {
        "id": "-efbZY_MawKK"
      },
      "source": [
        "**'pb'**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Npo9jV_uzaBF",
      "metadata": {
        "id": "Npo9jV_uzaBF"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/inputs/data\n",
        "\n",
        "d_inference = pd.read_csv('d_inference_hate_lone_tb.csv')\n",
        "d_inference.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c0zmJPDczmZd",
      "metadata": {
        "id": "c0zmJPDczmZd"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/code\n",
        "\n",
        "from predict import load_model, predict"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "1dU2AUW3hPPR",
      "metadata": {
        "id": "1dU2AUW3hPPR"
      },
      "outputs": [],
      "source": [
        "models_path = '/content/gdrive/My Drive/Colab/intrapolitical_suicidality/outputs/models/'\n",
        "pretrained_model_name = 'bert-base-uncased'\n",
        "model_class = BertForSequenceClassification\n",
        "tokenizer_class = BertTokenizer\n",
        "model_path = f'{models_path}pb_bert/pb_best_model.bin'\n",
        "\n",
        "target = 'pb'\n",
        "\n",
        "tokenizer = tokenizer_class.from_pretrained(pretrained_model_name)\n",
        "\n",
        "model = load_model(\n",
        "                   model_path,\n",
        "                   model_class,\n",
        "                   pretrained_model_name,\n",
        "                   )\n",
        "\n",
        "#texts = d_reflexive['text'].tolist()\n",
        "texts = d_inference['text'].tolist()\n",
        "\n",
        "predictions, probabilities = predict(\n",
        "                                     model,\n",
        "                                     tokenizer,\n",
        "                                     texts,\n",
        "                                     )\n",
        "\n",
        "#d_reflexive[f'{target}_label'] = predictions\n",
        "#d_reflexive[f'{target}_proba'] = probabilities\n",
        "d_inference[f'{target}_label'] = predictions\n",
        "d_inference[f'{target}_proba'] = probabilities\n",
        "\n",
        "#d_reflexive.head(3)\n",
        "d_inference.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "qffiQyKUhPTH",
      "metadata": {
        "collapsed": true,
        "id": "qffiQyKUhPTH"
      },
      "outputs": [],
      "source": [
        "%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/inputs/data\n",
        "\n",
        "#d_reflexive.to_excel('d_reflexive_labeled.xlsx')\n",
        "d_inference.to_csv('d_inference_labeled.csv')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8-MPUSq8J-Vv",
      "metadata": {
        "id": "8-MPUSq8J-Vv"
      },
      "source": [
        "### 6. Causal inference\n",
        "Transforms $\\mathcal{D}$<sub>inference</sub> for causal inference modeling. Collapses analytic sample to _block_ units, concatenates text covariate _$W_{p}$_, computes aggregate sentiment outcome _$Y_{p}$_. Fits intra-textual causal inference Models A and B.\n",
        "***"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%pwd"
      ],
      "metadata": {
        "id": "_3ULS8hzQwqe"
      },
      "id": "_3ULS8hzQwqe",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Autocode: comment sentiment"
      ],
      "metadata": {
        "id": "BN1hYHKPQzsL"
      },
      "id": "BN1hYHKPQzsL"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6Fgds5FGFj0N",
      "metadata": {
        "id": "6Fgds5FGFj0N"
      },
      "outputs": [],
      "source": [
        "d = pd.read_csv('d_inference_labeled.csv')\n",
        "\n",
        "d.set_index(\n",
        "             'index',\n",
        "             inplace = True,\n",
        "             )\n",
        "\n",
        "d.shape\n",
        "d[[\n",
        "   'hate_label',\n",
        "   'lone_label',\n",
        "   'tb_label',\n",
        "   'pb_label',\n",
        "   ]].apply(pd.Series.value_counts)\n",
        "\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "RNP6XRmHM0sb",
      "metadata": {
        "id": "RNP6XRmHM0sb"
      },
      "source": [
        "**Count: _n_ unique posts**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5JDkZjHhFj4Y",
      "metadata": {
        "id": "5JDkZjHhFj4Y"
      },
      "outputs": [],
      "source": [
        "d['p_uniq'] = (d['text'] != '<|RPT|>').astype(int)\n",
        "d['p_uniq'].value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "Ro8T70cdM8iy",
      "metadata": {
        "id": "Ro8T70cdM8iy"
      },
      "source": [
        "**Disaggregate: post author | commenter comments**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "LOrU_LVYFj-C",
      "metadata": {
        "collapsed": true,
        "id": "LOrU_LVYFj-C"
      },
      "outputs": [],
      "source": [
        "# Dupe c_text col\n",
        "\n",
        "d['p_au_c_text'] = d['c_text']\n",
        "\n",
        "# Dupe c_text text\n",
        "\n",
        "d.loc[d['p_au_reply'] == 1, 'p_au_c_text'] = d.loc[d['p_au_reply'] == 1, 'c_text']\n",
        "\n",
        "# Disaggregate about p_author_reply = 1/0\n",
        "\n",
        "d.loc[d['p_au_reply'] == 1, 'c_text'] = ' '\n",
        "d.loc[d['p_au_reply'] != 1, 'p_au_c_text'] = ' '\n",
        "\n",
        "d['p_uniq'].value_counts()\n",
        "d.head(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "AYkpONdUFkDR",
      "metadata": {
        "id": "AYkpONdUFkDR"
      },
      "outputs": [],
      "source": [
        "# Force to str prior to autocoding\n",
        "\n",
        "d['c_text'] = d['c_text'].astype(str)\n",
        "d['p_au_c_text'] = d['p_au_c_text'].astype(str)\n",
        "\n",
        "# Reduce\n",
        "\n",
        "d = d[[\n",
        "       'p_au',\n",
        "       'p_date',\n",
        "       'id',\n",
        "       'n_cmnt',\n",
        "       'text',\n",
        "       'p_sbrt',\n",
        "       'p_titl',\n",
        "       'c_au',\n",
        "       'c_sbrt',\n",
        "       'c_text',\n",
        "       'new_block',\n",
        "       'block',\n",
        "       'p_au_reply',\n",
        "       'hate_label',\n",
        "       'lone_label',\n",
        "       'tb_label',\n",
        "       'pb_label',\n",
        "       'p_uniq',\n",
        "       'p_au_c_text'\n",
        "       ]].copy()\n",
        "\n",
        "d.dtypes\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Autocode**"
      ],
      "metadata": {
        "id": "F1Efxf7oRD9C"
      },
      "id": "F1Efxf7oRD9C"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "p1rXXGXJPjsH",
      "metadata": {
        "id": "p1rXXGXJPjsH"
      },
      "outputs": [],
      "source": [
        "ac = Autocoder()\n",
        "\n",
        "d = ac.code_sentiment(\n",
        "                      d['c_text'].values,\n",
        "                      d,\n",
        "                      batch_size = 8,\n",
        "                      binarize = False,\n",
        "                      #threshold = 0.5,\n",
        "                      )\n",
        "\n",
        "d.head(30)\n",
        "d.to_csv('d_inference_labeled_autocoded.csv')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Compute outcomes, concatenate text covariates"
      ],
      "metadata": {
        "id": "7nCxayebQcb5"
      },
      "id": "7nCxayebQcb5"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "MfckYhRPPj09",
      "metadata": {
        "id": "MfckYhRPPj09",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "# import\n",
        "\n",
        "d = pd.read_csv('d_inference_labeled_autocoded.csv')\n",
        "\n",
        "d.rename(\n",
        "         columns = {'Unnamed: 0': 'index'},\n",
        "         inplace = True,\n",
        "         )\n",
        "\n",
        "d.set_index(\n",
        "             'index',\n",
        "             inplace = True,\n",
        "             )\n",
        "\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "yUgMFguF1pMZ",
      "metadata": {
        "id": "yUgMFguF1pMZ"
      },
      "source": [
        "**Text covariate _$W_{pA}$_: concatenate post author comments + title (by block)**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "FnpBLeSWPj42",
      "metadata": {
        "collapsed": true,
        "id": "FnpBLeSWPj42"
      },
      "outputs": [],
      "source": [
        "d['p_au_c_text'] = d['p_au_c_text'].astype(str)\n",
        "\n",
        "# Concat post author comments x block\n",
        "\n",
        "d['block_p_au_c_text'] = d.groupby('block')['p_au_c_text'].transform(lambda i: ' '.join(i))\n",
        "\n",
        "# Append post title to concatenated post author comments\n",
        "\n",
        "d['text_covar_a'] = d.apply(lambda row: row['block_p_au_c_text'] + row['p_titl'] if row['p_titl'] != '<|RPT|>' else row['block_p_au_c_text'], axis = 1)\n",
        "\n",
        "d['p_uniq'].value_counts()\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "NkuLBGMZ6l9S",
      "metadata": {
        "id": "NkuLBGMZ6l9S"
      },
      "source": [
        "**Text covariate _$W_{pB}$_: concatenated post author comments (by block)**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "jjRlDg_nPj8X",
      "metadata": {
        "id": "jjRlDg_nPj8X"
      },
      "outputs": [],
      "source": [
        "d.rename(\n",
        "         columns = {'block_p_au_c_text': 'text_covar_b'},\n",
        "         inplace = True,\n",
        "         )\n",
        "\n",
        "d.dtypes\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "JvFU3dJ4v2QH",
      "metadata": {
        "id": "JvFU3dJ4v2QH"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "# BACKUP BACKUP BACKUP 01\n",
        "\n",
        "d.to_csv(\n",
        "         'd_inference_labeled_autocoded_backup01.csv',\n",
        "         index = 'index',\n",
        "         )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "A_tMdxf3v2VW",
      "metadata": {
        "collapsed": true,
        "id": "A_tMdxf3v2VW"
      },
      "outputs": [],
      "source": [
        "# Delete sentiment fr post author replies\n",
        "\n",
        "d.loc[d['p_au_reply'] == 1, 'positive'] = None\n",
        "d.shape\n",
        "d['p_uniq'].value_counts()\n",
        "d.head(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6GnSwBa4v2Zu",
      "metadata": {
        "id": "6GnSwBa4v2Zu"
      },
      "outputs": [],
      "source": [
        "# Avg positive sentiment x block\n",
        "\n",
        "d['pos_block_mean'] = d.groupby('block')['positive'].transform('mean')\n",
        "\n",
        "# Mdn\n",
        "\n",
        "mdn = d['positive'].median()\n",
        "\n",
        "# IQR\n",
        "\n",
        "q1 = d['positive'].quantile(0.25)\n",
        "q3 = d['positive'].quantile(0.75)\n",
        "\n",
        "'mdn'\n",
        "print(mdn)\n",
        "\n",
        "'Q1'\n",
        "print(q1)\n",
        "\n",
        "'Q3'\n",
        "print(q3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "GAgpf-KkBwXz",
      "metadata": {
        "id": "GAgpf-KkBwXz"
      },
      "source": [
        "**Compute _$Y_{p}$_: >Mdn block-level positive sentiment**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "CRyclUW0v2d6",
      "metadata": {
        "id": "CRyclUW0v2d6"
      },
      "outputs": [],
      "source": [
        "d['y_pos_mdn'] = (d['pos_block_mean'] >= mdn).astype(int)\n",
        "d['y_pos_mdn'].value_counts()\n",
        "\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ON7wJ_dwv2mP",
      "metadata": {
        "id": "ON7wJ_dwv2mP"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "# BACKUP BACKUP BACKUP 02\n",
        "\n",
        "d.to_csv(\n",
        "         'd_inference_labeled_autocoded_backup02.csv',\n",
        "         index = 'index',\n",
        "         )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "UgjdBtWyJyCv",
      "metadata": {
        "id": "UgjdBtWyJyCv"
      },
      "outputs": [],
      "source": [
        "\n",
        "d = pd.read_csv(\n",
        "           'd_inference_labeled_autocoded_backup02.csv',\n",
        "           )\n",
        "\n",
        "d.head(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "RyGQZnlHDurl",
      "metadata": {
        "id": "RyGQZnlHDurl"
      },
      "source": [
        "**Collapse: block-level _$W_{pA}$_, _$W_{pB}$_, _$Y_{p}$_ aggregates for model input**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "hw68nsDMERST",
      "metadata": {
        "id": "hw68nsDMERST"
      },
      "outputs": [],
      "source": [
        "# Reduce for LGBMClassifier input\n",
        "\n",
        "d = d[[\n",
        "       'text',\n",
        "       'block',\n",
        "       'p_uniq',\n",
        "       'hate_label',\n",
        "       'lone_label',\n",
        "       'tb_label',\n",
        "       'pb_label',\n",
        "       'p_au_reply',\n",
        "       'text_covar_a',\n",
        "       'text_covar_b',\n",
        "       'y_pos_mdn',\n",
        "       ]].copy()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "VHDfcTQqLiDA",
      "metadata": {
        "id": "VHDfcTQqLiDA"
      },
      "outputs": [],
      "source": [
        "# p_au_reply_block: 1 = _block_ includes a post author reply -> retain for subset\n",
        "\n",
        "d['p_au_reply_block'] = d.groupby('block')['p_au_reply'].transform(lambda i: int(i.max() == 1))\n",
        "d['p_au_reply_block'].value_counts()\n",
        "\n",
        "d.head(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "rOYSh07KDtjt",
      "metadata": {
        "id": "rOYSh07KDtjt"
      },
      "outputs": [],
      "source": [
        "d['p_uniq'].value_counts()\n",
        "d = d[d['p_uniq'] != 0]\n",
        "\n",
        "d.reset_index(inplace = True)\n",
        "\n",
        "d.shape\n",
        "d.head(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "FmqwWiZ2Dtu7",
      "metadata": {
        "id": "FmqwWiZ2Dtu7"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "# BACKUP BACKUP BACKUP 03\n",
        "\n",
        "d.to_csv(\n",
        "         'd_inference_labeled_autocoded_backup03.csv',\n",
        "         index = 'index',\n",
        "         )"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Model A: all blocks ($\\mathcal{d}$<sub>all blocks</sub>)**"
      ],
      "metadata": {
        "id": "Ai0FCIIIVsSQ"
      },
      "id": "Ai0FCIIIVsSQ"
    },
    {
      "cell_type": "code",
      "source": [
        "d_all_blocks = d"
      ],
      "metadata": {
        "id": "W6Ttcm1jVrr3"
      },
      "id": "W6Ttcm1jVrr3",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "9OJ920ZcGsF1",
      "metadata": {
        "id": "9OJ920ZcGsF1"
      },
      "source": [
        "**Model B: restrict to posts with post author replies ($\\mathcal{d}$<sub>author blocks</sub>)**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "vPLn0-WlDt1Q",
      "metadata": {
        "collapsed": true,
        "id": "vPLn0-WlDt1Q"
      },
      "outputs": [],
      "source": [
        "d_au_blocks = d[d['p_au_reply_block'] == 1]\n",
        "\n",
        "d_au_blocks.reset_index(inplace = True)\n",
        "\n",
        "d_au_blocks.shape\n",
        "d_au_blocks.head(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "JA9yaATdGeJX",
      "metadata": {
        "id": "JA9yaATdGeJX"
      },
      "outputs": [],
      "source": [
        "'''\n",
        "# BACKUP BACKUP SUB\n",
        "\n",
        "d_sub.to_csv(\n",
        "         'd_sub_backup.csv',\n",
        "         index = 'index',\n",
        "         )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "uKM9xezOs1GQ",
      "metadata": {
        "id": "uKM9xezOs1GQ",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "#%cd /content/gdrive/My Drive/Colab/intrapolitical_suicidality/inputs/data\n",
        "\n",
        "#d_all_blocks = pd.read_csv(\n",
        "#                          'd_inference_labeled_autocoded_backup03.csv',\n",
        "#                          )\n",
        "\n",
        "d_all_blocks.head(3)\n",
        "\n",
        "#d_au_blocks = pd.read_csv(\n",
        "#                           'd_sub_backup.csv',\n",
        "#                           )\n",
        "\n",
        "d_au_blocks.head(3)\n",
        "\n",
        "    ### SJS 8/12: these col heads match 1:1, so ATE loop is stable - but clean these crappy indices up"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6CVDe8mcocBf",
      "metadata": {
        "id": "6CVDe8mcocBf"
      },
      "source": [
        "**8:2 subset for robustness check**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "zetUQB4IoaPU",
      "metadata": {
        "id": "zetUQB4IoaPU"
      },
      "outputs": [],
      "source": [
        "# Model A\n",
        "\n",
        "d_all_blocks = d_all_blocks.drop(columns = ['text_covar_b'])\n",
        "\n",
        "d_all_blocks_subset = d_all_blocks.sample(\n",
        "                                          frac = 0.8,\n",
        "                                          random_state = 56,\n",
        "                                          )\n",
        "\n",
        "\n",
        "\n",
        "d_all_blocks_subset.rename(\n",
        "                           columns = {'text_covar_a': 'text_covar_a_subset'},\n",
        "                           inplace = True,\n",
        "                           )\n",
        "\n",
        "d_all_blocks_subset.reset_index(\n",
        "                                drop = True,\n",
        "                                inplace = True,\n",
        "                                )\n",
        "\n",
        "d_all_blocks.shape\n",
        "d_all_blocks_subset.shape\n",
        "\n",
        "# Model B\n",
        "\n",
        "d_au_blocks = d_au_blocks.drop(columns = ['text_covar_a'])\n",
        "\n",
        "d_au_blocks_subset = d_au_blocks.sample(\n",
        "                                        frac = 0.8,\n",
        "                                        random_state = 56,\n",
        "                                        )\n",
        "d_au_blocks_subset.rename(\n",
        "                          columns = {'text_covar_b': 'text_covar_b_subset'},\n",
        "                          inplace = True,\n",
        "                          )\n",
        "\n",
        "d_au_blocks_subset.reset_index(\n",
        "                               drop = True,\n",
        "                               inplace = True,\n",
        "                               )\n",
        "\n",
        "\n",
        "d_au_blocks.shape\n",
        "d_au_blocks_subset.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "-n_y_fQFN3EK",
      "metadata": {
        "id": "-n_y_fQFN3EK"
      },
      "source": [
        "#### Model A: fit and estimate"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "treatments = [\n",
        "              'hate_label',\n",
        "              'lone_label',\n",
        "              'tb_label',\n",
        "              'pb_label',\n",
        "              ]\n",
        "\n",
        "ignore_cols_baseline = [\n",
        "                        'Unnamed: 0',\n",
        "                        'level_0',\n",
        "                        'index',\n",
        "                        'p_au',\n",
        "                        'p_date',\n",
        "                        'text',\n",
        "                        'p_sbrt',\n",
        "                        'block',\n",
        "                        'p_uniq',\n",
        "                        'p_au_reply',\n",
        "                        #'text_covar_b',\n",
        "                        'p_au_reply_block',\n",
        "                        ]\n",
        "\n",
        "tokens = [\n",
        "          'deserve',\n",
        "          'disabled',\n",
        "          'ugly',\n",
        "          'disgust',\n",
        "          'abuse',\n",
        "          'sex',\n",
        "          'lonely',\n",
        "          'alone',\n",
        "          ]\n",
        "\n",
        "models = {}\n",
        "\n",
        "for treatment in treatments:\n",
        "    model_name = f'cm_{treatment.split(\"_\")[0]}'\n",
        "    ignore_cols = ignore_cols_baseline + [t for t in treatments if t != treatment]\n",
        "\n",
        "    print('\\n-----------------------------------------------------------------------')\n",
        "    print(f'Fitting: {model_name}')\n",
        "    print('-----------------------------------------------------------------------')\n",
        "    print('\\n')\n",
        "\n",
        "    models[model_name] = CausalInferenceModel(\n",
        "                                              d_all_blocks, ### primary model\n",
        "                                              #d_all_blocks_subset, ### 80% sample: robustness check\n",
        "                                              method = 't-learner',\n",
        "                                              learner = LGBMClassifier(num_leaves = 500),\n",
        "                                              treatment_col = treatment,\n",
        "                                              outcome_col = 'y_pos_mdn',\n",
        "                                              text_col = 'text_covar_a', ### 'text_covar_a' = post author comments _and_ post titles\n",
        "                                              #text_col = 'text_covar_a_subset',\n",
        "                                              ngram_range=(\n",
        "                                                           1,\n",
        "                                                           3\n",
        "                                                           ),\n",
        "                                              min_df = 0.01,\n",
        "                                              stop_words = 'english',\n",
        "                                              ignore_cols = ignore_cols,\n",
        "                                              verbose = -1,\n",
        "                                              )\n",
        "\n",
        "    models[model_name].fit()\n",
        "\n",
        "    print('\\n=======================================================================')\n",
        "    print(f'Results: ATE for {model_name}')\n",
        "    print('=======================================================================')\n",
        "    ate = models[model_name].estimate_ate()\n",
        "    print(ate)\n",
        "    print('\\n')\n",
        "\n",
        "    top_features = models[model_name].interpret(\n",
        "                                                plot = False,\n",
        "                                                method = 'feature_importance',\n",
        "                                                )[1][:10]\n",
        "\n",
        "    print('\\n')\n",
        "    print('=======================================================================')\n",
        "    for token in tokens:\n",
        "        print(f'CATE for {model_name}: {token}')\n",
        "        models[model_name].estimate_ate(d_au_blocks['text_covar_b'].str.contains(token))\n",
        "\n",
        "    print('\\n=======================================================================')\n",
        "    print(f'Top features for {model_name}:')\n",
        "    print('=======================================================================')\n",
        "    print(top_features)\n",
        "    print('\\n')"
      ],
      "metadata": {
        "collapsed": true,
        "id": "UV_z4Jei2Mqe"
      },
      "id": "UV_z4Jei2Mqe",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Model B: fit and estimate"
      ],
      "metadata": {
        "id": "qqWNjCdX-8aw"
      },
      "id": "qqWNjCdX-8aw"
    },
    {
      "cell_type": "code",
      "source": [
        "treatments = [\n",
        "              'hate_label',\n",
        "              'lone_label',\n",
        "              'tb_label',\n",
        "              'pb_label',\n",
        "              ]\n",
        "\n",
        "ignore_cols_baseline = [\n",
        "                        'Unnamed: 0',\n",
        "                        'level_0',\n",
        "                        'index',\n",
        "                        'p_au',\n",
        "                        'p_date',\n",
        "                        'text',\n",
        "                        'p_sbrt',\n",
        "                        'block',\n",
        "                        'p_uniq',\n",
        "                        'p_au_reply',\n",
        "                        #'text_covar_a',\n",
        "                        'p_au_reply_block',\n",
        "                        ]\n",
        "\n",
        "tokens = [\n",
        "          'deserve',\n",
        "          'disabled',\n",
        "          'ugly',\n",
        "          'disgust',\n",
        "          'abuse',\n",
        "          'sex',\n",
        "          'lonely',\n",
        "          'alone',\n",
        "          ]\n",
        "\n",
        "models = {}\n",
        "\n",
        "for treatment in treatments:\n",
        "    model_name = f'cm_{treatment.split(\"_\")[0]}'\n",
        "    ignore_cols = ignore_cols_baseline + [t for t in treatments if t != treatment]\n",
        "\n",
        "    print('\\n-----------------------------------------------------------------------')\n",
        "    print(f'Fitting: {model_name}')\n",
        "    print('-----------------------------------------------------------------------')\n",
        "    print('\\n')\n",
        "\n",
        "    models[model_name] = CausalInferenceModel(\n",
        "                                              d_au_blocks,  ### primary model\n",
        "                                              #d_au_blocks_subset,  ### 80% sample: robustness check\n",
        "                                              method = 't-learner',\n",
        "                                              learner = LGBMClassifier(num_leaves = 500),\n",
        "                                              treatment_col = treatment,\n",
        "                                              outcome_col = 'y_pos_mdn',\n",
        "                                              text_col = 'text_covar_b',  ### 'text_covar_b' = post author comments\n",
        "                                              #text_col = 'text_covar_b_subset',\n",
        "                                              ngram_range=(\n",
        "                                                           1,\n",
        "                                                           3\n",
        "                                                           ),\n",
        "                                              min_df = 0.01,\n",
        "                                              stop_words = 'english',\n",
        "                                              ignore_cols = ignore_cols,\n",
        "                                              verbose = -1,\n",
        "                                              )\n",
        "\n",
        "    models[model_name].fit()\n",
        "\n",
        "    print('\\n=======================================================================')\n",
        "    print(f'Results: ATE for {model_name}')\n",
        "    print('=======================================================================')\n",
        "    ate = models[model_name].estimate_ate()\n",
        "    print(ate)\n",
        "    print('\\n')\n",
        "\n",
        "    top_features = models[model_name].interpret(\n",
        "                                                plot = False,\n",
        "                                                method = 'feature_importance',\n",
        "                                                )[1][:10]\n",
        "\n",
        "    print('\\n')\n",
        "    print('=======================================================================')\n",
        "    for token in tokens:\n",
        "        print(f'CATE for {model_name}: {token}')\n",
        "        models[model_name].estimate_ate(d_au_blocks['text_covar_b'].str.contains(token))\n",
        "\n",
        "    print('\\n=======================================================================')\n",
        "    print(f'Top features for {model_name}:')\n",
        "    print('=======================================================================')\n",
        "    print(top_features)\n",
        "    print('\\n')"
      ],
      "metadata": {
        "id": "C6CX9cKd2M1s",
        "collapsed": true
      },
      "id": "C6CX9cKd2M1s",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Concordances x token: nltk**"
      ],
      "metadata": {
        "id": "CLNCI-V6ipOX"
      },
      "id": "CLNCI-V6ipOX"
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "from nltk.text import Text\n",
        "\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "\n",
        "# Derived qualitatively (deductively)\n",
        "\n",
        "tokens = [\n",
        "          'deserve',\n",
        "          'disabled',\n",
        "          'ugly',\n",
        "          'disgust',\n",
        "          'abuse',\n",
        "          'sex',\n",
        "          'lonely',\n",
        "          'alone',\n",
        "          ]\n",
        "\n",
        "# Derived by feature importance (inductively)\n",
        "\n",
        "#tokens = ['thanks']\n",
        "\n",
        "# parse by T_p\n",
        "\n",
        "d_parsed = d_au_blocks[d_au_blocks['pb_label'] == 1]\n",
        "\n",
        "# transform to nltk text object\n",
        "\n",
        "text_col = d_parsed['text_covar_b'].dropna().tolist()\n",
        "joined_text = ' '.join(text_col)\n",
        "tokenized_text = nltk.word_tokenize(joined_text)\n",
        "nltk_text = Text(tokenized_text)\n",
        "\n",
        "# examine token in context\n",
        "\n",
        "for token in tokens:\n",
        "    nltk_text.concordance(\n",
        "                          token,\n",
        "                          lines = 50,\n",
        "                          width = 100,\n",
        "                          )\n"
      ],
      "metadata": {
        "collapsed": true,
        "id": "PPmfnZp7in78"
      },
      "id": "PPmfnZp7in78",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "id": "5b5c91a8-7f53-4499-b4a5-e7e878f0678a",
      "metadata": {
        "id": "5b5c91a8-7f53-4499-b4a5-e7e878f0678a"
      },
      "source": [
        "> End of its_train_test_pred.ipynb"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "6ALab8tymeX6"
      ],
      "gpuType": "T4",
      "machine_shape": "hm",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}