appendix1_reaction_time_model_selection_glmm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "# Copyright 2022 by the author(s) of CHI2023 submission \"Short-Form\n",
    "# Videos Degrade Our Capacity to Retain Intentions: Effect of Context\n",
    "# Switching On Prospective Memory\". All rights reserved.\n",
    "#\n",
    "# Use of this source code is governed by a GPLv3 license that\n",
    "# can be found in the LICENSE file.\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [],
   "source": [
    "library(lme4)\n",
    "library(dplyr)\n",
    "library(bayestestR)\n",
    "library(report)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"dataframe\">\n",
       "<caption>A data.frame: 6 × 8</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>folder_id</th><th scope=col>task</th><th scope=col>interrupt</th><th scope=col>measure</th><th scope=col>stimulus</th><th scope=col>success</th><th scope=col>correct</th><th scope=col>rt</th></tr>\n",
       "\t<tr><th></th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>1</th><td>101</td><td>LD</td><td>rest</td><td>train</td><td>TRINKEN</td><td>True</td><td>True</td><td>0.7375202</td></tr>\n",
       "\t<tr><th scope=row>2</th><td>101</td><td>LD</td><td>rest</td><td>train</td><td>EIRLHCH</td><td>True</td><td>True</td><td>0.8492210</td></tr>\n",
       "\t<tr><th scope=row>3</th><td>101</td><td>LD</td><td>rest</td><td>train</td><td>PLATZ  </td><td>True</td><td>True</td><td>1.2250568</td></tr>\n",
       "\t<tr><th scope=row>4</th><td>101</td><td>LD</td><td>rest</td><td>train</td><td>ALEEINL</td><td>True</td><td>True</td><td>0.8843554</td></tr>\n",
       "\t<tr><th scope=row>5</th><td>101</td><td>LD</td><td>rest</td><td>train</td><td>TEHS   </td><td>True</td><td>True</td><td>0.7557791</td></tr>\n",
       "\t<tr><th scope=row>6</th><td>101</td><td>LD</td><td>rest</td><td>train</td><td>LERNEN </td><td>True</td><td>True</td><td>0.5365034</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.frame: 6 × 8\n",
       "\\begin{tabular}{r|llllllll}\n",
       "  & folder\\_id & task & interrupt & measure & stimulus & success & correct & rt\\\\\n",
       "  & <int> & <chr> & <chr> & <chr> & <chr> & <chr> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t1 & 101 & LD & rest & train & TRINKEN & True & True & 0.7375202\\\\\n",
       "\t2 & 101 & LD & rest & train & EIRLHCH & True & True & 0.8492210\\\\\n",
       "\t3 & 101 & LD & rest & train & PLATZ   & True & True & 1.2250568\\\\\n",
       "\t4 & 101 & LD & rest & train & ALEEINL & True & True & 0.8843554\\\\\n",
       "\t5 & 101 & LD & rest & train & TEHS    & True & True & 0.7557791\\\\\n",
       "\t6 & 101 & LD & rest & train & LERNEN  & True & True & 0.5365034\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.frame: 6 × 8\n",
       "\n",
       "| <!--/--> | folder_id &lt;int&gt; | task &lt;chr&gt; | interrupt &lt;chr&gt; | measure &lt;chr&gt; | stimulus &lt;chr&gt; | success &lt;chr&gt; | correct &lt;chr&gt; | rt &lt;dbl&gt; |\n",
       "|---|---|---|---|---|---|---|---|---|\n",
       "| 1 | 101 | LD | rest | train | TRINKEN | True | True | 0.7375202 |\n",
       "| 2 | 101 | LD | rest | train | EIRLHCH | True | True | 0.8492210 |\n",
       "| 3 | 101 | LD | rest | train | PLATZ   | True | True | 1.2250568 |\n",
       "| 4 | 101 | LD | rest | train | ALEEINL | True | True | 0.8843554 |\n",
       "| 5 | 101 | LD | rest | train | TEHS    | True | True | 0.7557791 |\n",
       "| 6 | 101 | LD | rest | train | LERNEN  | True | True | 0.5365034 |\n",
       "\n"
      ],
      "text/plain": [
       "  folder_id task interrupt measure stimulus success correct rt       \n",
       "1 101       LD   rest      train   TRINKEN  True    True    0.7375202\n",
       "2 101       LD   rest      train   EIRLHCH  True    True    0.8492210\n",
       "3 101       LD   rest      train   PLATZ    True    True    1.2250568\n",
       "4 101       LD   rest      train   ALEEINL  True    True    0.8843554\n",
       "5 101       LD   rest      train   TEHS     True    True    0.7557791\n",
       "6 101       LD   rest      train   LERNEN   True    True    0.5365034"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df <- read.csv(\"data/rt.csv\")\n",
    "head(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PM Task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [],
   "source": [
    "df_pm_correct <- df %>% filter((task %in% c('PM'))) %>% filter((success %in% c('True'))) %>% filter((correct %in% c('True'))) %>% filter(!(measure %in% c('train')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning message in checkConv(attr(opt, \"derivs\"), opt$par, ctrl = control$checkConv, :\n",
      "“Model failed to converge with max|grad| = 0.00235194 (tol = 0.002, component 1)”\n",
      "refitting model(s) with ML (instead of REML)\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"dataframe\">\n",
       "<caption>A anova: 7 × 8</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>npar</th><th scope=col>AIC</th><th scope=col>BIC</th><th scope=col>logLik</th><th scope=col>deviance</th><th scope=col>Chisq</th><th scope=col>Df</th><th scope=col>Pr(&gt;Chisq)</th></tr>\n",
       "\t<tr><th></th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>m0</th><td> 6</td><td>855.589646</td><td>886.88109</td><td>-421.79482</td><td>843.58965</td><td>       NA</td><td>NA</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m1</th><td>10</td><td>416.362081</td><td>468.51448</td><td>-198.18104</td><td>396.36208</td><td>447.22757</td><td> 4</td><td>1.726651e-95</td></tr>\n",
       "\t<tr><th scope=row>m2</th><td>10</td><td>417.160226</td><td>469.31263</td><td>-198.58011</td><td>397.16023</td><td>  0.00000</td><td> 0</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m3</th><td>10</td><td>325.621815</td><td>377.77422</td><td>-152.81091</td><td>305.62182</td><td> 91.53841</td><td> 0</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m4</th><td>11</td><td> 97.997415</td><td>155.36505</td><td> -37.99871</td><td> 75.99742</td><td>229.62440</td><td> 1</td><td>7.198714e-52</td></tr>\n",
       "\t<tr><th scope=row>m5</th><td>11</td><td>108.351289</td><td>165.71893</td><td> -43.17564</td><td> 86.35129</td><td>  0.00000</td><td> 0</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m6</th><td>11</td><td> -4.318017</td><td> 53.04962</td><td>  13.15901</td><td>-26.31802</td><td>112.66931</td><td> 0</td><td>          NA</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A anova: 7 × 8\n",
       "\\begin{tabular}{r|llllllll}\n",
       "  & npar & AIC & BIC & logLik & deviance & Chisq & Df & Pr(>Chisq)\\\\\n",
       "  & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl>\\\\\n",
       "\\hline\n",
       "\tm0 &  6 & 855.589646 & 886.88109 & -421.79482 & 843.58965 &        NA & NA &           NA\\\\\n",
       "\tm1 & 10 & 416.362081 & 468.51448 & -198.18104 & 396.36208 & 447.22757 &  4 & 1.726651e-95\\\\\n",
       "\tm2 & 10 & 417.160226 & 469.31263 & -198.58011 & 397.16023 &   0.00000 &  0 &           NA\\\\\n",
       "\tm3 & 10 & 325.621815 & 377.77422 & -152.81091 & 305.62182 &  91.53841 &  0 &           NA\\\\\n",
       "\tm4 & 11 &  97.997415 & 155.36505 &  -37.99871 &  75.99742 & 229.62440 &  1 & 7.198714e-52\\\\\n",
       "\tm5 & 11 & 108.351289 & 165.71893 &  -43.17564 &  86.35129 &   0.00000 &  0 &           NA\\\\\n",
       "\tm6 & 11 &  -4.318017 &  53.04962 &   13.15901 & -26.31802 & 112.66931 &  0 &           NA\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A anova: 7 × 8\n",
       "\n",
       "| <!--/--> | npar &lt;dbl&gt; | AIC &lt;dbl&gt; | BIC &lt;dbl&gt; | logLik &lt;dbl&gt; | deviance &lt;dbl&gt; | Chisq &lt;dbl&gt; | Df &lt;dbl&gt; | Pr(&gt;Chisq) &lt;dbl&gt; |\n",
       "|---|---|---|---|---|---|---|---|---|\n",
       "| m0 |  6 | 855.589646 | 886.88109 | -421.79482 | 843.58965 |        NA | NA |           NA |\n",
       "| m1 | 10 | 416.362081 | 468.51448 | -198.18104 | 396.36208 | 447.22757 |  4 | 1.726651e-95 |\n",
       "| m2 | 10 | 417.160226 | 469.31263 | -198.58011 | 397.16023 |   0.00000 |  0 |           NA |\n",
       "| m3 | 10 | 325.621815 | 377.77422 | -152.81091 | 305.62182 |  91.53841 |  0 |           NA |\n",
       "| m4 | 11 |  97.997415 | 155.36505 |  -37.99871 |  75.99742 | 229.62440 |  1 | 7.198714e-52 |\n",
       "| m5 | 11 | 108.351289 | 165.71893 |  -43.17564 |  86.35129 |   0.00000 |  0 |           NA |\n",
       "| m6 | 11 |  -4.318017 |  53.04962 |   13.15901 | -26.31802 | 112.66931 |  0 |           NA |\n",
       "\n"
      ],
      "text/plain": [
       "   npar AIC        BIC       logLik     deviance  Chisq     Df Pr(>Chisq)  \n",
       "m0  6   855.589646 886.88109 -421.79482 843.58965        NA NA           NA\n",
       "m1 10   416.362081 468.51448 -198.18104 396.36208 447.22757  4 1.726651e-95\n",
       "m2 10   417.160226 469.31263 -198.58011 397.16023   0.00000  0           NA\n",
       "m3 10   325.621815 377.77422 -152.81091 305.62182  91.53841  0           NA\n",
       "m4 11    97.997415 155.36505  -37.99871  75.99742 229.62440  1 7.198714e-52\n",
       "m5 11   108.351289 165.71893  -43.17564  86.35129   0.00000  0           NA\n",
       "m6 11    -4.318017  53.04962   13.15901 -26.31802 112.66931  0           NA"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_pm_correct$interrupt <- factor(df_pm_correct$interrupt, levels=c('rest', 'twitter', 'youtube', 'tiktok'))\n",
    "df_pm_correct$measure   <- factor(df_pm_correct$measure,   levels=c('pre', 'post'))\n",
    "\n",
    "m0 <- lmer(rt  ~ interrupt + (1|folder_id), data = df_pm_correct)\n",
    "m1 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id), data = df_pm_correct, family = Gamma(link = \"log\"))\n",
    "m2 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id), data = df_pm_correct, family = Gamma(link = \"inverse\"))\n",
    "m3 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id), data = df_pm_correct, family = inverse.gaussian(link = \"inverse\"))\n",
    "m4 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id) + (1|stimulus), data = df_pm_correct, family = Gamma(link = \"log\"))\n",
    "m5 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id) + (1|stimulus), data = df_pm_correct, family = Gamma(link = \"inverse\"))\n",
    "m6 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id) + (1|stimulus), data = df_pm_correct, family = inverse.gaussian(link = \"inverse\"))\n",
    "\n",
    "anova(m0, m1, m2, m3, m4, m5, m6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"dataframe\">\n",
       "<caption>A bayesfactor_models: 7 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>Model</th><th scope=col>log_BF</th></tr>\n",
       "\t<tr><th></th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>m1</th><td>interrupt * measure + interrupt + measure + (1 | folder_id)                 </td><td>209.2529</td></tr>\n",
       "\t<tr><th scope=row>m2</th><td>interrupt * measure + interrupt + measure + (1 | folder_id)                 </td><td>208.8538</td></tr>\n",
       "\t<tr><th scope=row>m3</th><td>interrupt * measure + interrupt + measure + (1 | folder_id)                 </td><td>254.6230</td></tr>\n",
       "\t<tr><th scope=row>m4</th><td>interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)</td><td>365.8276</td></tr>\n",
       "\t<tr><th scope=row>m5</th><td>interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)</td><td>360.6507</td></tr>\n",
       "\t<tr><th scope=row>m6</th><td>interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)</td><td>416.9853</td></tr>\n",
       "\t<tr><th scope=row>m0</th><td>interrupt + (1 | folder_id)                                                 </td><td>  0.0000</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A bayesfactor\\_models: 7 × 2\n",
       "\\begin{tabular}{r|ll}\n",
       "  & Model & log\\_BF\\\\\n",
       "  & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\tm1 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id)                  & 209.2529\\\\\n",
       "\tm2 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id)                  & 208.8538\\\\\n",
       "\tm3 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id)                  & 254.6230\\\\\n",
       "\tm4 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id) + (1 \\textbar{} stimulus) & 365.8276\\\\\n",
       "\tm5 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id) + (1 \\textbar{} stimulus) & 360.6507\\\\\n",
       "\tm6 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id) + (1 \\textbar{} stimulus) & 416.9853\\\\\n",
       "\tm0 & interrupt + (1 \\textbar{} folder\\_id)                                                  &   0.0000\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A bayesfactor_models: 7 × 2\n",
       "\n",
       "| <!--/--> | Model &lt;chr&gt; | log_BF &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| m1 | interrupt * measure + interrupt + measure + (1 | folder_id)                  | 209.2529 |\n",
       "| m2 | interrupt * measure + interrupt + measure + (1 | folder_id)                  | 208.8538 |\n",
       "| m3 | interrupt * measure + interrupt + measure + (1 | folder_id)                  | 254.6230 |\n",
       "| m4 | interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus) | 365.8276 |\n",
       "| m5 | interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus) | 360.6507 |\n",
       "| m6 | interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus) | 416.9853 |\n",
       "| m0 | interrupt + (1 | folder_id)                                                  |   0.0000 |\n",
       "\n"
      ],
      "text/plain": [
       "   Model                                                                       \n",
       "m1 interrupt * measure + interrupt + measure + (1 | folder_id)                 \n",
       "m2 interrupt * measure + interrupt + measure + (1 | folder_id)                 \n",
       "m3 interrupt * measure + interrupt + measure + (1 | folder_id)                 \n",
       "m4 interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)\n",
       "m5 interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)\n",
       "m6 interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)\n",
       "m0 interrupt + (1 | folder_id)                                                 \n",
       "   log_BF  \n",
       "m1 209.2529\n",
       "m2 208.8538\n",
       "m3 254.6230\n",
       "m4 365.8276\n",
       "m5 360.6507\n",
       "m6 416.9853\n",
       "m0   0.0000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bayesfactor_models(m1, m2, m3, m4, m5, m6, denominator = m0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Generalized linear mixed model fit by maximum likelihood (Laplace\n",
       "  Approximation) [glmerMod]\n",
       " Family: Gamma  ( log )\n",
       "Formula: rt ~ interrupt * measure + interrupt + measure + (1 | folder_id) +  \n",
       "    (1 | stimulus)\n",
       "   Data: df_pm_correct\n",
       "\n",
       "     AIC      BIC   logLik deviance df.resid \n",
       "    98.0    155.4    -38.0     76.0     1349 \n",
       "\n",
       "Scaled residuals: \n",
       "    Min      1Q  Median      3Q     Max \n",
       "-2.0986 -0.6061 -0.2025  0.3880  5.7110 \n",
       "\n",
       "Random effects:\n",
       " Groups    Name        Variance Std.Dev.\n",
       " folder_id (Intercept) 0.010393 0.10195 \n",
       " stimulus  (Intercept) 0.002005 0.04478 \n",
       " Residual              0.050752 0.22528 \n",
       "Number of obs: 1360, groups:  folder_id, 60; stimulus, 3\n",
       "\n",
       "Fixed effects:\n",
       "                               Estimate Std. Error t value Pr(>|z|)  \n",
       "(Intercept)                   2.331e-01  1.063e-01   2.194   0.0283 *\n",
       "interrupttwitter             -3.224e-02  8.334e-02  -0.387   0.6989  \n",
       "interruptyoutube              1.986e-02  8.192e-02   0.242   0.8085  \n",
       "interrupttiktok              -4.258e-03  7.955e-02  -0.054   0.9573  \n",
       "measurepost                  -2.757e-02  2.112e-02  -1.305   0.1917  \n",
       "interrupttwitter:measurepost -6.232e-05  2.952e-02  -0.002   0.9983  \n",
       "interruptyoutube:measurepost -7.873e-02  3.065e-02  -2.569   0.0102 *\n",
       "interrupttiktok:measurepost  -3.065e-02  3.373e-02  -0.909   0.3635  \n",
       "---\n",
       "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
       "\n",
       "Correlation of Fixed Effects:\n",
       "            (Intr) intrrpttw intrrpty intrrpttk msrpst intrrpttw: intrrpty:\n",
       "intrrpttwtt -0.379                                                         \n",
       "interrptytb -0.402  0.501                                                  \n",
       "intrrpttktk -0.436  0.517     0.525                                        \n",
       "measurepost -0.104  0.133     0.135    0.139                               \n",
       "intrrpttwt:  0.074 -0.187    -0.097   -0.100    -0.715                     \n",
       "intrrptytb:  0.072 -0.092    -0.192   -0.096    -0.688  0.492              \n",
       "intrrpttkt:  0.065 -0.083    -0.085   -0.185    -0.625  0.448      0.431   "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "summary(m4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We fitted a general linear mixed model (Gamma family with a log link)\n",
      "(estimated using ML and Nelder-Mead optimizer) to predict rt with interrupt and\n",
      "measure\n",
      " (formula: rt ~ interrupt * measure + interrupt + measure). The model included\n",
      "folder_id and stimulus\n",
      " as random effects (formula: list(~1 | folder_id, ~1 | stimulus)). The model's\n",
      "total explanatory power is moderate (conditional R2 = 0.21) and the part\n",
      "related to the fixed effects alone (marginal R2) is of 0.02. The model's\n",
      "intercept, corresponding to interrupt = rest and measure = pre\n",
      ", is at 0.23 (95% CI [0.02, 0.44], t(1349) = 2.19, p = 0.028). Within this\n",
      "model:\n",
      "\n",
      "  - The effect of interrupt [twitter] is statistically non-significant and\n",
      "negative (beta = -0.03, 95% CI [-0.20, 0.13], t(1349) = -0.39, p = 0.699; Std.\n",
      "beta = -0.03, 95% CI [-0.20, 0.13])\n",
      "  - The effect of interrupt [youtube] is statistically non-significant and\n",
      "positive (beta = 0.02, 95% CI [-0.14, 0.18], t(1349) = 0.24, p = 0.808; Std.\n",
      "beta = 0.02, 95% CI [-0.14, 0.18])\n",
      "  - The effect of interrupt [tiktok] is statistically non-significant and\n",
      "negative (beta = -4.26e-03, 95% CI [-0.16, 0.15], t(1349) = -0.05, p = 0.957;\n",
      "Std. beta = -4.26e-03, 95% CI [-0.16, 0.15])\n",
      "  - The effect of measure [post] is statistically non-significant and negative\n",
      "(beta = -0.03, 95% CI [-0.07, 0.01], t(1349) = -1.31, p = 0.192; Std. beta =\n",
      "-0.03, 95% CI [-0.07, 0.01])\n",
      "  - The interaction effect of measure [post] on interrupt [twitter] is\n",
      "statistically non-significant and negative (beta = -6.23e-05, 95% CI [-0.06,\n",
      "0.06], t(1349) = -2.11e-03, p = 0.998; Std. beta = -6.23e-05, 95% CI [-0.06,\n",
      "0.06])\n",
      "  - The interaction effect of measure [post] on interrupt [youtube] is\n",
      "statistically significant and negative (beta = -0.08, 95% CI [-0.14, -0.02],\n",
      "t(1349) = -2.57, p = 0.010; Std. beta = -0.08, 95% CI [-0.14, -0.02])\n",
      "  - The interaction effect of measure [post] on interrupt [tiktok] is\n",
      "statistically non-significant and negative (beta = -0.03, 95% CI [-0.10, 0.04],\n",
      "t(1349) = -0.91, p = 0.364; Std. beta = -0.03, 95% CI [-0.10, 0.04])\n",
      "\n",
      "Standardized parameters were obtained by fitting the model on a standardized\n",
      "version of the dataset. 95% Confidence Intervals (CIs) and p-values were\n",
      "computed using a Wald t-distribution approximation.\n"
     ]
    }
   ],
   "source": [
    "print(report(m4), verbose=TRUE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LD Task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning message in checkConv(attr(opt, \"derivs\"), opt$par, ctrl = control$checkConv, :\n",
      "“Model failed to converge with max|grad| = 0.0132157 (tol = 0.002, component 1)”\n",
      "refitting model(s) with ML (instead of REML)\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"dataframe\">\n",
       "<caption>A anova: 7 × 8</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>npar</th><th scope=col>AIC</th><th scope=col>BIC</th><th scope=col>logLik</th><th scope=col>deviance</th><th scope=col>Chisq</th><th scope=col>Df</th><th scope=col>Pr(&gt;Chisq)</th></tr>\n",
       "\t<tr><th></th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>m0</th><td> 6</td><td> 6184.038</td><td> 6230.981</td><td>-3086.019</td><td> 6172.038</td><td>        NA</td><td>NA</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m1</th><td>10</td><td>-3442.307</td><td>-3364.069</td><td> 1731.154</td><td>-3462.307</td><td>9634.34476</td><td> 4</td><td> 0.00000e+00</td></tr>\n",
       "\t<tr><th scope=row>m2</th><td>10</td><td>-3473.944</td><td>-3395.705</td><td> 1746.972</td><td>-3493.944</td><td>  31.63659</td><td> 0</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m3</th><td>10</td><td>-4978.996</td><td>-4900.757</td><td> 2499.498</td><td>-4998.996</td><td>1505.05228</td><td> 0</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m4</th><td>11</td><td>-6184.898</td><td>-6098.835</td><td> 3103.449</td><td>-6206.898</td><td>1207.90185</td><td> 1</td><td>1.16953e-264</td></tr>\n",
       "\t<tr><th scope=row>m5</th><td>11</td><td>-6349.078</td><td>-6263.015</td><td> 3185.539</td><td>-6371.078</td><td> 164.17986</td><td> 0</td><td>          NA</td></tr>\n",
       "\t<tr><th scope=row>m6</th><td>11</td><td>-7625.126</td><td>-7539.064</td><td> 3823.563</td><td>-7647.126</td><td>1276.04821</td><td> 0</td><td>          NA</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A anova: 7 × 8\n",
       "\\begin{tabular}{r|llllllll}\n",
       "  & npar & AIC & BIC & logLik & deviance & Chisq & Df & Pr(>Chisq)\\\\\n",
       "  & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl>\\\\\n",
       "\\hline\n",
       "\tm0 &  6 &  6184.038 &  6230.981 & -3086.019 &  6172.038 &         NA & NA &           NA\\\\\n",
       "\tm1 & 10 & -3442.307 & -3364.069 &  1731.154 & -3462.307 & 9634.34476 &  4 &  0.00000e+00\\\\\n",
       "\tm2 & 10 & -3473.944 & -3395.705 &  1746.972 & -3493.944 &   31.63659 &  0 &           NA\\\\\n",
       "\tm3 & 10 & -4978.996 & -4900.757 &  2499.498 & -4998.996 & 1505.05228 &  0 &           NA\\\\\n",
       "\tm4 & 11 & -6184.898 & -6098.835 &  3103.449 & -6206.898 & 1207.90185 &  1 & 1.16953e-264\\\\\n",
       "\tm5 & 11 & -6349.078 & -6263.015 &  3185.539 & -6371.078 &  164.17986 &  0 &           NA\\\\\n",
       "\tm6 & 11 & -7625.126 & -7539.064 &  3823.563 & -7647.126 & 1276.04821 &  0 &           NA\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A anova: 7 × 8\n",
       "\n",
       "| <!--/--> | npar &lt;dbl&gt; | AIC &lt;dbl&gt; | BIC &lt;dbl&gt; | logLik &lt;dbl&gt; | deviance &lt;dbl&gt; | Chisq &lt;dbl&gt; | Df &lt;dbl&gt; | Pr(&gt;Chisq) &lt;dbl&gt; |\n",
       "|---|---|---|---|---|---|---|---|---|\n",
       "| m0 |  6 |  6184.038 |  6230.981 | -3086.019 |  6172.038 |         NA | NA |           NA |\n",
       "| m1 | 10 | -3442.307 | -3364.069 |  1731.154 | -3462.307 | 9634.34476 |  4 |  0.00000e+00 |\n",
       "| m2 | 10 | -3473.944 | -3395.705 |  1746.972 | -3493.944 |   31.63659 |  0 |           NA |\n",
       "| m3 | 10 | -4978.996 | -4900.757 |  2499.498 | -4998.996 | 1505.05228 |  0 |           NA |\n",
       "| m4 | 11 | -6184.898 | -6098.835 |  3103.449 | -6206.898 | 1207.90185 |  1 | 1.16953e-264 |\n",
       "| m5 | 11 | -6349.078 | -6263.015 |  3185.539 | -6371.078 |  164.17986 |  0 |           NA |\n",
       "| m6 | 11 | -7625.126 | -7539.064 |  3823.563 | -7647.126 | 1276.04821 |  0 |           NA |\n",
       "\n"
      ],
      "text/plain": [
       "   npar AIC       BIC       logLik    deviance  Chisq      Df Pr(>Chisq)  \n",
       "m0  6    6184.038  6230.981 -3086.019  6172.038         NA NA           NA\n",
       "m1 10   -3442.307 -3364.069  1731.154 -3462.307 9634.34476  4  0.00000e+00\n",
       "m2 10   -3473.944 -3395.705  1746.972 -3493.944   31.63659  0           NA\n",
       "m3 10   -4978.996 -4900.757  2499.498 -4998.996 1505.05228  0           NA\n",
       "m4 11   -6184.898 -6098.835  3103.449 -6206.898 1207.90185  1 1.16953e-264\n",
       "m5 11   -6349.078 -6263.015  3185.539 -6371.078  164.17986  0           NA\n",
       "m6 11   -7625.126 -7539.064  3823.563 -7647.126 1276.04821  0           NA"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_ld_correct <- df %>% filter((task %in% c('LD'))) %>% filter((success %in% c('True'))) %>% filter((correct %in% c('True'))) %>% filter(!(measure %in% c('train')))\n",
    "df_ld_correct$interrupt <- factor(df_ld_correct$interrupt, levels=c('rest', 'twitter', 'youtube', 'tiktok'))\n",
    "df_ld_correct$measure   <- factor(df_ld_correct$measure,   levels=c('pre', 'post'))\n",
    "\n",
    "m0 <- lmer(rt  ~ interrupt + (1|folder_id), data = df_ld_correct)\n",
    "m1 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id), data = df_ld_correct, family = Gamma(link = \"log\"))\n",
    "m2 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id), data = df_ld_correct, family = Gamma(link = \"inverse\"))\n",
    "m3 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id), data = df_ld_correct, family = inverse.gaussian(link = \"inverse\"))\n",
    "m4 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id) + (1|stimulus), data = df_ld_correct, family = Gamma(link = \"log\"))\n",
    "m5 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id) + (1|stimulus), data = df_ld_correct, family = Gamma(link = \"inverse\"))\n",
    "m6 <- glmer(rt ~ interrupt * measure + interrupt + measure + (1|folder_id) + (1|stimulus), data = df_ld_correct, family = inverse.gaussian(link = \"inverse\"))\n",
    "\n",
    "anova(m0, m1, m2, m3, m4, m5, m6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"dataframe\">\n",
       "<caption>A bayesfactor_models: 7 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>Model</th><th scope=col>log_BF</th></tr>\n",
       "\t<tr><th></th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>m1</th><td>interrupt * measure + interrupt + measure + (1 | folder_id)                 </td><td>4797.595</td></tr>\n",
       "\t<tr><th scope=row>m2</th><td>interrupt * measure + interrupt + measure + (1 | folder_id)                 </td><td>4813.413</td></tr>\n",
       "\t<tr><th scope=row>m3</th><td>interrupt * measure + interrupt + measure + (1 | folder_id)                 </td><td>5565.939</td></tr>\n",
       "\t<tr><th scope=row>m4</th><td>interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)</td><td>6164.978</td></tr>\n",
       "\t<tr><th scope=row>m5</th><td>interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)</td><td>6247.068</td></tr>\n",
       "\t<tr><th scope=row>m6</th><td>interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)</td><td>6885.092</td></tr>\n",
       "\t<tr><th scope=row>m0</th><td>interrupt + (1 | folder_id)                                                 </td><td>   0.000</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A bayesfactor\\_models: 7 × 2\n",
       "\\begin{tabular}{r|ll}\n",
       "  & Model & log\\_BF\\\\\n",
       "  & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\tm1 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id)                  & 4797.595\\\\\n",
       "\tm2 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id)                  & 4813.413\\\\\n",
       "\tm3 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id)                  & 5565.939\\\\\n",
       "\tm4 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id) + (1 \\textbar{} stimulus) & 6164.978\\\\\n",
       "\tm5 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id) + (1 \\textbar{} stimulus) & 6247.068\\\\\n",
       "\tm6 & interrupt * measure + interrupt + measure + (1 \\textbar{} folder\\_id) + (1 \\textbar{} stimulus) & 6885.092\\\\\n",
       "\tm0 & interrupt + (1 \\textbar{} folder\\_id)                                                  &    0.000\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A bayesfactor_models: 7 × 2\n",
       "\n",
       "| <!--/--> | Model &lt;chr&gt; | log_BF &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| m1 | interrupt * measure + interrupt + measure + (1 | folder_id)                  | 4797.595 |\n",
       "| m2 | interrupt * measure + interrupt + measure + (1 | folder_id)                  | 4813.413 |\n",
       "| m3 | interrupt * measure + interrupt + measure + (1 | folder_id)                  | 5565.939 |\n",
       "| m4 | interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus) | 6164.978 |\n",
       "| m5 | interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus) | 6247.068 |\n",
       "| m6 | interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus) | 6885.092 |\n",
       "| m0 | interrupt + (1 | folder_id)                                                  |    0.000 |\n",
       "\n"
      ],
      "text/plain": [
       "   Model                                                                       \n",
       "m1 interrupt * measure + interrupt + measure + (1 | folder_id)                 \n",
       "m2 interrupt * measure + interrupt + measure + (1 | folder_id)                 \n",
       "m3 interrupt * measure + interrupt + measure + (1 | folder_id)                 \n",
       "m4 interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)\n",
       "m5 interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)\n",
       "m6 interrupt * measure + interrupt + measure + (1 | folder_id) + (1 | stimulus)\n",
       "m0 interrupt + (1 | folder_id)                                                 \n",
       "   log_BF  \n",
       "m1 4797.595\n",
       "m2 4813.413\n",
       "m3 5565.939\n",
       "m4 6164.978\n",
       "m5 6247.068\n",
       "m6 6885.092\n",
       "m0    0.000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "bayesfactor_models(m1, m2, m3, m4, m5, m6, denominator = m0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Generalized linear mixed model fit by maximum likelihood (Laplace\n",
       "  Approximation) [glmerMod]\n",
       " Family: Gamma  ( log )\n",
       "Formula: rt ~ interrupt * measure + interrupt + measure + (1 | folder_id) +  \n",
       "    (1 | stimulus)\n",
       "   Data: df_ld_correct\n",
       "\n",
       "     AIC      BIC   logLik deviance df.resid \n",
       " -6184.9  -6098.8   3103.4  -6206.9    18458 \n",
       "\n",
       "Scaled residuals: \n",
       "    Min      1Q  Median      3Q     Max \n",
       "-3.3880 -0.5802 -0.2274  0.2892 11.6632 \n",
       "\n",
       "Random effects:\n",
       " Groups    Name        Variance Std.Dev.\n",
       " stimulus  (Intercept) 0.005965 0.07723 \n",
       " folder_id (Intercept) 0.006117 0.07821 \n",
       " Residual              0.085819 0.29295 \n",
       "Number of obs: 18469, groups:  stimulus, 316; folder_id, 60\n",
       "\n",
       "Fixed effects:\n",
       "                              Estimate Std. Error t value Pr(>|z|)    \n",
       "(Intercept)                  -0.256498   0.057769  -4.440 8.99e-06 ***\n",
       "interrupttwitter             -0.038286   0.078639  -0.487   0.6264    \n",
       "interruptyoutube              0.095409   0.077581   1.230   0.2188    \n",
       "interrupttiktok               0.108339   0.078599   1.378   0.1681    \n",
       "measurepost                  -0.008948   0.017218  -0.520   0.6033    \n",
       "interrupttwitter:measurepost -0.019794   0.010546  -1.877   0.0605 .  \n",
       "interruptyoutube:measurepost -0.004369   0.010762  -0.406   0.6848    \n",
       "interrupttiktok:measurepost  -0.049011   0.010566  -4.639 3.51e-06 ***\n",
       "---\n",
       "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
       "\n",
       "Correlation of Fixed Effects:\n",
       "            (Intr) intrrpttw intrrpty intrrpttk msrpst intrrpttw: intrrpty:\n",
       "intrrpttwtt -0.680                                                         \n",
       "interrptytb -0.690  0.506                                                  \n",
       "intrrpttktk -0.680  0.500     0.507                                        \n",
       "measurepost -0.148  0.020     0.018    0.021                               \n",
       "intrrpttwt:  0.046 -0.067    -0.034   -0.034    -0.307                     \n",
       "intrrptytb:  0.045 -0.033    -0.071   -0.033    -0.300  0.491              \n",
       "intrrpttkt:  0.046 -0.034    -0.034   -0.068    -0.307  0.501      0.490   "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "summary(m4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "vscode": {
     "languageId": "r"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We fitted a general linear mixed model (Gamma family with a log link)\n",
      "(estimated using ML and Nelder-Mead optimizer) to predict rt with interrupt and\n",
      "measure\n",
      " (formula: rt ~ interrupt * measure + interrupt + measure). The model included\n",
      "folder_id and stimulus\n",
      " as random effects (formula: list(~1 | folder_id, ~1 | stimulus)). The model's\n",
      "total explanatory power is moderate (conditional R2 = 0.16) and the part\n",
      "related to the fixed effects alone (marginal R2) is of 0.04. The model's\n",
      "intercept, corresponding to interrupt = rest and measure = pre\n",
      ", is at -0.26 (95% CI [-0.37, -0.14], t(18458) = -4.44, p < .001). Within this\n",
      "model:\n",
      "\n",
      "  - The effect of interrupt [twitter] is statistically non-significant and\n",
      "negative (beta = -0.04, 95% CI [-0.19, 0.12], t(18458) = -0.49, p = 0.626; Std.\n",
      "beta = -0.04, 95% CI [-0.19, 0.12])\n",
      "  - The effect of interrupt [youtube] is statistically non-significant and\n",
      "positive (beta = 0.10, 95% CI [-0.06, 0.25], t(18458) = 1.23, p = 0.219; Std.\n",
      "beta = 0.10, 95% CI [-0.06, 0.25])\n",
      "  - The effect of interrupt [tiktok] is statistically non-significant and\n",
      "positive (beta = 0.11, 95% CI [-0.05, 0.26], t(18458) = 1.38, p = 0.168; Std.\n",
      "beta = 0.11, 95% CI [-0.05, 0.26])\n",
      "  - The effect of measure [post] is statistically non-significant and negative\n",
      "(beta = -8.95e-03, 95% CI [-0.04, 0.02], t(18458) = -0.52, p = 0.603; Std. beta\n",
      "= -8.95e-03, 95% CI [-0.04, 0.02])\n",
      "  - The interaction effect of measure [post] on interrupt [twitter] is\n",
      "statistically non-significant and negative (beta = -0.02, 95% CI [-0.04,\n",
      "8.78e-04], t(18458) = -1.88, p = 0.061; Std. beta = -0.02, 95% CI [-0.04,\n",
      "8.78e-04])\n",
      "  - The interaction effect of measure [post] on interrupt [youtube] is\n",
      "statistically non-significant and negative (beta = -4.37e-03, 95% CI [-0.03,\n",
      "0.02], t(18458) = -0.41, p = 0.685; Std. beta = -4.37e-03, 95% CI [-0.03,\n",
      "0.02])\n",
      "  - The interaction effect of measure [post] on interrupt [tiktok] is\n",
      "statistically significant and negative (beta = -0.05, 95% CI [-0.07, -0.03],\n",
      "t(18458) = -4.64, p < .001; Std. beta = -0.05, 95% CI [-0.07, -0.03])\n",
      "\n",
      "Standardized parameters were obtained by fitting the model on a standardized\n",
      "version of the dataset. 95% Confidence Intervals (CIs) and p-values were\n",
      "computed using a Wald t-distribution approximation.\n"
     ]
    }
   ],
   "source": [
    "print(report(m4), verbose=TRUE)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.2.1"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}