add first analysis of first data v1

experiments/pilot-indirectSource/analysis/analysis-script-pilot-v1.Rmd

@@ -0,0 +1,145 @@
+---
+title: "Analysis Script Pilot Indirect Source v1"
+author: "Hening Wang"
+date: "2024-11-18"
+output: pdf_document
+editor_options: 
+  markdown: 
+    wrap: 72
+---
+Supress messages in this document so that the output is cleaner
+```{r setup, include=FALSE} 
+knitr::opts_chunk$set(warning = FALSE, message = FALSE) 
+```
+Import packages.
+```{r}
+rm(list = ls())
+
+options(warn = -1)
+library(tidyverse)
+library(lme4)
+library(lmerTest)
+library(ggplot2)
+library(aida)
+```
+
+Set up theme for ggplot.
+
+```{r}
+theme_set(theme_aida())
+
+##################################################
+## CSP-colors
+##################################################
+CSP_colors = c(
+  "#7581B3", "#99C2C2", "#C65353", "#E2BA78", "#5C7457", "#575463",
+  "#B0B7D4", "#66A3A3", "#DB9494", "#D49735", "#9BB096", "#D4D3D9",
+  "#414C76", "#993333"
+  )
+# setting theme colors globally
+scale_colour_discrete <- function(...) {
+  scale_colour_manual(..., values = CSP_colors)
+}
+scale_fill_discrete <- function(...) {
+  scale_fill_manual(..., values = CSP_colors)
+}
+```
+
+# Data Import & Preprocessing
+
+Import data
+
+```{r}
+data <- read.csv("../data/pilot-v1/results.csv")
+```
+
+Factorise variables: id, item, informationSource, listenerRole
+
+```{r}
+data$id <- as.factor(1:nrow(data))
+data$item <- as.factor(data$condition)
+data$informationSource <- as.factor(data$informationSource)
+data$listenerRole <- as.factor(data$listenerRole)
+```
+
+Recruitment criteria: 1. English native speakers 2. Approval rate \> 90%
+3. Completion \> 5 times 4. Not included in the prior studies of
+CommuniCause
+
+# Plots
+
+## A histogram showing the distribution of probs. 
+**Interpretation: Data is left-skewed.**
+
+```{r}
+ggplot(data, aes(x = probs)) +
+  geom_histogram(binwidth = 5, fill = CSP_colors[1]) +
+  labs(title = "Distribution of Slider Ratings",
+       x = "Slider Ratings",
+       y = "Frequency")
+
+```
+
+## Plot the dependent variable "probs" against conditions.
+
+**Interpretation:**
+
+1. No significant difference between conditions. 
+2. Perhaps due to:
+  + not enough sample size.
+  + the empirical distribution is severe skewed. This is problem of slider rating in general.
+  + manipulation too subtle.
+3. We can try to observe some trends though: 
+  + "Direct" information seems to have a higher mean rating than "indirect", as expected. 
+  + The role "scientist" seems to be more sensitive to indirectness of information. This could lead to a potential interaction as expected.
+
+Next step: It seems that manipulation did have some effects given 3.2, but it was too subtle to be statistically significant. We should probably rework the design of the study by addressing the three points in 2.
+
+```{r}
+ggplot(data, aes(x = informationSource, y = probs, fill = listenerRole)) +
+  geom_boxplot() +
+  labs(title = "Slider Ratings by Conditions",
+       x = "Information Source",
+       y = "Slider Ratings",
+       fill = "Listener Role")
+```
+
+## Plot transformed data
+
+Given that data is left-skewed. We perform a log transformation, and
+show a QQ plot after data transformation.
+
+It does not really help. Now the data is right-skewed.
+
+```{r}
+data$transformed_probs <- log(max(data$probs+1) - data$probs)
+qqnorm(data$transformed_probs)
+```
+
+Another empirical plot with transformed data.
+Nothing really interesting here.
+```{r}
+ggplot(data, aes(x = informationSource, y = transformed_probs, fill = listenerRole)) +
+  geom_boxplot() +
+  labs(title = "log transformed Slider Ratings by Conditions",
+       x = "Information Source",
+       y = "Slider Ratings",
+       fill = "Listener Role")
+```
+
+# Statistical Analysis
+**Note: Nothing is significant here.** 
+
+Fit a linear model with full interaction.
+```{r}
+model_full <- lm(probs ~ informationSource * listenerRole, data = data)
+summary(model_full)
+
+```
+
+Fit another lm with transformed data.
+
+```{r}
+model_full_transformed <- lm(transformed_probs ~ informationSource * listenerRole, data = data)
+summary(model_full_transformed)
+```

experiments/pilot-indirectSource/analysis/analysis.Rproj

@@ -0,0 +1,13 @@
+Version: 1.0
+
+RestoreWorkspace: Default
+SaveWorkspace: Default
+AlwaysSaveHistory: Default
+
+EnableCodeIndexing: Yes
+UseSpacesForTab: Yes
+NumSpacesForTab: 2
+Encoding: UTF-8
+
+RnwWeave: Sweave
+LaTeX: pdfLaTeX

experiments/pilot-indirectSource/data/pilot-v1/results.csv

@@ -0,0 +1,41 @@
+submission_id,age,attentionCheck,attentionCheckResult,comments,condition,education,experiment_duration,experiment_end_time,experiment_start_time,gender,informationSource,itemName,itemNr,languages,listenerRole,listenerTyp,probs,prolific_pid,prolific_session_id,prolific_study_id,responseTime,source,trialNR
+22,28,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,84157,1731674211159,1731674127002,male,direct,spaceColony,3,"Italian, English",colonist,,100,57e00216995a0b0001635a8a,673740038bc565914e82c6e8,67373c1dff5d8c820f450e62,38718,,0
+16,35,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated College,58863,1731674065436,1731674006573,male,direct,spaceColony,1,English,scientist,,67,61116d2487719a54001938b6,67373f8ef75ddc654a8c1fef,67373c1dff5d8c820f450e62,23626,,0
+17,39,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Higher degree,82821,1731674096710,1731674013889,female,direct,spaceColony,1,english,scientist,,99,5c0ea6ec974277000123ba5f,67373f96f97a36e40efb3423,67373c1dff5d8c820f450e62,50888,,0
+18,24,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,110248,1731674160358,1731674050110,female,indirect,spaceColony,4,english ,colonist,,94,5d69716987f62800187f378e,67373fbc244e4c88d99b29f4,67373c1dff5d8c820f450e62,66286,,0
+19,21,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Higher degree,82273,1731674165989,1731674083716,female,direct,spaceColony,1,English,scientist,,100,661a84526ee9e523c36c66db,67373fd96517f1bdde5d3965,67373c1dff5d8c820f450e62,44266,,0
+20,39,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated High-school,135683,1731674192779,1731674057096,female,direct,spaceColony,1,english,scientist,,100,5de3bb4674c6793bb8b1fdcc,67373fbee645a699857592d0,67373c1dff5d8c820f450e62,70246,,0
+21,39,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,indirect_scientist,Graduated College,114576,1731674198698,1731674084122,female,indirect,spaceColony,2,English,scientist,,100,5ff7320620d7133805a3cac7,67373fd9eed183810e118ac2,67373c1dff5d8c820f450e62,63255,,0
+23,40,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,219273,1731674256696,1731674037423,male,indirect,spaceColony,4,English,colonist,,49,5c3e80671580e9000122d168,67373fac0b20fdd7f67bbb9e,67373c1dff5d8c820f450e62,104787,,0
+24,21,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,88082,1731674257217,1731674169135,male,indirect,spaceColony,4,English,colonist,,75,63d40b4e0dd2e63262f403df,673740310b17f5ed953c8a4c,67373c1dff5d8c820f450e62,51013,,0
+27,33,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,92739,1731674249932,1731674157193,female,indirect,spaceColony,4,English,colonist,,66,5eab170759f5390b776df5a1,6737403bcf5abb90fd9198ec,67373c1dff5d8c820f450e62,51744,,0
+31,24,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,119316,1731674292287,1731674172971,female,indirect,spaceColony,4,English,colonist,,81,671aa8096357a79c1a5b3811,6737402a76d5015607ccaac1,67373c1dff5d8c820f450e62,54871,,0
+39,35,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,159161,1731674380469,1731674221308,female,indirect,spaceColony,4,English (UK),colonist,,80,6172de67839e8a220d6648b8,673740600f3c7eb70ca6ec89,67373c1dff5d8c820f450e62,61962,,0
+25,26,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,indirect_scientist,Graduated High-school,152146,1731674268166,1731674116020,male,indirect,spaceColony,2,English,scientist,,51,615d8a94a76a15626da937c5,67373ffd5cab06bf163a516e,67373c1dff5d8c820f450e62,36466,,0
+26,24,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,99616,1731674273092,1731674173476,female,direct,spaceColony,3,english,colonist,,21,5fecf25f2c2ee6a373d61c65,67374033601ddf376ef340b9,67373c1dff5d8c820f450e62,7456,,0
+34,50,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,indirect_scientist,Graduated High-school,123734,1731674315461,1731674191727,male,indirect,spaceColony,2,English,scientist,,83,627218ff9eeedee1aa9c45f5,67374040f97a36e40efb3458,67373c1dff5d8c820f450e62,87236,,0
+37,30,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated College,155792,1731674348803,1731674193011,male,direct,spaceColony,1,english,scientist,,90,6426124fd4520b59c2330332,67374042135bbac77fe09a85,67373c1dff5d8c820f450e62,71430,,0
+28,24,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,indirect_scientist,Graduated College,206802,1731674276088,1731674069286,other,indirect,spaceColony,2,English,scientist,,70,60dee81a675c2b4280e62726,67373fcbcba3b77571418a88,67373c1dff5d8c820f450e62,140259,,0
+29,53,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,indirect_scientist,Higher degree,117190,1731674249531,1731674132341,female,indirect,spaceColony,2,English,scientist,,69,5f589d27b9bdc9016c605c7e,67374029b6a6d1a4211f277d,67373c1dff5d8c820f450e62,54156,,0
+30,31,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,108892,1731674292426,1731674183534,male,indirect,spaceColony,4,English,colonist,,100,5ebfe132d46d24025eccd30d,6737403aeb242cad3b625335,67373c1dff5d8c820f450e62,31858,,0
+32,65,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,171879,1731674304306,1731674132427,male,direct,spaceColony,3,English,colonist,,100,63076cd447ae6e2366804685,67374005c30edb7574759629,67373c1dff5d8c820f450e62,105174,,0
+33,41,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated High-school,63876,1731674309161,1731674245285,female,direct,spaceColony,1,english,scientist,,25,5eb9499bcbc2140180e31037,6737407d77a96cc286967fc4,67373c1dff5d8c820f450e62,50858,,0
+35,26,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated College,177336,1731674318959,1731674141623,female,direct,spaceColony,1,ENGLISH,scientist,,88,63d50bcf290a52a9f24cc357,6737400f490c87a0dd116a3e,67373c1dff5d8c820f450e62,114342,,0
+38,49,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,182917,1731674362239,1731674179322,male,indirect,spaceColony,4,english,colonist,,100,572bcdbb34b25a0010ddd977,673740378e905a73f01cf718,67373c1dff5d8c820f450e62,77303,,0
+40,44,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,104663,1731674383170,1731674278507,female,indirect,spaceColony,4,English,colonist,,90,653bb9ac59c92c4b9f32f258,67374090834499a073b86dcc,67373c1dff5d8c820f450e62,64215,,0
+36,56,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,131250,1731674336781,1731674205531,female,direct,spaceColony,3,english,colonist,,76,66feea3ea82be9c8e036bd6e,67374051b57b3be2f4a6ed21,67373c1dff5d8c820f450e62,66880,,0
+41,28,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Higher degree,112506,1731674393558,1731674281052,male,direct,spaceColony,1,"English, Welsh",scientist,,95,628f93509647b89df3644ad1,6737409f117151cafbc009ad,67373c1dff5d8c820f450e62,56796,,0
+42,58,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,115279,1731674395538,1731674280259,female,direct,spaceColony,3,english,colonist,,76,60c6906540d3bc4eed4c35a0,6737409a513a6110e34b0e6c,67373c1dff5d8c820f450e62,53149,,0
+43,50,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,127911,1731674403714,1731674275803,male,direct,spaceColony,3,English,colonist,,61,56296779c8ffc200055134c9,67374095135bbac77fe09a94,67373c1dff5d8c820f450e62,57045,,0
+44,30,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Higher degree,206760,1731674417852,1731674211092,male,direct,spaceColony,1,English,scientist,,91,568d0641b5a2c2000cb657d0,6737405ee77fb67372825e0f,67373c1dff5d8c820f450e62,108988,,0
+45,22,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,Interesting survey,direct_colonist,Higher degree,136728,1731674440035,1731674303307,male,direct,spaceColony,3,English,colonist,,80,662fa014fd36bb7b70097c24,673740b4aceb9e06c5ccccea,67373c1dff5d8c820f450e62,67415,,0
+46,47,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Higher degree,306472,1731674453561,1731674147089,male,direct,spaceColony,1,English,scientist,,92,6554cfc335ac9ff968a5f162,67373fedcf3e3a0cb79b29fc,67373c1dff5d8c820f450e62,238960,,0
+47,29,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated College,189325,1731674460366,1731674271041,male,direct,spaceColony,1,"english, japanese",scientist,,79,5d7f74f4ed58b60016490887,67374071f852ab52f5b5af50,67373c1dff5d8c820f450e62,132840,,0
+48,46,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,None,direct_scientist,Graduated College,179794,1731674485554,1731674305760,male,direct,spaceColony,1,English,scientist,,100,6661734a551b4354f952cd57,673740af0b7cebfefd3e85fc,67373c1dff5d8c820f450e62,78824,,0
+49,24,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,direct_scientist,Graduated High-school,115060,1731674501310,1731674386250,female,direct,spaceColony,1,English,scientist,,52,5f8375aef397bf12ad765a62,67373f9ace8dec57ac9fcbbd,67373c1dff5d8c820f450e62,53554,,0
+50,38,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,177975,1731674518247,1731674340272,female,direct,spaceColony,3,english,colonist,,100,5ffcf2992e58f31b272ab721,673740aa4860c94a523f131f,67373c1dff5d8c820f450e62,63445,,0
+51,55,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Higher degree,435195,1731674646569,1731674211374,female,direct,spaceColony,3,English,colonist,,66,5713cc324cfd6f0009aa7329,673740543a11eec3de007729,67373c1dff5d8c820f450e62,77116,,0
+52,29,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated College,299893,1731674646822,1731674346929,female,indirect,spaceColony,4,"English , IsiZulu",colonist,,61,5fb57363656d5607d4a94149,673740b5556763cc6f01579d,67373c1dff5d8c820f450e62,139704,,0
+53,78,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated High-school,355294,1731674647938,1731674292644,female,indirect,spaceColony,4,English,colonist,,50,60b94f837bfda8b1977feef0,673740994649b52eeff1f7af,67373c1dff5d8c820f450e62,137400,,0
+54,23,Ensure that there is a steady supply of Xeliherb to meet daily survival needs,true,,direct_colonist,Graduated High-school,187239,1731674891503,1731674704264,female,direct,spaceColony,3,English,colonist,,99,606dea112436631b694c56ad,6737422ccf5b84eaeaf6fba6,67373c1dff5d8c820f450e62,88100,,0
+55,39,Gather data to uncover patterns in Xeliherb growth that could inform future research,true,,indirect_scientist,Higher degree,815226,1731675079403,1731674264177,female,indirect,spaceColony,2,Xhosa,scientist,,89,5fa4faf94527261b54915c03,6737408881a9922c5bb08ec6,67373c1dff5d8c820f450e62,21900,,0