ucpd_facts.Rmd

---
title: "University of Chicago Police Department Open Data Analysis"
output:
  html_document:
    toc: true
    toc_float: true
    df_print: paged
    theme: sandstone
    number_sections: true
---

```{r initial_setup, include=FALSE}
library(tidyverse)
library(lubridate)
library(kableExtra)
library(scales)
library(sf)
library(tigris)
library(tidycensus)
library(leaflet)
library(ggplot2)

knitr::opts_chunk$set(echo = FALSE, warning = FALSE, message = FALSE)

#prepare data
traffic <- read_csv("files/traffic_cleaned.csv")
traffic <- traffic %>% mutate(Year = year(Date))
traffic <- traffic %>% filter(complete.cases(traffic),
                              between(Year, 2015, 2022))

interview <- read_csv("files/interview_cleaned.csv")
interview <- interview %>% mutate(Year = year(Date)) %>%
  filter(between(Year, 2015, 2022))

incident <- read_csv("files/incident_cleaned.csv")
incident <- incident %>% mutate(Year = year(Occured)) %>%
  filter(between(Year, 2015, 2022), Outcome != "void")
incident$Month <- as.Date(cut(incident$Occured,
  breaks = "month"))

traffic <- traffic %>% mutate(is_black = ifelse(Race == "African American", 1, 0),
                              month = as.Date(cut(Date, breaks = "month")),
                              week = as.Date(cut(Date, breaks = "week")),
                              searched = ifelse(Search == "Yes", 1, 0),
                          cited = ifelse(Disposition == "Citation", 1, 0),
                          searched_and_cited = ifelse(searched == 1 & cited == 1, 1, 0),
                          searched_and_not_cited = ifelse(searched == 1 & cited == 0, 1, 0),
                          warning = ifelse(Disposition == "Warning", 1, 0))

traffic_full <- traffic

interview <- interview %>% mutate(is_black = ifelse(Race == "African American", 1, 0),
                                  searched = ifelse(Search == "Yes", 1, 0))
mini_traffic <- traffic %>% filter(Disposition != "Arrest", Race != "NHPI or Indian", `IDOT Classification` != "Other")

#geospatial data
UCPD_area <- st_read("files/Areas_UCPD/Areas.shp")

cook_pop <-
  get_acs(geography = "tract",
        variables = c("B02001_001", "B02001_002", "B02001_003"),
        state = "17",
        county = "031",
        cache_table = TRUE,
        output = "wide",
        geometry = TRUE)
cook_pop <- cook_pop[, -grep("\\dM", colnames(cook_pop))]

cook_pop <- cook_pop %>% rename(total_pop = B02001_001E,
                                white_pop = B02001_002E,
                                black_pop = B02001_003E)
cook_pop <- st_transform(cook_pop, st_crs(UCPD_area))
```

```{r fact_strs}
fact_1 <- paste0(traffic %>% summarize(sum(is_black, na.rm=T) / n()) %>% as.numeric() %>% percent(., .1), " of traffic stops and ", interview %>% summarize(sum(is_black) / n()) %>% as.numeric() %>% percent(., .1))

cited_df <- mini_traffic %>% group_by(is_black) %>% summarize(sum(cited) / n())

fact_2 <- paste0(percent(as.numeric(cited_df[2,2]), .1), " vs. ", 
                 percent(as.numeric(cited_df[1,2]), .1))


search_rate_tbl <- mini_traffic %>% group_by(is_black) %>% summarize(search_rate = sum(searched) / n(),
                                                                     about_in = as.integer(1 / search_rate))

searched_share_tbl <- mini_traffic %>% group_by(searched) %>% summarize(searched_share = sum(is_black) / n())
fact_3 <- paste0("<b>1 out of ", search_rate_tbl[2,3][[1]], 
                 "</b> African American drivers it stops, compared to <b>1 out of ",
                 search_rate_tbl[1,3][[1]],
                 "</b> non-African Americans motorists, with <b>",
                 percent(searched_share_tbl[2,2][[1]], .1),
                 "</b> of all vehicle searches conducted on African Americans")
```


# Introduction

UCPD releases three types of data on their data page. This data is not directly downloadable and was web scraped into a database which was then standardized. The three types of data include *[The Daily Incident Report](https://incidentreports.uchicago.edu/)*, which includes crimes and fires reported to UCPD in its patrol area, *[Traffic Stops](https://incidentreports.uchicago.edu/trafficStops.php)*, and *[Field Interviews](https://incidentreports.uchicago.edu/fieldInterviews.php)*.

# Summary

*Data includes `r str_glue("{min(incident$Occured, na.rm=T)} to {max(incident$Occured, na.rm=T)}")`.*

The University of Chicago Police Department (UCPD) polices about 3.5 square miles of Chicago's south side. About 50,000 Chicagoans, of which about 50% are African American, live permanently in their patrol area. UCPD's data shows that African Americans are stopped, searched, and cited at rates higher than any other racial group. Including that:

- Since 2015, <b>`r fact_1` of street interviews</b> UCPD conducted were with African Americans
- During traffic stops, African Americans received citations at almost double the rate of non-African Americans <b>(`r fact_2`)</b>
- UCPD searches `r fact_3`
- Half of all searches conducted on African Americans ended without a citation
- 36 out of 37 traffic stop searches in 2019 were conducted on African American drivers
- In 2019, African Americans were cited at 10.8% and non-African Americans were cited at 6.0%

## Last 365 Days

```{r}
maxd <- max(incident$Occured)
mind <- maxd - 365

lasty_traffic <- traffic %>% filter(Date >= max(Date) - 365*24*60*60)
lasty_interview <- interview%>% filter(Date >= max(Date) - 365*24*60*60)

lasty_searched <- lasty_traffic %>% count(searched, is_black)
lasty_cited <- lasty_traffic %>% count(cited, is_black)

s1 <- lasty_searched %>% summarize(n = sum(n[searched == 1])) %>% pull(n)
s2 <- lasty_searched %>% summarize(n = sum(n[searched == 1 & is_black == 1])) %>% pull(n)

i1 <- lasty_cited %>% summarize(n = sum(n[cited == 1])) %>% pull(n)
i2 <- lasty_cited %>% summarize(n = sum(n[cited == 1 & is_black == 1])) %>% pull(n)
```


For the last 12 months of data which range from `r str_glue("{mind} to {maxd}")`.

- For the last 12 months, `r str_glue("there have been {nrow(lasty_traffic)} traffic stops. {sum(lasty_traffic$is_black)} drivers were African American ({percent(sum(lasty_traffic$is_black) / nrow(lasty_traffic))}).")`
- For the last 12 months, `r str_glue("there have been {nrow(lasty_interview)} interviews. {sum(lasty_interview$is_black)} of the interviewed were African American ({percent(sum(lasty_interview$is_black) / nrow(lasty_interview))}).")`
- For the last 12 months, UCPD has searched `r str_glue("{s1} drivers. {s2} were African American ({percent(s2/s1)}).")`
- For the last 12 months, UCPD has issued `r str_glue("{i1} citations. {i2} were issued to African American drivers ({percent(i2/i1)}).")` 

# Dashboard Overview of Police Actions
```{r}
traffic_cnts <- traffic_full %>% group_by(month) %>% summarize(Quantity = n(),
                                                          Citations = sum(cited))

traffic_cnts2 <- traffic_full %>% group_by(month, is_black) %>% summarize(Citation_rate = sum(cited) / n()) %>% mutate(cat_name = ifelse(is_black == 0, "Non-African American", "African American"))

traffic_cnts25 <- traffic_full %>% filter(Race %in% c("African American", "Caucasian")) %>% group_by(month, Race) %>% summarize(Citation_rate = sum(cited) / n())
traffic_cnts3 <- traffic_full %>% group_by(month, Race) %>% summarize(citation_num = n()) %>% filter(Race != "NHPI or Indian")

ggplot(data = traffic_cnts, aes(month, Quantity)) + 
  geom_smooth(se=FALSE, color='black', size=2) + 
  geom_point(alpha = 0.2) + 
  geom_smooth(aes(y=Citations),se=FALSE, size=2) + 
  geom_point(aes(y=Citations),color='blue', alpha=0.2) +
  labs(x=NULL, y="Number of Actions per Month", title="Average Monthly Number of Traffic Actions", subtitle="Stops (black). Citations (blue).") + theme_classic()

ggplot(data = traffic_cnts2, aes(month, Citation_rate, color=factor(cat_name))) + 
  geom_smooth(se=FALSE, size=2) + 
  geom_point(alpha = 0.2) + 
  labs(x=NULL, y="Percent Cited", title="Citation Rates for African Americans vs. Non-African Americans", color="Racial Group") + theme_classic()

ggplot(data = traffic_cnts25, aes(month, Citation_rate, color=factor(Race))) + 
  geom_smooth(se=FALSE, size=2) + 
  geom_point(alpha = 0.2) + 
  labs(x=NULL, y="Percent Cited", title="Citation Rates for African Americans vs. Caucasians", color="Race") + theme_classic()

ggplot(data = traffic_cnts3, aes(month, citation_num, color=factor(Race))) + 
  geom_smooth(se=FALSE, size=2) + 
  geom_point(alpha = 0.2) + 
  labs(x=NULL, y="Number of Citations per month", title="Traffic Citations by Race", color="Racial Group") + theme_classic()



theft_by_month <- incident %>% filter(Date=="Theft") %>% group_by(Month) %>% summarize(cnt=n())

ggplot(data=theft_by_month, aes(Month, cnt)) + geom_smooth() + geom_point(alpha = 0.2) +
  labs(x=NULL, y="Thefts per Month", title="Reported Thefts per Month") + theme_classic()

battery_by_month <- incident %>% filter(Date=="Battery") %>% group_by(Month) %>% summarize(cnt=n())

ggplot(data=battery_by_month, aes(Month, cnt)) + geom_smooth() + geom_point(alpha = 0.2) +
  labs(x=NULL, y="Battery per Month", title="Reported Batteries per Month") + theme_classic()

assault_by_month <- incident %>% filter(Date=="Assault") %>% group_by(Month) %>% summarize(cnt=n())

ggplot(data=assault_by_month, aes(Month, cnt)) + geom_smooth() + geom_point(alpha = 0.2) +
  labs(x=NULL, y="Assault per Month", title="Reported Assaults per Month") + theme_classic()

ggplot(mini_traffic %>% group_by(Race) %>% summarize(`Citation Rate Once Stopped` = sum(cited) / n()), aes(Race, `Citation Rate Once Stopped`)) + geom_bar(position="dodge", stat="identity") + labs(title="Traffic Citation Rate") + theme_minimal() +
  scale_y_continuous(labels=percent_format()) +
  theme(text = element_text(size=15))

ggplot(mini_traffic %>% group_by(Race) %>% summarize(`Search Rate Once Stopped` = sum(searched) / n()), aes(Race, `Search Rate Once Stopped`)) + geom_bar(position="dodge", stat="identity") + labs(title="Traffic Search Rate") + theme_minimal() +
  scale_y_continuous(labels=percent_format()) +
  theme(text = element_text(size=15))

ggplot(interview %>% filter(Race != "other") %>% group_by(Race) %>% summarize(`Number of Searches` = sum(searched)), 
       aes(Race, `Number of Searches`)) + labs(title="Interview Searches") + geom_bar(position="dodge", stat="identity") + theme_minimal() +
  theme(text = element_text(size=15))

```






# Summary Tables

## Number of interactions with UCPD by Year
```{r}
kable(traffic %>% group_by(Year) %>% 
        summarize(Total = n(), `% African American` = percent(sum(is_black) / n(), .1)), 
      caption="Table 1: Traffic Stops") %>% kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

kable(incident %>% group_by(Year) %>% 
        summarize(Total = n()), caption="Table 2: Incidents") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

kable(interview %>% group_by(Year) %>% summarize(Total = n(), `% African American` = percent(sum(is_black) / n(), .1)), 
      caption="Table 3: Interviews") %>% kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)
```

## Traffic Outcomes

```{r}
kable(traffic %>% group_by(Year, Disposition) %>% 
  summarize(Total = n()) %>% 
  pivot_wider(names_from = c("Disposition"), values_from = Total) %>%
  mutate(Arrest = replace_na(Arrest, 0)), caption="Table 4: Traffic Outcomes") %>% kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)


race_cnts <- mini_traffic %>% group_by(Race) %>% summarize(`Total (by Race)` = n())

dis_race <- mini_traffic %>% group_by(Disposition, Race) %>% 
        summarize(Total = n()) %>% left_join(race_cnts) %>%
  mutate(`Share in Category` = percent(Total / `Total (by Race)`, .1))

search_race <- mini_traffic %>% group_by(Search, Race) %>% 
        summarize(Total = n()) %>% left_join(race_cnts) %>%
  mutate(`Share in Category` = percent(Total / `Total (by Race)`, .1))

violation_race <- mini_traffic %>% group_by(`IDOT Classification`, Race) %>% 
        summarize(Total = n()) %>% left_join(race_cnts) %>%
  mutate(`Share in Category` = percent(Total / `Total (by Race)`, .1)) %>% select(-Total, -`Total (by Race)`) %>% 
  pivot_wider(names_from = c("Race"), values_from = `Share in Category`)

kable(dis_race %>% select(-`Total (by Race)`), caption="Table 5: Traffic Outcomes by Race") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

kable(search_race %>% select(-`Total (by Race)`), caption="Table 6: Traffic Search Outcomes by Race") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

kable(violation_race, caption="Table 7: Traffic Violation Outcomes by Race") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

all_interactions <- 
  mini_traffic %>% group_by(Year, Race) %>% summarize(pct_search = sum(searched) / n(),
                                                     pct_cited = sum(cited) / n(),
                                                     pct_search_cite = sum(searched_and_cited) / n(),
                                                     pct_search_nocite = sum(searched_and_not_cited) / n(),
                                                     pct_warning = sum(warning) / n(),
                                                     count = n())
all_interactions <- all_interactions %>% mutate_at(vars(pct_search:pct_warning), percent)

kable(all_interactions, caption = "Table 8: Traffic Stop Outcomes by Race and Year") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)


```





## Interview Outcomes
```{r}
mini_interview <- interview %>% filter(!is.na(Search), Race != "Hispanic")

race_cnts_interview <- mini_interview %>% group_by(Race) %>% summarize(`Total (by Race)` = n())

int_search_race <- 
  mini_interview %>% group_by(Search, Race) %>% summarize(Total = n()) %>% 
  left_join(race_cnts_interview) %>%
  mutate(`Share in Category` = percent(Total / `Total (by Race)`, .1))

int_disp_race <- 
  mini_interview %>% group_by(Disposition, Race) %>% summarize(Total = n()) %>% 
  left_join(race_cnts_interview) %>%
  mutate(`Share in Category` = percent(Total / `Total (by Race)`, .1)) %>% select(-Total, -`Total (by Race)`) %>% 
  pivot_wider(names_from = c("Race"), values_from = `Share in Category`) %>% mutate(Caucasian = replace_na(Caucasian, "0%"))

tbldata <- interview %>% group_by(`Initiated By`, Year) %>% count() %>% pivot_wider(names_from = "Year", values_from = "n")

kable(tbldata, caption = "Table 9: Interview Initiated By") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

kable(int_search_race %>% select(-`Total (by Race)`), caption="Table 10: Interview Search Outcomes by Race") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

kable(int_disp_race, caption="Table 11: Interview Disposition Outcomes by Race") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)


```

## Incident Outcomes
```{r}
ggplot(incident %>% filter(Year > 2015), aes(Month)) + geom_bar() + 
  labs(y="Number of Incidents Per Month") + theme_minimal()

outcome_by_year <- incident %>% group_by(Year) %>% count(Outcome)

ggplot(outcome_by_year %>% filter(Outcome == "arrested"), aes(Year, n)) + geom_bar(position="dodge", stat="identity") + theme_minimal() + labs(y="Arrests per Year") +
  theme(text = element_text(size=15))

ggplot(outcome_by_year %>% filter(Outcome == "closed"), aes(Year, n)) + geom_bar(position="dodge", stat="identity") + theme_minimal() + labs(y="Closed per Year") +
  theme(text = element_text(size=15))

ggplot(outcome_by_year %>% filter(Outcome == "cpd" | Outcome == "referred"), aes(Year, n)) + geom_bar(position="dodge", stat="identity") + theme_minimal() + labs(y="Referred per Year") +
  theme(text = element_text(size=15))

kable(outcome_by_year %>% pivot_wider(names_from = "Outcome", values_from = "n"), caption="Table 12: Incident Outcomes by Year") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)


```

# UCPD Patrol Area

```{r}
rslt <- st_join(UCPD_area, cook_pop)
rslt <- rslt %>% as.data.frame() %>% select(-geometry) %>% left_join(cook_pop %>% select(GEOID), by=c("GEOID"))
rslt <- st_transform(st_as_sf(rslt), 4326)

kable(rslt %>% as.data.frame() %>%
  summarize(`Total Population` = sum(total_pop),
                   `Total White Population` = sum(white_pop),
                   `Total Black Population` = sum(black_pop)),
  caption="Table 13: Population in Census Tracts which intersect UCPD Patrol Area") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

weighted_est <- st_interpolate_aw(cook_pop[c('total_pop', 'white_pop', 'black_pop')], UCPD_area, extensive = TRUE)
kable(weighted_est %>% as.data.frame() %>% summarize(`Total Population` = round(sum(total_pop)),
                   `Total White Population` = round(sum(white_pop)),
                   `Total Black Population` = round(sum(black_pop))),
  caption="Table 14: Projected Population in UCPD Patrol Area") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = FALSE)

rslt <- rslt %>% mutate(share_non_white = (total_pop - white_pop) / total_pop)

pal <- colorNumeric(palette="Blues", rslt$share_non_white, na.color="Grey")

UCPD_area <- st_transform(UCPD_area, 4326)

leaflet(width="100%") %>% addTiles() %>% 
  addPolygons(data=rslt, 
              fillColor = ~pal(share_non_white),
              weight = 2, 
              opacity = 1,
              color = "grey",
              dashArray = 3,
              fillOpacity = 0.7) %>%
  addPolygons(data=UCPD_area,
              weight =5,
              opacity =1,
              fillOpacity = 0,
              color="black") %>%
addLegend(pal = pal, values = rslt$share_non_white, 
          opacity = 0.7, title = "Share Non-White Population", position = "bottomright")
```

# Github Directory
https://github.com/erhla/UCPD