README.Rmd

---
title:
output: github_document
---

```{r setup, echo = FALSE}
knitr::opts_chunk$set(
  warning = FALSE,
  echo = TRUE,
  collapse = TRUE,
  message = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-"
)
```

<!-- badges: start -->
[![R-CMD-check](https://github.com/NicChr/timeplyr/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/NicChr/timeplyr/actions/workflows/R-CMD-check.yaml)
[![CRAN status](https://www.r-pkg.org/badges/version/timeplyr)](https://CRAN.R-project.org/package=timeplyr)
<!-- badges: end -->

# timeplyr

# **Fast Tidy Tools for Date and Datetime Manipulation**

This package provides a set of functions to make working with date and datetime data much easier!

While most time-based packages are designed to work 
with clean and pre-aggregate data, timeplyr contains a set of tidy tools to 
complete, expand and summarise both raw and aggregate date/datetime data.

Significant efforts have been made to ensure that grouped calculations are 
fast and efficient thanks to the excellent functionality within the 
[collapse](https://sebkrantz.github.io/collapse/reference/collapse-package.html) package.

## Installation

You can install and load `timeplyr` using the below code.

```{r gh-installation, message = FALSE, eval = FALSE}
# CRAN version
install.packages("timeplyr")

# Development version
remotes::install_github("NicChr/timeplyr")
```

```{r package_load}
library(timeplyr)
```

# Basic examples

## Convert `ts`, `mts`, `xts`, `zoo`and `timeSeries` objects using `ts_as_tibble`

```{r}
library(tidyverse)
eu_stock <- EuStockMarkets %>%
  ts_as_tibble()
eu_stock
```

## Easily plot time series using `time_ggplot`

```{r}
eu_stock %>%
  time_ggplot(time, value, group)
```


For the next examples we use flights departing from New York City in 2013.

```{r}
library(nycflights13)
library(lubridate)
flights <- flights %>%
  mutate(date = as_date(time_hour))
```

```{r, echo=FALSE}
theme_set(theme_minimal())
```

## `time_by`

### Group your time variable by any time unit

```{r}
flights_monthly <- flights %>%
  select(date, arr_delay) %>%
  time_by(date, "month")

flights_monthly
```

We can then use this to create a monthly summary of the number of flights
and average arrival delay

```{r}
flights_monthly %>%
  summarise(n = n(),
            mean_arr_delay = mean(arr_delay, na.rm = TRUE))
```

If the time unit is left unspecified, the `time` functions try to 
find the highest time unit possible.

```{r}
flights %>%
  time_by(time_hour)
```

## `time_complete()`

### Complete missing gaps in time

```{r}
flights %>%
  count(time_hour) %>%
  time_complete(time_hour)
```

### We can also make use of timeplyr time intervals

```{r}
quarters <- time_aggregate(flights$date, time_by = "quarter", as_interval = TRUE)
interval_count(quarters)

# Or simply
flights %>%
  time_by(date, time_by = "quarter", as_interval = TRUE) %>%
  count()
```

#### Ensure full weeks by setting from to the start of the week

```{r}
start <- dmy("17-Jan-2013")
flights %>%
  time_by(date, "week", 
          from = floor_date(start, unit = "week")) %>%
  count()
```

#### Check for missing gaps in time

```{r missing_dates}
missing_dates(flights$date) # No missing dates
```

```{r}
time_num_gaps(flights$time_hour, time_by = "hours") # Missing hours
```

To check for regularity use `time_is_regular`

```{r}
hours <- sort(flights$time_hour)
time_is_regular(hours, time_by = "hours")
time_is_regular(hours, time_by = "hours", allow_gaps = FALSE)
time_is_regular(hours, time_by = "hours", allow_dups = FALSE)

# By-group
time_num_gaps(flights$time_hour, g = flights$origin, time_by = "hours")
time_is_regular(flights$time_hour, g = flights$origin, time_by = "hours")
```


## `time_expand()`

Here we create monthly sequences for each destination
that accounts for the start and end dates of each destination

```{r}
flights %>%
  group_by(dest) %>%
  time_expand(date, time_by = "month") %>%
  summarise(n = n(), start = min(date), end = max(date))
```

To create the same grid of months for each dest, we can do the following

```{r}
flights %>%
  time_expand(date, dest, time_by = "month") %>%
  summarise(n = n(), start = min(date), end = max(date), .by = dest)
```

The ability to create time sequences by group is 
one of the most powerful features of timeplyr.

```{r}
flights %>%
  time_by(date, "month", as_interval = TRUE) %>%
  summarise(across(c(arr_time, dep_time), ~ mean(.x, na.rm = TRUE)))
```

# Grouped rolling time functions

## By-group rolling mean over the last 3 calendar months

```{r}
eu_stock <- eu_stock %>%
  mutate(date = date_decimal(time))

eu_stock %>%
    mutate(month_mean = time_roll_mean(value, window = months(3), 
                                       time = date, 
                                       g = group)) %>%
    time_ggplot(date, month_mean, group)
```

## By-group rolling (locf) NA fill

```{r}
# Prerequisite: Create Time series with missing values
x <- ts(c(NA, 3, 4, NA, 6, NA, NA, 8))
g <- cheapr::seq_id(c(3, 5)) # Two groups of size 3 + 5

.roll_na_fill(x) # Simple locf fill
roll_na_fill(x, fill_limit = 1) # Fill up to 1 NA

roll_na_fill(x, g = g) # Very efficient on large data too
```

## `year_month` and `year_quarter`

timeplyr has its own lightweight 'yearmonth' and `yearquarter' classes
inspired by the excellent 'zoo' and 'tsibble' packages.


```{r}
today <- today()
year_month(today)
```

The underlying data for a `year_month` is the number of months
since 1 January 1970 (epoch).

```{r}
unclass(year_month("1970-01-01"))
unclass(year_month("1971-01-01"))
```

To create a sequence of 'year_months', one can use base arithmetic

```{r}
year_month(today) + 0:12
year_quarter(today) + 0:4
```

## `time_elapsed()`

Let's look at the time between consecutive flights for a specific flight number

```{r}
set.seed(42)
flight_201 <- flights %>%
  distinct(time_hour, flight) %>%
  filter(flight %in% sample(flight, size = 1)) %>%
  arrange(time_hour)

tail(sort(table(time_elapsed(flight_201$time_hour, "hours"))))
```

Flight 201 seems to depart mostly consistently every 24 hours

We can efficiently do the same for all flight numbers

```{r}
# We use fdistinct with sort as it's much faster and simpler to write
all_flights <- flights %>%
  fdistinct(flight, time_hour, sort = TRUE)
all_flights <- all_flights %>%
  mutate(elapsed = time_elapsed(time_hour, g = flight, fill = 0))

# Flight numbers with largest relative deviation in time between flights
all_flights %>%
  q_summarise(elapsed, .by = flight) %>%
  mutate(relative_iqr = p75 / p25) %>%
  arrange(desc(relative_iqr))
```

`time_seq_id()` allows us to create unique IDs for regular sequences
A new ID is created every time there is a gap in the sequence

```{r}
flights %>%
  select(time_hour) %>%
  arrange(time_hour) %>%
  mutate(time_id = time_seq_id(time_hour)) %>%
  filter(time_id != lag(time_id)) %>%
  count(hour(time_hour))
```

We can see that the gaps typically occur at 11pm and the sequence
resumes at 5am. 

### Other convenience functions are included below

## `calendar()`

#### Easily join common date information to your data

```{r}
flights_calendar <- flights %>%
    select(time_hour) %>%
    reframe(calendar(time_hour))
```

Now that gaps in time have been filled and we have 
joined our date table, it is easy to count by any time dimension we like

```{r}
flights_calendar %>% 
  fcount(isoyear, isoweek)
flights_calendar %>% 
  fcount(isoweek = iso_week(time))
flights_calendar %>% 
  fcount(month_l)
```

## `.time_units`

See a list of available time units

```{r}
.time_units
```

## `age_years()`

Calculate ages (years) accurately

```{r}
age_years(dmy("28-02-2000"))
```

## `time_seq()`

A lubridate version of `seq()` for dates and datetimes

```{r}
start <- dmy(31012020)
end <- start + years(1)
seq(start, end, by = "month") # Base R version
time_seq(start, end, time_by = "month") # lubridate version

```

`time_seq()` doesn't mind mixing dates and datetimes

```{r}
time_seq(start, as_datetime(end), time_by = "2 weeks")
```

## `time_seq_v()`

A vectorised version of `time_seq()`
Currently it is vectorised over from, to and by

```{r}
# 3 sequences
time_seq_v(from = start, 
           to = end, 
           time_by = list("months" = 1:3))
# Equivalent to 
c(time_seq(start, end, time_by = "month"),
  time_seq(start, end, time_by = "2 months"),
  time_seq(start, end, time_by = "3 months"))
```

## `time_seq_sizes()`
Vectorised function that calculates time sequence lengths

```{r}
seq_lengths <- time_seq_sizes(start, start + days(c(1, 10, 20)), 
                              time_by = list("days" = c(1, 5, 10)))
seq_lengths

# Use time_seq_v2() if you know the sequence lengths
seqs <- time_seq_v2(seq_lengths, start, time_by = list("days" = c(1, 5, 10)))
seqs
```

Dealing with impossible dates and datetimes is very simple

```{r}
time_seq(start, end, time_by = "month", roll_month = "postday") # roll impossible months forward
time_seq(start, end, time_by = "month", roll_month = "NA") # no roll

time_seq(start, end, time_by = dmonths(1)) # lubridate version with durations
```

## `iso_week()`

Simple function to get formatted ISO weeks.

```{r}
iso_week(today())
iso_week(today(), day = TRUE)
iso_week(today(), year = FALSE)
```

## `time_cut()`

Create pretty time axes using `time_breaks()`

```{r}
times <- flights$time_hour
dates <- flights$date

date_breaks <- time_breaks(dates, n = 12)
time_breaks <- time_breaks(times, n = 12, time_floor = TRUE)

weekly_data <- flights %>%
    time_by(time = date, time_by = "week",
            to = max(time_span(date, time_by = "week")),
            .name = "date") %>%
    count()
weekly_data %>%
  ggplot(aes(x = interval_start(date), y = n)) + 
  geom_bar(stat = "identity", fill = "#0072B2") + 
  scale_x_date(breaks = date_breaks, labels = scales::label_date_short())

flights %>%
  ggplot(aes(x = time_hour)) + 
  geom_bar(fill = "#0072B2") + 
  scale_x_datetime(breaks = time_breaks, labels = scales::label_date_short())
```