README.Rmd

---
output: github_document
---

# `datawizard`: Easy Data Wrangling and Statistical Transformations <img src='man/figures/logo.png' align="right" height="139" />

```{r, echo=FALSE, warning=FALSE, message=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  dpi = 300,
  out.width = "100%",
  fig.path = "man/figures/",
  comment = "#>"
)

set.seed(333)
library(datawizard)
```

[![DOI](https://joss.theoj.org/papers/10.21105/joss.04684/status.svg)](https://doi.org/10.21105/joss.04684)
[![downloads](http://cranlogs.r-pkg.org/badges/datawizard)](https://cran.r-project.org/package=datawizard)
[![total](https://cranlogs.r-pkg.org/badges/grand-total/datawizard)](https://cranlogs.r-pkg.org/) 

<!-- ***:sparkles: Hockety pockety wockety wack, prepare this data forth and back*** -->

<!-- ***Hockety pockety wockety wock, messy data is in shock*** -->

<!-- ***Hockety pockety wockety woss, you can cite i-it from JOSS*** <sup>(soon)</sup> -->

<!-- ***Hockety pockety wockety wass, datawizard saves your ass! :sparkles:*** -->

`{datawizard}` is a lightweight package to easily manipulate, clean, transform, and prepare your data for analysis. It is part of the [easystats ecosystem](https://easystats.github.io/easystats/), a suite of R packages to deal with your entire statistical analysis, from cleaning the data to reporting the results. 

It covers two aspects of data preparation: 

- **Data manipulation**: `{datawizard}` offers a very similar set of functions to that of the *tidyverse* packages, such as a `{dplyr}` and `{tidyr}`, to select, filter and reshape data, with a few key differences. 1) All data manipulation functions start with the prefix `data_*` (which makes them easy to identify). 2) Although most functions can be used exactly as their *tidyverse* equivalents, they are also string-friendly (which makes them easy to program with and use inside functions). Finally, `{datawizard}` is super lightweight (no dependencies, similar to [poorman](https://github.com/nathaneastwood/poorman)), which makes it awesome for developers to use in their packages.

- **Statistical transformations**: `{datawizard}` also has powerful functions to easily apply common data [transformations](https://easystats.github.io/datawizard/reference/index.html#statistical-transformations), including standardization, normalization, rescaling, rank-transformation, scale reversing, recoding, binning, etc. 



</br>

<img src='https://media.giphy.com/media/VcizxCUIgaKpa/giphy.gif' width="300"/>

</br>

# Installation

[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/datawizard)](https://cran.r-project.org/package=datawizard) [![insight status badge](https://easystats.r-universe.dev/badges/datawizard)](https://easystats.r-universe.dev) [![R-CMD-check](https://github.com/easystats/datawizard/workflows/R-CMD-check/badge.svg?branch=main)](https://github.com/easystats/datawizard/actions)

Type | Source | Command
---|---|---
Release | CRAN | `install.packages("datawizard")`
Development | r-universe | `install.packages("datawizard", repos = "https://easystats.r-universe.dev")`
Development | GitHub | `remotes::install_github("easystats/datawizard")`

> **Tip**
>
> **Instead of `library(datawizard)`, use `library(easystats)`.**
> **This will make all features of the  easystats-ecosystem available.**
>
> **To stay updated, use `easystats::install_latest()`.**

# Citation

To cite the package, run the following command:

```{r, comment=""}
citation("datawizard")
```

# Features

Most courses and tutorials about statistical modeling assume that you are working with a clean and tidy dataset. In practice, however, a major part of doing statistical modeling is preparing your data--cleaning up values, creating new columns, reshaping the dataset, or transforming some variables. `{datawizard}` provides easy to use tools to perform these common, critical, and sometimes tedious data preparation tasks. 

## Data wrangling 

### Select, filter and remove variables

The package provides helpers to filter rows meeting certain conditions...

```{r}
data_match(mtcars, data.frame(vs = 0, am = 1))
```

... or logical expressions:

```{r}
data_filter(mtcars, vs == 0 & am == 1)
```

Finding columns in a data frame, or retrieving the data of selected columns, can be  achieved using `extract_column_names()` or `data_select()`:

```{r}
# find column names matching a pattern
extract_column_names(iris, starts_with("Sepal"))

# return data columns matching a pattern
data_select(iris, starts_with("Sepal")) |> head()
```

It is also possible to extract one or more variables:

```{r}
# single variable
data_extract(mtcars, "gear")

# more variables
head(data_extract(iris, ends_with("Width")))
```

Due to the consistent API, removing variables is just as simple:

```{r}
head(data_remove(iris, starts_with("Sepal")))
```

### Reorder or rename

```{r}
head(data_relocate(iris, select = "Species", before = "Sepal.Length"))
```

```{r}
head(data_rename(iris, c("Sepal.Length", "Sepal.Width"), c("length", "width")))
```

### Merge

```{r}
x <- data.frame(a = 1:3, b = c("a", "b", "c"), c = 5:7, id = 1:3)
y <- data.frame(c = 6:8, d = c("f", "g", "h"), e = 100:102, id = 2:4)

x
y

data_merge(x, y, join = "full")

data_merge(x, y, join = "left")

data_merge(x, y, join = "right")

data_merge(x, y, join = "semi", by = "c")

data_merge(x, y, join = "anti", by = "c")

data_merge(x, y, join = "inner")

data_merge(x, y, join = "bind")
```

### Reshape

A common data wrangling task is to reshape data.

Either to go from wide/Cartesian to long/tidy format

```{r}
wide_data <- data.frame(replicate(5, rnorm(10)))

head(data_to_long(wide_data))
```

or the other way

```{r}
long_data <- data_to_long(wide_data, rows_to = "Row_ID") # Save row number

data_to_wide(long_data,
  names_from = "name",
  values_from = "value",
  id_cols = "Row_ID"
)
```

### Empty rows and columns

```{r}
tmp <- data.frame(
  a = c(1, 2, 3, NA, 5),
  b = c(1, NA, 3, NA, 5),
  c = c(NA, NA, NA, NA, NA),
  d = c(1, NA, 3, NA, 5)
)

tmp

# indices of empty columns or rows
empty_columns(tmp)
empty_rows(tmp)

# remove empty columns or rows
remove_empty_columns(tmp)
remove_empty_rows(tmp)

# remove empty columns and rows
remove_empty(tmp)
```

### Recode or cut dataframe

```{r}
set.seed(123)
x <- sample(1:10, size = 50, replace = TRUE)

table(x)

# cut into 3 groups, based on distribution (quantiles)
table(categorize(x, split = "quantile", n_groups = 3))
```

## Data Transformations

The packages also contains multiple functions to help transform data.

### Standardize

For example, to standardize (*z*-score) data:

```{r}
# before
summary(swiss)

# after
summary(standardize(swiss))
```

### Winsorize

To winsorize data:

```{r}
# before
anscombe

# after
winsorize(anscombe)
```

### Center

To grand-mean center data

```{r}
center(anscombe)
```

### Ranktransform

To rank-transform data:

```{r}
# before
head(trees)

# after
head(ranktransform(trees))
```

### Rescale

To rescale a numeric variable to a new range:

```{r}
change_scale(c(0, 1, 5, -5, -2))
```

### Rotate or transpose

```{r}
x <- mtcars[1:3, 1:4]

x

data_rotate(x)
```


## Data properties

`datawizard` provides a way to provide comprehensive descriptive summary for all variables in a dataframe:

```{r}
data(iris)
describe_distribution(iris)
```

Or even just a variable

```{r}
describe_distribution(mtcars$wt)
```

There are also some additional data properties that can be computed using this package.

```{r}
x <- (-10:10)^3 + rnorm(21, 0, 100)
smoothness(x, method = "diff")
```

## Function design and pipe-workflow

The design of the `{datawizard}` functions follows a design principle that makes it easy for user to understand and remember how functions work:

1. the first argument is the data
2. for methods that work on data frames, two arguments are following to `select` and `exclude` variables
3. the following arguments are arguments related to the specific tasks of the functions

Most important, functions that accept data frames usually have this as their first argument, and also return a (modified) data frame again. Thus, `{datawizard}` integrates smoothly into a "pipe-workflow".

```{r}
iris |>
  # all rows where Species is "versicolor" or "virginica"
  data_filter(Species %in% c("versicolor", "virginica")) |>
  # select only columns with "." in names (i.e. drop Species)
  data_select(contains("\\.")) |>
  # move columns that ends with "Length" to start of data frame
  data_relocate(ends_with("Length")) |>
  # remove fourth column
  data_remove(4) |>
  head()
```
  
# Contributing and Support

In case you want to file an issue or contribute in another way to the package, please follow [this guide](https://easystats.github.io/datawizard/CONTRIBUTING.html). For questions about the functionality, you may either contact us via email or also file an issue.

# Code of Conduct

Please note that this project is released with a 
[Contributor Code of Conduct](https://easystats.github.io/datawizard/CODE_OF_CONDUCT.html). By participating in this project you agree to abide by its terms.