README.Rmd

---
title: "rmetrics"
author: "Jonas Schropp"
date: "`r Sys.Date()`"
output: md_document
---

# Introduction

`rmetrics` is a R package to calculate a large variety of metrics to evaluate 
machine learning models and statistical models. It is primarily designed as a 
backend to my package `revaluate`, but the large number of provided metrics makes 
it useful as a standalone package. \newline 

The package provides classification metrics for *binary*, *multiclass* or 
*ordered* dependent variables, with the special case of binary dependent 
variables that are evaluated using upper and lower cutoffs and a grey area 
in between. \newline

All functions in the package provide several methods based on the class of the 
first input variable. In most cases, the functions work with raw counts from the 
confusion matrix (TP, FP, TN, FN etc), a `table` object or a `data.frame`. \newline 

The package is **under active development**! Function names or arguments might 
change unannounced, methods might be added or even removed. Do not use it in a 
production environment (yet)!

# Installation

`rmetrics` is not yet on CRAN. You can install the package via github:

```{r, eval = FALSE}
devtools::install_github("jonas-schropp/rmetrics")
```

```{r}
library(rmetrics)
```

# Usage

Let's calculate Aickin's alpha as an example:
```{r}
# Simulate some data
pred <- c(rep(1, 50), rep(2, 50), rep(3, 50))
ref <- c(rep(1, 25), rep(2, 5), rep(3, 20),
               rep(2, 40), rep(1, 5), rep(3, 5),
               rep(3, 30), rep(1, 10), rep(2, 10))

tbl <- table(pred, ref)

calc_aickin(tbl = tbl)
```
We could also use a data.frame object as input and set the CI to 99%:
```{r}
calc_aickin(
  data.frame(pred, ref),
  prediction = "pred",
  reference = "ref",
  ci.level = 0.99
  )
```

Or not request a CI at all:
```{r}
calc_aickin(tbl, ci.type = FALSE)
```

Most functions also support raw counts for the entries of the confusion matrix 
as inputs:
```{r}
calc_tnr(tn = 275, fp = 8, 
         ci.type = "clopper-pearson", 
         ci.level = 0.95)
```


# Available metrics

For a full list of functions, see the reference manual. 

# Contribute to this project

If you want to contribute to this project, please get in touch! \newline

If there is a metric you want to see in this package that is not available, 
please either submit a `feature request` or implement it yourself and submit a 
`pull request`.

# Planned features

This package is under active development. The current focus for development is 
1) to implement more metrics for ordinal classifiers, 2) to add confidence 
interval calculations for more of the metrics and 3) to add much more 
documentation and references.