Skip to content

Commit

Permalink
{SLmetrics} Version 0.1-1 💪
Browse files Browse the repository at this point in the history
* Explicit `NA`-handling 🛠️  (#10)

* [UPDATE] `accuracy()`-function

* The `accuracy()`-function now handles missings values according to na.rm
* The `accuracy()`-function uses `std::vector` to offset the loss in speed from missing values checking
* Updated tools; the modifyRcppExports is now more dynamic and uses regex instead of hardcoded functions.

NOTE: All checks passed locally.

* [UPDATE] `zerooneloss()`-function 🚀

* The `zerooneloss()`-function now accepts the argument `na.rm`
* Internal optimaztion of the function to make it faster

* [DOCUMENTATION] Updated NEWS 📚

* The NEWS have been updated to reflect the new changes.

* [UPDATE] `RMSE()`- `RMSLE()`- and `huberloss()` 🚀

* All functions now has an argument `na.rm` to explicitly handle missing values.
* Migrated to std::vector instead of NumericVector
* Fixed a small bug in `accuracy.Rd`

* [UPDATE] Unit-tests and NA-handlings 🔥

* Unit-tests: The unit-tests now tests if misssing values are being handled according to the `na.rm`-argument
* NA-handling in functions: `rsq()`-, `ccc()`-, `mae()`-, `mape()`-, `mse()`-, `pinball()`-, `rae()`-, `rmse()`-, `rrmse()`- and `smape()`-functions now all uses `std::vector` instead of `NumericVector`, and has explicit NA-handling.

All tests passed locally.

* [VERSION-BUMP] 0.1-0 ---> 0.1-1 🪜

> [!NOTE]
>
> Checks fail on PR for OSX. But all else passes.

* [BUG-FIX] `plot()`-method for `ROC()` and `prROC()` 🔨 (#11)

* The plots are now correctly displaying lines when `panels = FALSE`
* NEWS.md updated accordingly.
  • Loading branch information
serkor1 authored Dec 8, 2024
1 parent fc8c7cf commit 29a2b5d
Show file tree
Hide file tree
Showing 75 changed files with 1,570 additions and 1,240 deletions.
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ po/*~
rsconnect/
.Rproj.user

# playground
playground/
# sandbox
sandbox/
docs
inst/doc

Expand Down
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: SLmetrics
Title: Machine Learning Performance Evaluation on Steroids
Version: 0.1-0
Version: 0.1-1
Authors@R: c(
person(
given = "Serkan",
Expand Down
80 changes: 78 additions & 2 deletions NEWS.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,90 @@ knitr::opts_chunk$set(
set.seed(1903)
```

# Version 0.1-0
# Version 0.1-1

> Version 0.1-0 is considered pre-release of {SLmetrics}. We do not
> Version 0.1-1 is considered pre-release of {SLmetrics}. We do not
> expect any breaking changes, unless a major bug/issue is reported and its nature
> forces breaking changes.
## General

* **Backend changes:** All pair-wise metrics arer moved from {Rcpp} to C++, this have reduced execution time by half. All pair-wise metrics are now faster.

## Improvements

* **NA-controls:** All pair-wise metrics that doesn't have a `micro`-argument were handling missing values as according to C++ and {Rcpp} internals. See [Issue](https://github.com/serkor1/SLmetrics/issues/8). Thank you @EmilHvitfeldt for pointing this out. This has now been fixed so functions uses an `na.rm`-argument to explicitly control for this. See below,

```{r}
# 1) define factors
actual <- factor(c("no", "yes"))
predicted <- factor(c(NA, "no"))
# 2) accuracy with na.rm = TRUE
SLmetrics::accuracy(
actual = actual,
predicted = predicted,
na.rm = TRUE
)
# 2) accuracy with na.rm = FALSE
SLmetrics::accuracy(
actual = actual,
predicted = predicted,
na.rm = FALSE
)
```

## Bug-fixes

* The `plot.prROC()`- and `plot.ROC()`-functions now adds a line to the plot when `panels = FALSE`. See Issue https://github.com/serkor1/SLmetrics/issues/9.

```{r}
# 1) define actual
# classes
actual <- factor(
sample(letters[1:2], size = 100, replace = TRUE)
)
# 2) define response
# probabilities
response <- runif(100)
# 3) calculate
# ROC and prROC
# 3.1) ROC
roc <- SLmetrics::ROC(
actual,
response
)
# 3.2) prROC
prroc <- SLmetrics::prROC(
actual,
response
)
# 4) plot with panels
# FALSE
par(mfrow = c(1,2))
plot(
roc,
panels = FALSE
)
plot(
prroc,
panels = FALSE
)
```



# Version 0.1-0

## General

* {SLmetrics} is a collection of Machine Learning performance evaluation functions for supervised learning. Visit the online documentation on [GitHub Pages](https://serkor1.github.io/SLmetrics/).

## Examples
Expand Down
126 changes: 113 additions & 13 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,110 @@

# Version 0.1-0
# Version 0.1-1

> Version 0.1-0 is considered pre-release of {SLmetrics}. We do not
> Version 0.1-1 is considered pre-release of {SLmetrics}. We do not
> expect any breaking changes, unless a major bug/issue is reported and
> its nature forces breaking changes.
## General

- **Backend changes:** All pair-wise metrics arer moved from {Rcpp} to
C++, this have reduced execution time by half. All pair-wise metrics
are now faster.

## Improvements

- **NA-controls:** All pair-wise metrics that doesn’t have a
`micro`-argument were handling missing values as according to C++
and {Rcpp} internals. See
[Issue](https://github.com/serkor1/SLmetrics/issues/8). Thank you
@EmilHvitfeldt for pointing this out. This has now been fixed so
functions uses an `na.rm`-argument to explicitly control for this.
See below,

<!-- end list -->

``` r
# 1) define factors
actual <- factor(c("no", "yes"))
predicted <- factor(c(NA, "no"))

# 2) accuracy with na.rm = TRUE
SLmetrics::accuracy(
actual = actual,
predicted = predicted,
na.rm = TRUE
)
```

#> [1] 0

``` r
# 2) accuracy with na.rm = FALSE
SLmetrics::accuracy(
actual = actual,
predicted = predicted,
na.rm = FALSE
)
```

#> [1] NaN

## Bug-fixes

- The `plot.prROC()`- and `plot.ROC()`-functions now adds a line to
the plot when `panels = FALSE`. See Issue
<https://github.com/serkor1/SLmetrics/issues/9>.

<!-- end list -->

``` r
# 1) define actual
# classes
actual <- factor(
sample(letters[1:2], size = 100, replace = TRUE)
)

# 2) define response
# probabilities
response <- runif(100)

# 3) calculate
# ROC and prROC

# 3.1) ROC
roc <- SLmetrics::ROC(
actual,
response
)

# 3.2) prROC
prroc <- SLmetrics::prROC(
actual,
response
)

# 4) plot with panels
# FALSE
par(mfrow = c(1,2))
plot(
roc,
panels = FALSE
)
```

![](NEWS_files/figure-gfm/unnamed-chunk-2-1.png)<!-- -->

``` r
plot(
prroc,
panels = FALSE
)
```

![](NEWS_files/figure-gfm/unnamed-chunk-2-2.png)<!-- -->

# Version 0.1-0

## General

- {SLmetrics} is a collection of Machine Learning performance
Expand All @@ -25,7 +125,7 @@ print(
)
```

#> [1] b a b b a c b c c a
#> [1] a a a b c a b c c a
#> Levels: a b c

``` r
Expand All @@ -37,7 +137,7 @@ print(
)
```

#> [1] c c a b a b c c a c
#> [1] c c b b a a b b c b
#> Levels: a b c

``` r
Expand All @@ -55,16 +155,16 @@ summary(
#> Confusion Matrix (3 x 3)
#> ================================================================================
#> a b c
#> a 1 0 2
#> b 1 1 2
#> a 1 2 2
#> b 0 2 0
#> c 1 1 1
#> ================================================================================
#> Overall Statistics (micro average)
#> - Accuracy: 0.30
#> - Balanced Accuracy: 0.31
#> - Sensitivity: 0.30
#> - Specificity: 0.65
#> - Precision: 0.30
#> - Accuracy: 0.40
#> - Balanced Accuracy: 0.51
#> - Sensitivity: 0.40
#> - Specificity: 0.70
#> - Precision: 0.40

``` r
# 2) calculate false positive
Expand All @@ -75,7 +175,7 @@ SLmetrics::fpr(
```

#> a b c
#> 0.2857143 0.1666667 0.5714286
#> 0.2000000 0.3750000 0.2857143

### Supervised regression metrics

Expand All @@ -96,4 +196,4 @@ SLmetrics::huberloss(
)
```

#> [1] 0.394088
#> [1] 0.4757045
Binary file added NEWS_files/figure-gfm/unnamed-chunk-2-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added NEWS_files/figure-gfm/unnamed-chunk-2-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 29a2b5d

Please sign in to comment.