Explicit `NA`-handling 🛠️ #10

serkor1 · 2024-12-06T06:00:16Z

About

Issue #8 concerns non-transparent NA-handling in pairwise regression and classification metrics. When developing this package it was intended that NA-values were to be handled outside of the functions using wrappers; this was the main idea up until 4bd30bd (See point 5).

Explicit and transparent NA-handling is essential for any package. Posting this Issue #8 significantly improved the quality of {SLmetrics} - thank you @EmilHvitfeldt.

Changes

All pair-wise regression and classification functions now uses na.rm-arguments, to explicitly handle NA-values.
All pair-wise regression and classification functions now uses std::vector instead of NumericVector, it appears to faster (See 'On performance')

On performance

rm(list = ls()); gc()
#>          used (Mb) gc trigger (Mb) max used (Mb)
#> Ncells 542439 29.0    1186437 63.4   726777 38.9
#> Vcells 977321  7.5    8388608 64.0  1972559 15.1
Rcpp::sourceCpp(
  "src/classification_Accuracy.cpp"
)

# 1) generate classes
# 1.1) actual classes
actual <- factor(
  sample(
    x       = letters[1:3],
    size    = 2e4,
    replace = TRUE
  )
)

# 1.2) predicted classes
predicted <-  factor(
  sample(
    x       = letters[1:3],
    size    = 2e4,
    replace = TRUE
  )
)

# 2) benchmarrk
microbenchmark::microbenchmark(
  `(without NA-handlins)` = SLmetrics::accuracy(actual, predicted),
  `(With NA handling)`    = accuracy.factor(actual, predicted)
)
#> Unit: microseconds
#>                   expr    min      lq      mean median     uq      max neval
#>  (without NA-handlins) 16.051 16.1670 107.41617 16.446 16.522 5968.758   100
#>     (With NA handling) 15.461 15.5465  15.73808 15.791 15.877   16.770   100

^{Created on 2024-12-06 with reprex v2.1.1}

Note

This PR ONLY addresses the raised issue for .factor and .numeric methods. The cmatrix-method needs
significant rework.

This will be fixed soon (TM)

* The `accuracy()`-function now handles missings values according to na.rm * The `accuracy()`-function uses `std::vector` to offset the loss in speed from missing values checking * Updated tools; the modifyRcppExports is now more dynamic and uses regex instead of hardcoded functions. NOTE: All checks passed locally.

* The `zerooneloss()`-function now accepts the argument `na.rm` * Internal optimaztion of the function to make it faster

* The NEWS have been updated to reflect the new changes.

* All functions now has an argument `na.rm` to explicitly handle missing values. * Migrated to std::vector instead of NumericVector * Fixed a small bug in `accuracy.Rd`

* Unit-tests: The unit-tests now tests if misssing values are being handled according to the `na.rm`-argument * NA-handling in functions: `rsq()`-, `ccc()`-, `mae()`-, `mape()`-, `mse()`-, `pinball()`-, `rae()`-, `rmse()`-, `rrmse()`- and `smape()`-functions now all uses `std::vector` instead of `NumericVector`, and has explicit NA-handling. All tests passed locally.

* [UPDATE] `accuracy()`-function * The `accuracy()`-function now handles missings values according to na.rm * The `accuracy()`-function uses `std::vector` to offset the loss in speed from missing values checking * Updated tools; the modifyRcppExports is now more dynamic and uses regex instead of hardcoded functions. NOTE: All checks passed locally. * [UPDATE] `zerooneloss()`-function 🚀 * The `zerooneloss()`-function now accepts the argument `na.rm` * Internal optimaztion of the function to make it faster * [DOCUMENTATION] Updated NEWS 📚 * The NEWS have been updated to reflect the new changes. * [UPDATE] `RMSE()`- `RMSLE()`- and `huberloss()` 🚀 * All functions now has an argument `na.rm` to explicitly handle missing values. * Migrated to std::vector instead of NumericVector * Fixed a small bug in `accuracy.Rd` * [UPDATE] Unit-tests and NA-handlings 🔥 * Unit-tests: The unit-tests now tests if misssing values are being handled according to the `na.rm`-argument * NA-handling in functions: `rsq()`-, `ccc()`-, `mae()`-, `mape()`-, `mse()`-, `pinball()`-, `rae()`-, `rmse()`-, `rrmse()`- and `smape()`-functions now all uses `std::vector` instead of `NumericVector`, and has explicit NA-handling. All tests passed locally. * [VERSION-BUMP] 0.1-0 ---> 0.1-1 🪜 > [!NOTE] > > Checks fail on PR for OSX. But all else passes.

* Explicit `NA`-handling 🛠️ (#10) * [UPDATE] `accuracy()`-function * The `accuracy()`-function now handles missings values according to na.rm * The `accuracy()`-function uses `std::vector` to offset the loss in speed from missing values checking * Updated tools; the modifyRcppExports is now more dynamic and uses regex instead of hardcoded functions. NOTE: All checks passed locally. * [UPDATE] `zerooneloss()`-function 🚀 * The `zerooneloss()`-function now accepts the argument `na.rm` * Internal optimaztion of the function to make it faster * [DOCUMENTATION] Updated NEWS 📚 * The NEWS have been updated to reflect the new changes. * [UPDATE] `RMSE()`- `RMSLE()`- and `huberloss()` 🚀 * All functions now has an argument `na.rm` to explicitly handle missing values. * Migrated to std::vector instead of NumericVector * Fixed a small bug in `accuracy.Rd` * [UPDATE] Unit-tests and NA-handlings 🔥 * Unit-tests: The unit-tests now tests if misssing values are being handled according to the `na.rm`-argument * NA-handling in functions: `rsq()`-, `ccc()`-, `mae()`-, `mape()`-, `mse()`-, `pinball()`-, `rae()`-, `rmse()`-, `rrmse()`- and `smape()`-functions now all uses `std::vector` instead of `NumericVector`, and has explicit NA-handling. All tests passed locally. * [VERSION-BUMP] 0.1-0 ---> 0.1-1 🪜 > [!NOTE] > > Checks fail on PR for OSX. But all else passes. * [BUG-FIX] `plot()`-method for `ROC()` and `prROC()` 🔨 (#11) * The plots are now correctly displaying lines when `panels = FALSE` * NEWS.md updated accordingly.

* [UPDATE] `accuracy()`-function * The `accuracy()`-function now handles missings values according to na.rm * The `accuracy()`-function uses `std::vector` to offset the loss in speed from missing values checking * Updated tools; the modifyRcppExports is now more dynamic and uses regex instead of hardcoded functions. NOTE: All checks passed locally. * [UPDATE] `zerooneloss()`-function 🚀 * The `zerooneloss()`-function now accepts the argument `na.rm` * Internal optimaztion of the function to make it faster * [DOCUMENTATION] Updated NEWS 📚 * The NEWS have been updated to reflect the new changes. * [UPDATE] `RMSE()`- `RMSLE()`- and `huberloss()` 🚀 * All functions now has an argument `na.rm` to explicitly handle missing values. * Migrated to std::vector instead of NumericVector * Fixed a small bug in `accuracy.Rd` * [UPDATE] Unit-tests and NA-handlings 🔥 * Unit-tests: The unit-tests now tests if misssing values are being handled according to the `na.rm`-argument * NA-handling in functions: `rsq()`-, `ccc()`-, `mae()`-, `mape()`-, `mse()`-, `pinball()`-, `rae()`-, `rmse()`-, `rrmse()`- and `smape()`-functions now all uses `std::vector` instead of `NumericVector`, and has explicit NA-handling. All tests passed locally. * [VERSION-BUMP] 0.1-0 ---> 0.1-1 🪜 > [!NOTE] > > Checks fail on PR for OSX. But all else passes.

serkor1 added 5 commits December 4, 2024 20:11

[UPDATE] zerooneloss()-function 🚀

de466f4

* The `zerooneloss()`-function now accepts the argument `na.rm` * Internal optimaztion of the function to make it faster

[DOCUMENTATION] Updated NEWS 📚

7aba10a

* The NEWS have been updated to reflect the new changes.

[UPDATE] RMSE()- RMSLE()- and huberloss() 🚀

8d24c8a

* All functions now has an argument `na.rm` to explicitly handle missing values. * Migrated to std::vector instead of NumericVector * Fixed a small bug in `accuracy.Rd`

serkor1 added documentation Improvements or additions to documentation enhancement New feature or request labels Dec 6, 2024

[VERSION-BUMP] 0.1-0 ---> 0.1-1 🪜

eadd237

serkor1 changed the title ~~Expllicit NA-handling 🛠️~~ Explicit NA-handling 🛠️ Dec 6, 2024

serkor1 merged commit 6ff6107 into development Dec 6, 2024
6 of 7 checks passed

serkor1 deleted the patch-1 branch December 6, 2024 06:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Explicit `NA`-handling 🛠️ #10

Explicit `NA`-handling 🛠️ #10

serkor1 commented Dec 6, 2024

Explicit NA-handling 🛠️ #10

Explicit NA-handling 🛠️ #10

Conversation

serkor1 commented Dec 6, 2024

About

Changes

On performance

Explicit `NA`-handling 🛠️ #10

Explicit `NA`-handling 🛠️ #10