full rerender

learn/develop/metrics/index.html.md

@@ -19,7 +19,6 @@ include-after-body: ../../../resources.html
 
 
 
-
 ## Introduction
 
 To use code in this article,  you will need to install the following packages: rlang and tidymodels.
@@ -63,7 +62,6 @@ You are required to supply a `case_weights` argument to `mse_vec()` for the func
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -94,13 +92,11 @@ mse_vec <- function(truth, estimate, na_rm = TRUE, case_weights = NULL, ...) {
 
 
 
-
 At this point, you've created the vector version of the mean squared error metric.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -117,36 +113,33 @@ mse_vec(
 
 
 
-
 Intelligent error handling is immediately available.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
 mse_vec(truth = "apple", estimate = 1)
 #> Error in `mse_vec()`:
-#> ! `truth` should be a numeric, not a string.
+#> ! `truth` should be a numeric vector, not a string.
 
 mse_vec(truth = 1, estimate = factor("xyz"))
 #> Error in `mse_vec()`:
-#> ! `estimate` should be a numeric, not a <factor> object.
+#> ! `estimate` should be a numeric vector, not a <factor>
+#>   object.
 ```
 :::
 
 
 
 
-
 `NA` values are removed if `na_rm = TRUE` (the default). If `na_rm = FALSE` and any `NA` values are detected, then the metric automatically returns `NA`.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -163,15 +156,13 @@ mse_vec(truth = c(NA, .5, .4), estimate = c(1, .6, .5), na_rm = FALSE)
 
 
 
-
 ### Data frame implementation
 
 The data frame version of the metric should be fairly simple. It is a generic function with a `data.frame` method that calls the yardstick helper, `numeric_metric_summarizer()`, and passes along the `mse_vec()` function to it along with versions of `truth` and `estimate` that have been wrapped in `rlang::enquo()` and then unquoted with `!!` so that non-standard evaluation can be supported.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -201,13 +192,11 @@ mse.data.frame <- function(data, truth, estimate, na_rm = TRUE, case_weights = N
 
 
 
-
 And that's it. The yardstick package handles the rest.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -221,13 +210,11 @@ mse(solubility_test, truth = solubility, estimate = factor("xyz"))
 
 
 
-
 Let's test it out on a grouped data frame.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -269,7 +256,6 @@ solubility_resampled %>%
 
 
 
-
 ## Class example: miss rate
 
 Miss rate is another name for the false negative rate, and is a classification metric in the same family as `sens()` and `spec()`. It follows the formula:
@@ -287,7 +273,6 @@ The vector implementation for classification metrics initially has a very simila
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -341,13 +326,11 @@ miss_rate_vec <- function(truth,
 
 
 
-
 Another change from the numeric metric is that a call to `finalize_estimator()` is made. This is the infrastructure that auto-selects the type of estimator to use.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -360,13 +343,11 @@ miss_rate_vec(two_class_example$truth, two_class_example$predicted)
 
 
 
-
 What happens if you try and pass in a multiclass result?
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -381,7 +362,6 @@ miss_rate_vec(fold1$obs, fold1$pred)
 
 
 
-
 This isn't great, as currently multiclass `miss_rate()` isn't supported and it would have been better to throw an error if the `estimator` was not `"binary"`. Currently, `finalize_estimator()` uses its default implementation which selected `"macro"` as the `estimator` since `truth` was a factor with more than 2 classes. When we implement multiclass averaging, this is what you want, but if your metric only works with a binary implementation (or has other specialized multiclass versions), you might want to guard against this.
 
 To fix this, a generic counterpart to `finalize_estimator()`, called `finalize_estimator_internal()`, exists that helps you restrict the input types. If you provide a method to `finalize_estimator_internal()` where the method name is the same as your metric name, and then set the `metric_class` argument in `finalize_estimator()` to be the same thing, you can control how the auto-selection of the `estimator` is handled.
@@ -393,7 +373,6 @@ It is also good practice to call `validate_estimator()` which handles the case w
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -447,7 +426,6 @@ miss_rate_vec <- function(truth,
 
 
 
-
 ### Supporting multiclass miss rate
 
 Like many other classification metrics such as `precision()` or `recall()`, miss rate does not have a natural multiclass extension, but one can be created using methods such as macro, weighted macro, and micro averaging. If you have not, I encourage you to read `vignette("multiclass", "yardstick")` for more information about how these methods work.
@@ -459,7 +437,6 @@ Let's first remove the "binary" restriction we created earlier.
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -470,7 +447,6 @@ rm(finalize_estimator_internal.miss_rate)
 
 
 
-
 The main changes below are:
 
 -   The binary implementation is moved to `miss_rate_binary()`.
@@ -482,7 +458,6 @@ The main changes below are:
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -567,15 +542,13 @@ miss_rate_multiclass <- function(data, estimator) {
 
 
 
-
 For the macro case, this separation of weighting from the core implementation might seem strange, but there is good reason for it. Some metrics are combinations of other metrics, and it is nice to be able to reuse code when calculating more complex metrics. For example, `f_meas()` is a combination of `recall()` and `precision()`. When calculating a macro averaged `f_meas()`, the weighting must be applied 1 time, at the very end of the calculation. `recall_multiclass()` and `precision_multiclass()` are defined similarly to how `miss_rate_multiclass()` is defined and returns the unweighted vector of calculations. This means we can directly use this in `f_meas()`, and then weight everything once at the end of that calculation.
 
 Let's try it out now:
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -592,15 +565,13 @@ miss_rate_vec(fold1$obs, fold1$pred)
 
 
 
-
 #### Data frame implementation
 
 Luckily, the data frame implementation is as simple as the numeric case, we just need to add an extra `estimator` argument and pass that through.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -657,15 +628,13 @@ miss_rate(hpc_cv, obs, VF)
 
 
 
-
 ## Using custom metrics
 
 The `metric_set()` function validates that all metric functions are of the same metric type by checking the class of the function. If any metrics are not of the right class, `metric_set()` fails. By using `new_numeric_metric()` and `new_class_metric()` in the above custom metrics, they work out of the box without any additional adjustments.
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```{.r .cell-code}
@@ -683,48 +652,49 @@ numeric_mets(solubility_test, solubility, prediction)
 
 
 
-
 ## Session information {#session-info}
 
 
 
 
-
 ::: {.cell layout-align="center"}
 
 ```
 #> ─ Session info ─────────────────────────────────────────────────────
 #>  setting  value
-#>  version  R version 4.4.0 (2024-04-24)
-#>  os       macOS Sonoma 14.5
+#>  version  R version 4.4.2 (2024-10-31)
+#>  os       macOS Sequoia 15.3.1
 #>  system   aarch64, darwin20
 #>  ui       X11
 #>  language (EN)
 #>  collate  en_US.UTF-8
 #>  ctype    en_US.UTF-8
 #>  tz       America/Los_Angeles
-#>  date     2024-09-03
-#>  pandoc   2.17.1.1 @ /opt/homebrew/bin/ (via rmarkdown)
+#>  date     2025-03-07
+#>  pandoc   3.6.1 @ /usr/local/bin/ (via rmarkdown)
+#>  quarto   1.6.42 @ /Applications/quarto/bin/quarto
 #> 
 #> ─ Packages ─────────────────────────────────────────────────────────
-#>  package    * version    date (UTC) lib source
-#>  broom      * 1.0.6      2024-05-17 [1] CRAN (R 4.4.0)
-#>  dials      * 1.3.0      2024-07-30 [1] CRAN (R 4.4.0)
-#>  dplyr      * 1.1.4      2023-11-17 [1] CRAN (R 4.4.0)
-#>  ggplot2    * 3.5.1      2024-04-23 [1] CRAN (R 4.4.0)
-#>  infer      * 1.0.7      2024-03-25 [1] CRAN (R 4.4.0)
-#>  parsnip    * 1.2.1.9002 2024-08-30 [1] local
-#>  purrr      * 1.0.2      2023-08-10 [1] CRAN (R 4.4.0)
-#>  recipes    * 1.1.0.9000 2024-07-19 [1] local
-#>  rlang      * 1.1.4      2024-06-04 [1] CRAN (R 4.4.0)
-#>  rsample    * 1.2.1      2024-03-25 [1] CRAN (R 4.4.0)
-#>  tibble     * 3.2.1      2023-03-20 [1] CRAN (R 4.4.0)
-#>  tidymodels * 1.2.0      2024-03-25 [1] CRAN (R 4.4.0)
-#>  tune       * 1.2.1      2024-04-18 [1] CRAN (R 4.4.0)
-#>  workflows  * 1.1.4      2024-02-19 [1] CRAN (R 4.4.0)
-#>  yardstick  * 1.3.1      2024-03-21 [1] CRAN (R 4.4.0)
+#>  package    * version date (UTC) lib source
+#>  broom      * 1.0.7   2024-09-26 [1] CRAN (R 4.4.1)
+#>  dials      * 1.4.0   2025-02-13 [1] CRAN (R 4.4.2)
+#>  dplyr      * 1.1.4   2023-11-17 [1] CRAN (R 4.4.0)
+#>  ggplot2    * 3.5.1   2024-04-23 [1] CRAN (R 4.4.0)
+#>  infer      * 1.0.7   2024-03-25 [1] CRAN (R 4.4.0)
+#>  parsnip    * 1.3.0   2025-02-14 [1] CRAN (R 4.4.2)
+#>  purrr      * 1.0.4   2025-02-05 [1] CRAN (R 4.4.1)
+#>  recipes    * 1.1.1   2025-02-12 [1] CRAN (R 4.4.1)
+#>  rlang      * 1.1.5   2025-01-17 [1] CRAN (R 4.4.2)
+#>  rsample    * 1.2.1   2024-03-25 [1] CRAN (R 4.4.0)
+#>  tibble     * 3.2.1   2023-03-20 [1] CRAN (R 4.4.0)
+#>  tidymodels * 1.3.0   2025-02-21 [1] CRAN (R 4.4.1)
+#>  tune       * 1.3.0   2025-02-21 [1] CRAN (R 4.4.1)
+#>  workflows  * 1.2.0   2025-02-19 [1] CRAN (R 4.4.1)
+#>  yardstick  * 1.3.2   2025-01-22 [1] CRAN (R 4.4.1)
 #> 
-#>  [1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
+#>  [1] /Users/emilhvitfeldt/Library/R/arm64/4.4/library
+#>  [2] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
+#>  * ── Packages attached to the search path.
 #> 
 #> ────────────────────────────────────────────────────────────────────
 ```