Skip to contents

Select the calibration model

Source: R/calibrate.R
select_calibration_model.Rd

Picks a model from the table of compare_calibration_models().

Usage

select_calibration_model(
  comparison,
  rule = c("one_se_logloss", "best_auc"),
  k_folds = 5
)

Arguments

comparison

Table returned by compare_calibration_models().

rule

Selection rule.

k_folds

Number of folds of the comparison (for the standard error).

Value

The name of the selected model.

Details

  • "best_auc": the model with the highest mean AUC (the rule of calibrate() in 2.x).

  • "one_se_logloss": among the models whose mean log-loss is within one standard error (logloss_sd / sqrt(k_folds)) of the lowest, the simplest (lowest complexity); models singular in most folds are left out, unless all are, which raises a warning. The log-loss rewards correct probabilities, not only their ranking, and the one-standard- error rule prefers the simpler model when the data cannot tell the candidates apart. This is the default of calibrate() since 3.0.0.