Usage
compute_metrics(hist_dir = "history", metrics, y_test, ind_quiz, read_fun)
Arguments
- hist_dir
- string. directory where the history of the submissions are stored.
contains one subdirectory per team.
- metrics
- named list of functions. Each function in the list computes
a performance criterion and is defined as:
function(y_pred, y_test)
- y_test
- character or numeric vector. the test set output.
- ind_quiz
- logical vector with the same length as
y_test
. ind_quiz[i]=TRUE
if y_test[i]
in the quiz subset.
- read_fun
- function that reads a submission file and returns a vector of predictions.
Value
compute_metrics
returns a named list with one named member per team.
Each member is a data.frame
where the rows are the submission files sorted by date
and the columns are:
- date
- the date of the submission
- file
- the file name of the submission
- .quiz
- the score obtained on the quiz subset
- .test
- the score obtained on the test set
Description
Compute metrics of the submissions in the history.