λminδminhηPrecisionRecallF1 scoreDet. delay (sec) 0.01 0.01 200 8 0.454 0.697 0.550 1.553 0.05 0.01 200 8 0.447 0.697 0.545 1.691 0.05 0.01 50 8 0.438 0.699 0.539 2.249 0.01 0.01 50 8 0.421 0.711 0.529 1.736 0.01 0.01 100 8 0.398 0.737 0.517 1.293 0.05 0.01 100 8 0.388 0.711 0.502...
We will adapt the definition of the score to our situation with weights. This can be used to perform n-fold cross-validation. First, we introduce gi as an indicator function whether or not an observation i is included in the training set. This vector will be multiplied element-wise with ...
The smaller the score, the closer the probability distribution is to the true distribution, and the better the performance of the forecast. CRPS generalize the mean absolute error (MAE): while the forecasts are probabilistic, the observations are deterministic. If the forecast is deterministic, it...