Evaluation Settings

 

 

Evaluation procedure

An evaluation procedure implements a certain metric in order to assess participants' solutions. Generally, it compares user's answers with the target ones.

The choice of a proper metric depends on different factors, e.g. what kind of problem it is (classification or regression). You can choose one of the standard evaluation procedures or implement your own.

Timeout

You can set a timeout on the evaluation of a single solution. It is especially useful when the solutions are in the form of algorithms, not already predicted values. You can choose distinct timeouts for the preliminary and final tests.