TunedIT Research - Frequently Asked Questions

 

 

What is TunedIT Research?

TunedIT Research is an integrated platform for sharing, evaluation and comparison of machine learning (ML) and data mining (DM) algorithms. Its aim is to help researchers and users evaluate learning methods in reproducible experiments and to enable valid comparison of different algorithms. TunedIT serves also as a place where researchers may share their implementations and datasets with others.

Why do scientists need such a platform?

See the motivation.

If I use my private resource in a test and submit result to Knowledge Base (KB), will other users see the result on KB page?

No. To see the result the user must have access rights to all the resources used in a given test: algorithm, dataset and evaluation procedure.

What if an error occurs during test and "Send results to Knowledge Base" option is checked? Is the error sent to KB? Is it included in results shown in KB page?

Errors caused by the tested algorithm are sent to KB. Other errors: caused by evaluation procedure or testing environment (like problems with network connection) are not. Currently, errors submitted to KB are not included in the results shown at KB page.

What programing language should I use to implement new algorithms and evaluation procedures?

Java.

What API my algorithm should implement to be suitable for TunedTester?

It depends on evaluation procedures that will be used for this algorithm. The preferred way is to use API of Debellor and implement the algorithm as a subclass of org.debellor.core.Cell. This should be understood by most of evaluation procedures, including ClassificationTT70 and RegressionTT70. For regular classification/regression algorithms you can also use Weka or Rseslib API and implement the algorithm as a subclass of weka.classifiers.Classifier or rseslib.processing.classification.Classifier.

In what data format should I save my dataset so that TunedTester can use it?

It depends on evaluation procedures that will be used. Currently, ARFF is the preferred format supported by most of evaluation procedures, including ClassificationTT70 and RegressionTT70.

I receive OutOfMemory errors when running TunedTester.

This may occur if data used in tests are too large to fit in memory. Try to increase the amount of memory available to TunedTester: edit tunedtester.bat (on Windows) or tunedtester.sh file (on Linux) and change the parameter value -Xmx256m to something bigger, like -Xmx1024m - this will increase memory size from 256 MB to 1 GB. Due to restrictions of JVM, on 32-bit systems this value cannot exceed 1.5 GB.

See also the discussion forum to view and post questions and answers.