Launching a challenge - Tutorial

 

 

1. Create an empty challenge

Go to New Challenge page. Provide basic information (can be changed later on):

  • Title - human-readable title to appear on the competition pages.
  • Name - machine-readable name that will be used to create URL address of contest pages.
  • Folder - where in Repository you'd like to store contest files: datasets, solutions, supplemental materials.
  • Type - see Types of Challenges

The challenge will not show up publicly on the list of challenges until you explicitly publish it.

2. Upload datasets

Go to Evaluation Settings tab and choose datasets from your local disk that should be uploaded to the challenge folder and used in evaluation of solutions. Challenge folder is the single place in Repository where all challenge resources are kept. It's accessible through Repository pages, so you can easily browse its contents, upload additional files that might be helpful for participants, create subfolders, add file descriptions.

3. Configure evaluation of solutions

TunedIT has very flexible testing mechanism. Evaluation methods are implemented as pluggable Java class, evaluation procedures. You may use any procedure you want, either chosen from predefined ones or implemented by your own.

4. Write overview and task description

There are two places where you can put information for participants: Overview and Task pages. Descriptions are written in wiki text. You can use formatting and insert links to other pages.

overview.png

5. Check & Publish

Go to Checklist page to view all the settings and check if they're properly configured. If you wish, you may perform dry-run tests to see if everything works as expected. Then, press Publish to make the challenge visible on Challenges page.

publish.png

6. Accept solutions

The challenge will be opened automatically when the start date passes. You only have to start TunedTester application, which will be automatically downloading and evaluating new solutions submitted by participants. Preliminary results will show up on Leaderboard immediately after calculation.

leaderboard.png

7. Announce final results

The challenge will be closed automatically when the end date passes, and final results will be published on Leaderboard. You may wish to write summary of the challenge on Overview page.