LL4IR Round #3 Results

This page presents the LL4IR Round #3 results. This round ran from July 15, 2015 till July 31, 2015.

#Impressions is the total number of times when rankings (for any of the test queries) from the given team/system have been displayed to users. The number of wins, losses, and ties are calculated against the production system, where a win is defined as the experimental system having more clicks on results assigned to it by Team Draft Interleaving than clicks on results assigned to the production system. Outcome is defined as #wins/(#wins+#losses). An outcome value below 0.5 means that the experimental system performed worse than the production system (i.e., in overall, it has more losses than wins). (Note that we fixed the issue with the expected outcome that we encountered in the LL4IR Official Results.)

Participants can request the outcomes, for the entire query set (both test and train) as well as for each query individually, via GET /api/participant/outcome/(key)/(qid).