Two papers at SIGIR2011 accepted

1: The TREC Files: the (ground) truth is out there (No kidding :))

The results of a retrieval system for a certain benchmark image database can be evaluated by several methods, each one employing various evaluation criteria. Traditional tools for information retrieval (IR) evaluation, such as TREC’s trec_eval, have outdated (command-line) interfaces with many unused features or ‘switches’ accumulated over the years. They are usually seen as cumbersome applications by new IR researchers, steepening the learning curve. We introduce a new, platform independent application for IR evaluation with a graphical easy-to-use interface: The TREC Files Evaluator. The application supports all standard measures used for evaluation in TREC, CLEF, and elsewhere, such as MAP, P10, P20, and bpref, as well as the Averaged Normalized Modified Retrieval Rank (ANMRR) proposed by MPEG for image retrieval evaluation [1]. Additional features include a batch mode and significance testing of the results against a pre-selected baseline run.

2: Bag-of-Visual-Words vs Global Image Descriptors on Two-Stage Multimodal Retrieval

Using Bag-of-Visual Words (BOVW) is fast becoming a widely used representation for content based image retrieval mainly, because of their better retrieval effectiveness over global feature representations on collections with images being near-duplicate to the test queries. In this experimental study we demonstrate that this advantage of BOVW is diminished when visual diversity is enhanced by using a secondary modality, such as text, to pre-filter images. In detail, the TOP-SURF descriptor is evaluated against Compact Composite Descriptors on a two-stage image retrieval system, which first uses a text modality to rank the collection and then perform CBIR only on the top-K items.