Sunday, June 19, 2011

Automatic Illustration with Cross-media Retrieval in Large-scale Collections

Filipe Coelho, Cristina Ribeiro - CBMI 2011


In this paper, we approach the task of finding suit-able images to illustrate text, from specific news stories to more generic blog entries. We have developed an automatic illustration system supported by multimedia information retrieval, that analyzes text and presents a list of candidate images to illustrate it. The system was tested on the SAPO-Labs media collection, containing almost two million images with short descriptions, and the MIRFlickr-25000 collection, with photos and user tags from Flickr. Visual content is described by the Joint Composite Descriptor and indexed by a Permutation-Pre x Index. Illustration is a three-stage process using textual search, score filtering and visual clustering. A preliminary evaluation using exhaustive and approximate visual searches demonstrates the capabilities of the visual descriptor and  approximate indexing scheme used.




No comments: