An adaptable image retrieval system with relevance feedback using kernel machines and selective sampling

This paper presents an adaptable content-based image retrieval (CBIR) system developed using regularization theory, kernel-based machines, and Fisher information measure. The system consists of a retrieval subsystem that carries out similarity matching using image-dependant information, multiple mapping subsystems that adaptively modify the similarity measures, and a relevance feedback mechanism that incorporates user information. The adaptation process drives the retrieval error to zero in order to exactly meet either an existing multiclass classification model or the user high-level concepts using reference-model or relevance feedback learning, respectively. To facilitate the selection of the most informative query images during relevance feedback learning a new method based upon the Fisher information is introduced. Model-reference and relevance feedback learning mechanisms are thoroughly tested on a domain-specific image database that encompasses a wide range of underwater objects captured using an electro-optical sensor. Benchmarking results with two other relevance feedback learning methods are also provided.

http://portal.acm.org/citation.cfm?id=1657314.1657336&coll=GUIDE&dl=GUIDE

ACM Multimedia 2010

ACM Multimedia 2010 is the worldwide premier multimedia conference and a key event to display scientific achievements and innovative industrial products. ACM Multimedia 2010 offers to scientists and practitioners in the area of Multimedia plenary scientific and technical sessions – with keynote speeches, oral, poster and video presentations, technical demonstrations and exhibits -, opportunities for deepenings – organized as discussion rooms, symposiums and tutorials and panels with worldwide recognized scientists and opinion leaders -, and competitions of research teams on relevant and challenging questions about the industry’s two-five years horizon for multimedia. As a companion event, the Interactive Art program will provide the opportunity of interaction between artists and computer scientists and investigation on the application of multimedia technologies to art and cultural heritage. Workshops associated to ACM Multimedia 2010 will provide in-focus forums of discussion on some of the most relevant and timely scientific topics in the field.

In more detail the complete program of ACM Multimedia 2010 will include:

Scientific presentations

• Full Paper and Short Paper program: including oral and poster presentations selected according to the ACM Multimedia policy and organized in four distinct tracks: Multimedia Content Analysis, Multimedia Systems, Multimedia Interaction, Multimedia Applications.
• Brave New Ideas program: implemented as a special sessions track containing papers extending the boundaries of multimedia research and eventually opening new lines of research development.
• Video program: implemented as a special session track of video displays where researchers can demonstrate their systems or applications without having to bring the equipment for a “live” demo.

Interactive Art program

• Interactive Art program: implemented as a special session track of multimedia art exhibition, and of applications, and technical solutions for cultural heritage.

Competitions

• Open Source Software Competition: aimed at emphasizing the impact the ACM Multimedia conference on software development and practice.
• Multimedia Grand Challenge: presenting challenges from industry leaders, geared to engage the multimedia research community in solving relevant questions about the industry’s 2-5 year horizon.

In-depth courses and discussions

• Tutorials: addressing the state-of-the-art developments of multimedia of interest from novices to mature researchers, from people working in academia to industry professionals.
• Panels: consisting of discussions with recognized experts, addressing  some of the most timely and controversial topics in the field of multimedia.
• Discussion Rooms: a new opportunity for researchers to gather freely and start non-structured debates on hot topics of multimedia.
• Doctoral Symposium: an opportunity for students involved in the preparation of a PhD to discuss their research issues and ideas with senior researchers, and receive constructive feedback.

Demos and exhibits

• Technical Demos: showing what is currently possible regarding all aspects of multimedia technology and its applications.
• Industrial Exhibit: providing the exhibition of new multimedia products of industries and companies to  stimulate the dialogue between industry and the research community.

Workshops

• Workshops will complement the themes of the main conference with focus on specific thematic subjects that receive significant attention from the research community.

http://www.acmmm10.org/conference/general-info/

Limitations of Content-based Image Retrieval

Theo Pavlidis
© Copyright 2008

http://theopavlidis.com/technology/CBIR/PaperB/vers3.htm

Abstract

This is a viewpoint paper where I discuss my impressions from the current state of the art and then I express opinions on what might be fruitful approaches. I find the current results in CBIR very limited in spite of over 20 years of research efforts. Certainly, I am not the only one who thinks that way, the lead editorial of a recent special issue of the IEEE Proceedings on multimedia retrieval was titled "The Holy Grail of Multimedia Information Retrieval: So Close or Yet So Far Away?"

I offer certain reasons for this state of affairs, especially for the discrepancy between high quality results shown in papers and poorer results in practice. The main reason seems to be that the lessons about feature selection and the "curse of dimensionality" in pattern recognition have been ignored in CBIR. Because there is little connection between pixel statistics and the human interpretation of an image (the "semantic gap") the use of large number of generic features makes highly likely that results will not be scalable, i.e. they will not hold on collections of images other than the ones used during the development of the method. In other words, the transformation from images to features (or other descriptors) is many-to-one and when the data set is relatively small, there are no collisions. But as the size of the set increases unrelated images are likely to be mapped into the same features.

I propose that generic CBIR will have to wait both for algorithmic advances in image understanding and advances in computer hardware. In the meantime I suggest that efforts should be focused on retrieval of images in specific applications where it is feasible to derive semantically meaningful features.

There are two appendices with examples of image retrieval. One presents the results obtained from some on line systems and the other presents some experiments I conducted to demonstrate how a method that yields impressive results in the author(s) paper gives poor results in independent tests. I have included a set of images that provide a challenge to the current CBIR methodologies. Two other appendices illustrate the severe limitations of color and edge strength histograms respectively for image characterization.

http://theopavlidis.com/technology/CBIR/PaperB/vers3.htm

Semantic Annotation of Medical Images

Sascha Seifert, Michael Kelm, Manuel Möller, Saikat Mukherjee, Alexander Cavallaro, Martin Huber, Dorin Comaniciu: “Semantic Annotation of Medical Images”, to appear in Proc. of SPIE Medical Imaging, San Diego, 2010.

Abstract: Diagnosis and treatment planning for patients can be signi cantly improved by comparing with clinical images of other patients with similar anatomical and pathological characteristics. This requires the images to be annotated using common vocabulary from clinical ontologies. Current approaches to such annotation are typically manual, consuming extensive clinician time, and cannot be scaled to large amounts of imaging data in hospitals. On the other hand, automated image analysis while being very scalable do not leverage standardized semantics and thus cannot be used across specific applications. In our work, we describe an automated and context-sensitive work based on an image parsing system complemented by an ontology-based context-sensitive annotation tool. An unique characteristic of our framework is that it brings together the diverse paradigms of machine learning based image analysis and ontology based modeling for accurate and scalable semantic image annotation.

This entry was posted on Wednesday, October 21st, 2009 at 5:27 pm and is filed under 2010, Co-Author, Conference, DFKI, English, MEDICO, Semantic Image Retrieval. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

http://www.manuelm.org/publications/?p=108

A Multimodal Dialogue Mashup for Medical Image Semantics

Daniel Sonntag, Manuel Möller: “A Multimodal Dialogue Mashup for Medical Image
Semantics”, to appear in Proceedings of the International Conference on Intelligent User Interfaces (IUI 2010), Hong Kong, China, 7.-10. Februar 2010

Abstract: This paper presents a multimodal dialogue mashup where different users are involved in the use of different user interfaces for the annotation and retrieval of medical images. Our solution is a mashup that integrates a multimodal interface for speech-based annotation of medical images and dialogue-based image retrieval with a semantic image annotation tool for manual annotations on a desktop computer. A remote RDF repository connects the annotation and querying task into a common framework and serves as the semantic backend system for the advanced multimodal dialogue a radiologist can use.

This entry was posted on Monday, November 30th, 2009 at 10:24 am and is filed under 2010, Co-Author, Conference, DFKI, English, MEDICO, User Interfaces. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

http://www.manuelm.org/publications/?p=113

Explore Similar Images with Google Image Swirl

Google has released an interactive similar images explorer. The app is called Google Image Swirl, and it’s using the wonder wheel Flash visualization you might know from web search results.

Here’s how it works: you enter a query, like “lion”. After a bit of loading, and if your keyword is supported (not all queries are), you’ll be presented with some visual base categories Google could find:

Opening a category by clicking on it will start the star exploration, with your image in center, surrounded by similar images. Clicking on a surrounding image will put it in focus, and new surrounding images are loaded dynamically:

Once you reached a final image with no more new neighbors, a click on it will take you to the original source site it’s crawled from:

In all of this, even due to its scripted Flash nature, the back button will take you to your last focus image. The app is fast, accessible, scaling pretty well to many keywords (not all – Google mentions there’s 200,000 at the moment), and on first glance it looks useful, too.

Already though, Google has a “Find similar images” link below pics in Google Images. It looks less glorious but feels similarly fast to use. The difference with Image Swirl seems to be that Swirl always takes into account your original keyword, meaning that new images will not only look similar, but also always shows e.g. a lion, if that’s what you entered. The similar images explorer currently available in Google Images on the other hand gets rid of your keyword and start a free-style visual exploration... i.e. the more clicks away from your original image, the less likely you’ll still be looking at lions. Both approaches have their merits, and it would be nice to see an additional pattern matching app which allows you to visually browse using a set of base shapes, which then become more and more refined, so that you could intuitively drill down towards your target pic.

http://blogoscoped.com/index2.html