I-SEARCH Multimodal Dataset
| Domain | Multimodal Search |
| Media | Text, Image, 3D, Audio, Video |
| Size | 45MB |
| Instances | 10305 |
| File Format | XML |
| Creation Date | 2012-06-08 |
| Task | Retrieval |
| Copyright | Creative Commons Attribution 3.0 Unported License. |
| URL | http://vcl.iti.gr/is/UC/ |
Description
The EU-funded project I-SEARCH aims to provide a novel unified framework for multimodal content indexing, search and retrieval. The I-SEARCH framework will be able to handle specific types of multimedia and multimodal content (text, 2D image, sketch, video, 3D objects and audio) along with real-world and user-related information, which can be used as queries and retrieve any available relevant content of any of the aforementioned types.
The searchable items within I-SEARCH will span from very simple media items (e.g., a single image or an audio file) to highly complex multimedia collections (e.g., a 3D object together with multiple 2D images and audio files) along with accompanying information. All the above multimedia collections are called Content Objects (CO). For a formal representation of COs, a novel description framework is introduced by I-SEARCH: the Rich Unified Content Description (RUCoD).
The latest version of RUCoD schema (v1.4.1) is now available [1]
A multimodal dataset has been created in I-SEARCH to demonstrate the Generic Multimodal Search Use Case. The dataset consists of 10305 COs classified into 51 categories. The COs consist of images, 3D objects, sounds and videos accompanied by textual information, tags and location information (if available).
Source
content is data mined from various web sources, images : flickr [2], 3d : Google 3D warehouse [3], audio : freesound [4], videos : YouTube [5],
Ground Truth Annotation
The RUCoDs are stored in separate folders (one folder for each category) thus classification information can be directly extracted. This can be used as ground truth for search and retrieval tasks.
Features
Low-level descriptors have been extracted for the 3D objects and images of the dataset. The links to descriptors are available at the RUCoD XML files (<L_Descriptor type="ImageType"> for the image descriptors and <L_Descriptor type="Object3D"> for the 3D object descriptors).
Licensing / Copyright
Creative Commons Attribution 3.0 Unported License.
Citation
P. Daras, A. Axenopoulos, V. Darlagiannis, D. Tzovaras, X. Le Bourdon, L. Joyeux, A. Verroust-Blondet, V. Croce, T. Steiner, A. Massari, A. Camurri, S. Morin, A-D. Mezaour, L. Sutton, S. Spiller, "Introducing a Unified Framework for Content Object Description", International Journal of Multimedia Intelligence and Security, Special Issue on “Challenges in Scalable Context Aware Multimedia Computing”, Volume 2, Number 3–4/2011, DOI 10.1504/IJMIS.2011.044765, Pages: 351-375, January 2012.
