TRIPOD Low-Level Image Content Analysis

Abstract

Many different sources of information can potentially be used to annotate and classify photographic images. Within the TRIPOD project a number of these information sources are to be used in combination to assist with the labelling of images for captioning and search. Information is to be derived from geographical information sources exploiting GPS data indicating the location at which an image was captured, but in addition to this we can also generate valuable information by analysing the visual content of image. This report describes the motivation, technologies and application of image content analysis tools for the TRIPOD project. Within TRIPOD images analysis is based on robust open-domain low-level content analysis tools which can be used individually or in combination with other information sources to label, classify or potentially filter out images which are not of interest within the TRIPOD system. Low-level image analysis forms Task 3.7 within WP3 “Caption Augmentation for Images with Existing Captions” of the TRIPOD project. The information generated by the low-level features analysis tools produced in Task 3.7 will be used in combination with the outputs of geographical and web-based information sources studied in Tasks 3.1 to 3.5 within Task 3.6 Caption Building. Within TRIPOD analysis of low-level image features focuses on the identification of simple image primitives such as colours and textures, generic features such as image quality, in the sense of focus and depth, and standard objects for the domain of interest. In the context of TRIPOD we are often interested in objects such as the presence of buildings, but we are generally less interested in images dominated by people who may be occluding the view of a feature of geographical interest. Thus we use a classifier to find buildings and identify photos containing people. This report further motivates the use of visual feature analysis within the TRIPOD project, gives an outline description of the selected low-level features for use within TRIPOD, and evaluates their performance for sample content typical of the TRIPOD geographical domain, and gives example scenarios for the use of these extracted features for indexing and search within TRIPOD.

Authors

Main Author(s): Mark Hughes, Gareth Jones Participants: DCU

Citations

Links

Link to the Deliverable: http://tripod.shef.ac.uk/outcomes/public_deliverables/Tripod_D3.3.pdf Project Website : http://tripod.shef.ac.uk/