TRIPOD Work package 3 Final Report

Abstract

WP3‟s purpose is to develop web services which can be chained together in a range of ways to add useful metadata to existing images, such as typically held by image libraries with a caption in the form of a title, and possibly additional keyword information. The starting point for almost all caption enhancement in WP3 is the identification of placenames within captions, and the grounding of these captions to a footprint, which typically takes the form of a point, but which may be extended to a polygon if sufficient knowledge about the placename is available, or through analysing spatial relations within the caption. Having identified a footprint for a caption, it is possible to associate captions with scenetypes through the use of land-cover data or part of speech analysis and the concept ontology, and then generate sets of candidate keywords to associate with an image. These keywords are scored through web search and the most likely keywords are associated with the caption. Furthermore, simple content-based techniques are applied to help in caption enhancement, for example by testing for whether an image contains buildings or was taken indoors or outdoors. Within this deliverable, we present an example analysis for five image captions, with a detailed discussion of the strengths and weaknesses of the individual services developed and their application to the problem. Since Tripod aims to explore the application of methods to multiple languages, a set of exemplar services were developed to deal with Latvian language captions, which can be integrated into the overall Tripod framework.

Authors

Main Author(s): Ross Purves, Martin Tomko, Alistair Edwardes, Xin Fan, Mark Hall, Mark Hughes, Andrew Salway, Inguna Skadiņa, Philip Smart Participants: UZH, USFD, CU, Tilde, DU

Citations

Links

Link to Deliverable : http://tripod.shef.ac.uk/outcomes/public_deliverables/Tripod_D3.5.pdf Project Website : http://tripod.shef.ac.uk/