The main goal of the TREC Video Retrieval Evaluation (TRECVID) is to promote progress in content-based analysis of and retrieval from digital video via open, metrics-based evaluation. TRECVID is a laboratory-style evaluation that attempts to model real world situations or significant component tasks involved in such situations.
Up until now, TRECVID has used test data from a small number of known professional sources - broadcast news organizations, TV program producers, and surveillance systems - that imposed limits on program style, content, production qualities, language, etc. In 2003 - 2006 TRECVID supported experiments in automatic segmentation, indexing, and content-based retrieval of digital video using broadcast news in English, Arabic, and Chinese. TRECVID also completed two years of pilot studies on exploitation of unedited video rushes provided by the BBC. In 2007 - 2009 TRECVID provided participants with cultural, news magazine, documentary, and education programming supplied by the Netherlands Institute for Sound and Vision. Tasks using this video included segmentation, search, feature extraction, and copy detection. Systems were tested in rushes video summarization using the BBC rushes. Surveillance event detection was evaluated using airport surveillance video provided by the UK Home Office. Many resources created by NIST and the TRECVID community are available for continued research on this data independent of TRECVID. See the Past data section of the TRECVID website for pointers.
In 2010 TRECVID will confront known-item search and semantic indexing systems with a new set of videos (referred to in what follows as IACC.1) characterized by a high degree of diversity in creator, content, style, production qualities, orginal collection device/encoding, language, etc - as is common in much "web video". The collection also has associated keywords and descriptions provided by the video donor. The videos are available under Creative Commons licenses from the Internet Archive. The only selection criteria imposed by TRECVID beyond the Creative Commons licensing will be one of video duration - they will be short (less than 4 minutes). In addition to the IACC.1 data set, NIST is developing an Internet multimedia test collection (HAVIC) with the Linguistic Data Consortium and plans to use it in an exploratory pilot task in TRECVID 2010. The airport surveillance video (Gatwick and i-LIDS MCT) used in TRECVID 2009 will be used again in 2010 as will some of the Sound and Vision (S&V) videos used in 2009.
In TRECVID 2010 NIST will evaluate systems on the following tasks using the [data] indicated::
-
Known-item search task (interactive, manual, automatic) [IACC.1]
-
Semantic indexing [IACC.1]
-
Content-based multimedia copy detection [IACC.1]
-
Event detection in airport surveillance video [i-LIDS]
-
Instance search [TV2009 S&V]
TRECVID 2010 will also offer the following pilot task:
- Event detection in Internet multimedia [HAVIC]
View the website for more information: www-nlpir.nist.gov/projects/trecvid/l |
Print
