Logo TU Vienna  Logo IFS Information & Software Engineering Group
Vienna University of Technology
Institute of Software Technology and Interactive Systems
Information & Software Engineering Group

Music Information Retrieval

Logo Music Information 
Retrieval at TU Vienna IFS
  [Topics] [Projects] [Downloads] [People] [Publications] [Press] [Events]  

RP extract





Innovative Tools for Creative People

TU Wien IFS is a partner of the MusicBricks project. Our aim is to provide innovative music technologies from European institutions to developers, hackers, digital makers and creative SMEs and companies.

"#MusicBricks are a compendium of both physical and virtual interfaces and APIs, that allow creators, developers and digital content makers easy access to core building blocks of music."

This is a collection of #MusicBricks tools provided by TU Wien:

  MusicBricks project logo

Rhythm & Timbre Feature Extraction from Music

RP_extract is a library that processes audio data (WAV, PCM, or MP3) as input and analyzes the spectral rhythmic and timbral information in the audio to create different audio descriptors (a.k.a. features).

These audio descriptors can be used to find similar sounding songs, create automatic playlists, make music recommendations etc. Depending on the needs, a  range of audio features is available:

Rhythmic descriptors:
  • RP: captures Rhythm Patterns (see picture at the right - audible frequency vs. repetition frequency) and is able to find songs with similar rhythm
  • RH: Rhythm Histogram: simplified rhythm descriptor (roughly containing rhythmic strength for different bpm values)
Timbral descriptors:
  • SSD: Statistical Spectrum Descriptor: describing timbral aspects of the audio Sonogram; by that it is possible to find songs with similar timbral characteristics
  • MVD: The Modulation Frequency Variance Descriptor measures variations over critical audible frequency bands for a specific rhythmic repetitions (derived from a rhythm pattern).
Temporal descriptors:
  • TSSD: Timbral variations over time (based on SSD)
  • TRH: Temporal Rhythm Histograms - rhythmic variations over time (based on RH)

Features can also be combined (e.g. to cover rhythmic and timbral aspects).

This library is available free and open source:


Feel free to use it in your own projects (but let us know about it :-).

  Audio Feature Extraction


SMINT API & "Search by Sound" Music Similarity Retrieval system

The Search by Sound system developed by Spectralmind (a TU Wien spin-out) can analyze music tracks and store their fingerprints (features) in a database. Via an API, one can query for similar sounding songs, using a combination of audio-feature based and meta-data query (e.g. "give me rhythmically similar songs from the genre 'Electronic'").

The system can be set up with a custom (or empty) music library where additional songs can be added via the SMINT API. We also provide a pre-analyzed library with 50,000 songs from freemusicarchive.org.

The backend analyzing all the songs run on our server. We provide a demo frontend usable in the Web browser and an API to add, delete and retrieve (similar) songs. 

The API methods are:         

  • track/add ... Add a Track by sending a URL where the track a) can be downloaded or b) providing a local file location (on the same server as the API is running).         
  • track/delete/:smint_track_id … Removes a track from the system.
  • track/:smint_track_id … Returns a list of tracks that are similar to the given trackid.
  • track_external_key/:external_key … Returns a list of tracks that are similar to the given external key. 
  • version … Returns version information on the API. 

The API returns an HTTP status code and an XML document.


Please contact us for the API URL and the Web frontend. ( lidy AT ifs.tuwien.ac.at )

  Search by Sound Logo


last edited 08.05.2015 by Thomas Lidy