Department of Software Technology
Vienna University of Technology

• 1. Data: description of the music data used
• 2. Feature Vectors: feature vectors used for training the map
• 3. Self-Organizing Map: SOM of the music files plus evaluation
• 4. Islands of Music: an Islands-of-music representation of the Collection 77
• 5. Component Planes: a closer look at the distribution of various features across the map.
• 6. Model Vectors: a closer look at the characteristics of the model vectors.

(Up to the top of the page)

2. Feature Vectors

• col77_segments.in.gz: 898 vectors representing the individual 6-second segments of music in the 1.200 dimensional feature space. (gnu-zipped, 4.1 MB)
• col77_median.in.gz: 77 vectors representing the pieces of music based on the median of their segments' vectors, again represented in 1.200 dimensional feature space. (gnu-zipped, 359 KB)
• somejb2.tv: an artificial template vector listing the 1.200 dimensions, used for the SOMLib DL system input.

(Up to the top of the page)

3. Self-Organizing Map

We also provide a hierarchical structuring of the individual segments of music. This provides a more fine-grained representation of the music archive, allowing each piece of music to be assigned to multiple locations, if its segments should belong to differing musical styles. For the segment-map we provide links to the 6-second segments, allowing precise comparison of their mutual similarity. (Please note: due to copyright-reasons we do NOT provide full MP3-files of titles, but rather segments of a few seconds length, downsampled to phone-line quality)
Furthermore, we provide the property files used for GHSOM training, allowing you to reproduce the examples presented below.

• SOM 2: The parameters for this GHSOM training run were set such, that all the data details had to be explained on a single layer map. We thus find no hierarchical structuring of the data on this map. The required absolute granularity of data representation was set to a lower value than with SOM 1 listed below, so the resulting SOM is more compact, having grown to a size of 5x5 units, i.e. using 25 units to represent the 77 songs. (GHSOM property file)

• SOM 1: The parameters for this GHSOM training run were set such, that all the data details had to be explained on a single layer map. We thus find no hierarchical structuring of the data on this map. The required absolute granularity of data representation is identical to the GHSOM below, thus this SOM basically resembles the GHSOMs listed above, but projected onto one layer. The resulting map is rather large, having grown to 8x7 units. Thus, several "empty" units separate the various clusters. (GHSOM property file)

• GHSOM 1: This map evolved into a 4x2 top-layer map with one further layer added to all but one unit (the one in the bottom left corner) in the hierarchical organization. It provides a somewhat more strongly structured view of the data set. In the top layer, soft classical music is in th eupper left corner, with titles becomming more dynamic and aggressive as you move towards the bottom righ tcorner. The organization of the second-level maps follows that overall orientation. (GHSOM property file)
  Integrated representation: This link takes you to the same GHSOM, yet in this representation the first two layers of the GHSOM have been integrated into a single map, allowing you to get a flat overview of the entire first two layers.

  GHSOM1 - Integrated View: Image of GHSOM 1 with the first 2 layers of the hierarchy integarted into a single top-layer map. (Click on the image to go to the actual map.)

• Segment-GHSOM 1: In order to see in how far the individual segments of the songs differ, we present a GHSOM trained with the individual segments of the pieces of music, rather than using their median as vector representation. Thus, each segment is located according to its sound characteristics. Although most segments of a given piece of music are mapped together onto one unit, we do find some segments that differ sufficiently from the remaining segments as to be mapped onto differing locations.
  The map evolves to a 3x3 map on the top layer (with all segments being listed at the individual units, making the individual cells uncomfortably large, but it helps in analyzing the maps). All units are expanded onto a second layer, with 83 units being expanded to a 3rd-layer map.
  The links present in the GHSOM point to the individual 6-second segments as they are used for feature vector representation. (GHSOM property file)

(Up to the top of the page)

4. Islands of Music

We now present the Islands of Music interface to a flat SOM trained with the 77-Collection, and take a look at the various component planes of the resulting SOM.

The figure above represents the standard SOM grid of a 7x7 SOM. If we apply the SDH visualization, we obtain the cluster representation depicted in the figure below. An interactive map, with the MP3 files linked to is provided right afterwards.

In the map below you will find the same clusters, yet interactive links to the MP3 files allow you to explore the clustering, to listen to the various sound characteristics, and get an impression of how the map organized the pieces of music. In the following step, we will take a closer look at the characteristic features that can be extracted from the mappings.

Click on the white bullets to liste to the music files (requires MP3-plugin)

The following image shows how peaks can be labeled to help navigation on the map. Basically, we can take a look at the values for each of the features separately, resulting in so-called component planes. Since the single dimensions are not very informative aggregated attributes can be formed from these and can be used to summarize characteristics of the clusters. There are several possibilities to form aggregated attributes, for example, using the sum, mean, or median of all or only a subset of the dimensions. Furthermore, it is possible to compare different subsets to each other. In the following 4 aggregated attributes, which point out some of the possibilities, are presented. If the user understands the MFS it is possible that the user directly creates the aggregated attributes depending on personal preferences. The used aggregated attributes are Maximum Fluctuation Strength, Bass, Non-Aggressive, and Low Frequencies Dominant. The names have been chosen to indicate what they describe. All peaks above the sea level (mountains and hills) have been labeled with descriptions. For example, the islands located at the upper left corner of the map is labeled maxflux ++, non-aggressive -, low-freq-dom +, 111bpm!, 232bpm!, 495bpm. These labels indicate disco music.

(Up to the top of the page)

5. Component Planes

We start with a representation of the low-frequencies dominating attribute, depicted below. The low-freq-dom is high for pieces of music where mostly the lower frequencies are active. There are two groups where this is the case. One is classical music like Für Elise by Beethoven. The other is music with a strong bass like the songs by Bomfunk MC's.

In the figure below, areas on the map are highlighted that have a strong bass beat. On this map the songs, which have the highest bass values, are located in the lower right corner and are from Bomfunk MC's. There is a strong correlation between the bass and the maxflux, which we will take a look at in the subsequent image.

In the figure below we can now take a closer look at which areas on the map have high or low maxflux values. The maximum fluctuation strength is high for pieces of music that have a strong re-occurring beat. On this map the songs, which have the highest maxflux values, are located in the lower right corner and are from Bomfunk MC's.

Last, but not least, we will take a look at an attribute entitled non-aggressive. The non-aggressive is high for pieces of music that don't have a strong re-occurring fast beat. On this map a song, which has very high non-aggressive values, is Memory by Barbara Streisand.

(Up to the top of the page)

6. Model Vectors

The figure below shows the un-scaled model vectors. They represent the data the same way the SOM algorithm sees them. Each model vector is titled with its coordinates on the 7x7 SOM. The linear x-axis of each MV represents the modulation frequency in the range from 0-10Hz. The y-axis represents the critical-band rate scale in the range from 1-20 Bark.

Last, but not least, we provide a rhythmic representation of each model vector. The images are created by summing up the MFS values over the critical-band rate scale and normalizing them so that their maximum equals one. Peeks are subsequently selected using a simple heuristic.

(Up to the top of the page)