{"id":2818,"date":"2014-11-03T12:20:19","date_gmt":"2014-11-03T12:20:19","guid":{"rendered":"https:\/\/irsg.bcs.org\/informer\/?p=2818"},"modified":"2014-11-03T12:20:19","modified_gmt":"2014-11-03T12:20:19","slug":"music-information-retrieval-at-the-johannes-kepler-university-linz-austria","status":"publish","type":"post","link":"https:\/\/archive-irsg.bcs.org\/informer\/?p=2818","title":{"rendered":"Music Information Retrieval at JKU Linz"},"content":{"rendered":"

The Department of Computational Perception<\/a> at the Johannes Kepler University Linz<\/a> was founded in October 2004, with the appointment of Prof. Gerhard Widmer<\/a>. Its mission is to develop computational models and algorithms that permit computers to perceive and “understand” aspects of the external world, where we interpret “perception” in the widest sense of the word, as the extraction of useful high-level information from complex, possibly low-level data, including\u00a0text,\u00a0audio, video, image, and sensor data.<\/p>\n

While the research carried out at the department is highly interdisciplinary, connecting fields like machine learning, music understanding, signal processing, and web mining, the area of music information retrieval (MIR) has always been a focus.\u00a0In the following, a selection of our research directions related to music retrieval and recommendation<\/strong> is described and corresponding prototype applications are presented. Supporting videos are available from our YouTube channel<\/a>.<\/p>\n

Audio content-based search, retrieval, and browsing<\/h1>\n

The development of highly efficient music feature extractors<\/strong> from audio, which describe for instance aspects of timbre or rhythm, is one of our core research topics. Given such features, we then elaborate approaches to automatic playlist generation<\/strong>, browsing music collections,<\/strong> and similarity-based search and retrieval<\/strong>, among others. Our algorithms have shown superior results in the audio music similarity and retrieval<\/a> task of the annual Music Information Retrieval Evaluation eXchange (MIREX)<\/a> competition several times. A prototype application for similarity-based music retrieval is the “Wolperdinger” music search engine, developed by Dominik Schnitzer<\/a>,\u00a0a former PhD student. This interface facilitates retrieving songs most similar to the currently played one,\u00a0in terms of audio similarity. To this end, the user just needs to click on the seed song and is immediately presented a new playlist of alike songs.<\/p>\n

\"\"<\/a>
The "Wolperdinger" system for music search and retrieval.<\/figcaption><\/figure>\n

A prototype for browsing collections of music that is potentially unknown to the user is the “nepTune<\/a>” interface. Given an arbitrary collection of digital music files, “nepTune” performs content-based clustering to create a virtual island landscape which allows the user to freely navigate in this collection and hear the closest sounds via a surround sound system. A demo video is available here<\/a>. This research area is led by Peter Knees<\/a>.<\/p>\n

\"\"<\/a>
The "nepTune" interface to browse music collections.<\/figcaption><\/figure>\n

Music identification from audio and from performance<\/h1>\n

Exploiting other kinds of audio features, we further elaborate methods to automatically segment an audio stream and\u00a0identify music pieces<\/strong> therein, even when the input signal is distorted or exhibits changes in tempo. To this end,\u00a0Reinhard Sonnleitner<\/a> develops tempo- and\u00a0pitch-invariant fingerprinting methods and\u00a0systems\u00a0that are able to correctly identify music tracks from a short audio recording even in very noisy environments\u00a0like live mixes by DJs, where tempo and pitch usually are manipulated in various\u00a0ways. In this context, music segmentation<\/strong> techniques, which we research too, play an important role to detect tracks in audio streams.<\/p>\n

Furthermore, similar techniques can be used in the domain of classical music,\u00a0where the goal is to identify the piece that is being performed<\/strong> based on a\u00a0database of sheet music. Here, we mainly focus on live piano performances and\u00a0develop methods that are able to achieve correct identification within a few\u00a0seconds. Coupled with a score following<\/strong> algorithm, this technology can be used as\u00a0a versatile music identification and tracking system. Without the need to look\u00a0for the score sheet, a musician can simply sit down at the piano and play a few bars. The system identifies the piece, finds the correct position,\u00a0tracks the progress over time, and even turns the pages automatically. A demo video is available\u00a0here<\/a>.\u00a0Research\u00a0in this direction, including working prototype applications, is conducted by\u00a0Andreas Arzt<\/a>.<\/p>\n

\"\"<\/a>
The "Piano Music Companion" for music identification and following.<\/figcaption><\/figure>\n

User-aware recommendation and playlist generation<\/h1>\n

Taking into account contextual aspects of\u00a0the listener is vital to create user-aware <\/strong>models of listening behavior<\/strong>. We develop and use such models to\u00a0specifically address the tasks of automatic playlist adaptation<\/strong>, music recommendation<\/strong>, and music browsing<\/strong>,\u00a0according to the listener’s current preferences. These preferences are influenced by a variety of factors, such as time, weather, activity, location, and social context (alone, with friends or family). A person might want to listen to an agitating rock song when doing sports, for instance, but prefer some relaxing reggae music when being at the beach on a sunny day. This line of research is directed by Markus Schedl<\/a>.<\/p>\n

Playlist generation on smart phones<\/h2>\n

Given today’s wide availability of smart mobile devices, we implemented one of our approaches to automatic playlist generation and adaptation in an Android application, called “Mobile Music Genius<\/a>“. This music player monitors more than 100 aspects of the listener’s context while she interacts with the player. It learns and constantly improves models that describe the relationship between the user context and her preferred artist or song. Feedback, such as play, pause, or skip events is used to infer where a user likes or dislikes an item.<\/p>\n

Playlists can be created manually or automatically by defining a seed song and some properties of the playlist (e.g., number of songs or whether songs by the seed artist should be included). In the latter case, the playlist is populated with songs most similar to the seed, which are sought using a model of tag-based similarity.<\/p>\n

During playback, the player continuously monitors the user context and compares it to the temporally preceding context. Once the discrepancy between the two exceeds a sensitivity threshold, a playlist update is triggered after the current song. In this case, the new context information is fed into a classifier trained on the user’s previous context and music preference data. The classifier then outputs a list of songs that were listened to in similar context, which are in turn added to the playlist.<\/p>\n

\"\"<\/a>
Playlist generation in "Mobile Music Genius".<\/figcaption><\/figure>\n

Geospatial music recommendation from social media data<\/h2>\n

While our research on user-aware playlist generation takes into account a wide variety of contextual listener aspects in a relatively small amount (a few thousands records), we also exploit the abundance of information that is present in social media data (hundreds of millions of records), even though much fewer aspects of the user context are directly available. As our focus is on MIR, we concentrate on Last.fm<\/a> and Twitter<\/a>, from which we mine\u00a0music- and listener-related information<\/strong>. This work already resulted in two data sets available for research, i.e., “MusicMicro<\/a>” and “Million Musical Tweets Dataset<\/a>“. In addition to the posting text, tweets are frequently attached GPS information, which enable interesting research tasks, such as geospatial music recommendation, interfaces to explore music listening behavior around the world, and music popularity estimation.
\nUsing the aforementioned data sources, in particular, information about listening events and listener’s characteristics, we elaborate methods to enhance music recommendation algorithms: content-based, collaborative filtering, and hybrids. We are particularly interested in hybrid fusion schemes<\/strong> and in analyzing the influence of user characteristics<\/strong> on the performance of music recommendation algorithms.<\/p>\n

Browsing interfaces to explore listening behavior<\/h2>\n

Another related task is the development of browsing interfaces to explore music listening events and music preferences on a worldwide scale. Such interfaces should be capable of analyzing differences between regions or countries<\/strong>. One of our prototypes is the “Music Tweet Map<\/a>” web application, which allows its user to retrieve and to explore the listening events of microbloggers<\/strong>, by time, location, genre, artist, and track. It was developed and is maintained by David Hauger<\/a>.<\/p>\n

\"\"<\/a>
Exploring listening events by location and genre using "Music Tweet Map".<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"

The Department of Computational Perception at the Johannes Kepler University Linz was founded in October 2004, with the appointment of Prof. Gerhard Widmer. Its mission is to develop computational models and algorithms that permit computers to perceive and “understand” aspects of the external world, where we interpret “perception” in the widest sense of the word,… Continue reading Music Information Retrieval at JKU Linz<\/span><\/a><\/p>\n","protected":false},"author":37,"featured_media":2935,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[183,204],"tags":[],"class_list":["post-2818","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-autumn-2014","category-org-overview","entry"],"_links":{"self":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/posts\/2818","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/users\/37"}],"replies":[{"embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2818"}],"version-history":[{"count":0,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/wp\/v2\/posts\/2818\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=\/"}],"wp:attachment":[{"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2818"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2818"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive-irsg.bcs.org\/informer\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2818"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}