Media Arts Scenario

University of Mons drives this scenario together with the NUMedia Institute for New Media Art Technology.

The goal of the Media Arts scenario is the artist as mediator between public and available multimedia content. Availability of tools for hyperlinking and annotating various forms of content, and the content itself, will constitute tremendous assets in increasing productivity, in providing a platform for experimenting novel approaches in storytelling, or more generally in stimulating creativity. In a context of reuse and repurposing, an artist can browse content in order to rapidly find the data he needs in order to create new content, or even an artistic performance. Indeed, several experiments of live content browsing seen as an art or entertainment form can be found, including the art of skilled live music remixing. Recently, richer content is experimented live by artists, and content browsers are part of the performance, facilitating retrieval, or supporting the storytelling, in settings where the screen is part of a hybrid stage.

Scenario Implementation and Demo

To acquire proposals for LinkedTV media art scenarios, University of Mons organised a special session for LinkedTV during the INTETAIN 2013 conference, a leading event on intelligent technologies for interactive entertainment. At that event it launched the MEDIADROM competition to the gathered artists as well as distributing the call afterwards to the media art community. The competition called for proposals how LinkedTV could be applied in “artful post-TV” scenarios, i.e. thinking outside of the classical ‘broadcast television’ box. Four proposals were retained after review for further development within the LinkedTV project.

Next, the media artists presented their concepts at a special session during MMM2014 (January 2014) and on that basis concepts suitable for implementation with LinkedTV technology were selected. In the end, one concept has been developed into a complete artistic installation available for presentation in Istanbul or in Mons – the SocialDocumentary.

Social Documentary

This concept is based around the use of multimodal and tangible interfaces for manipulating and searching multimedia content, focusing on the recent protests in the Gezi Park in Istanbul. We plan to process with LinkedTV large amounts of user generated content around the protests such as photographs, videos and tweets. We will explore how media linking can enable a new type of exploration of content around a social movement, combined with the usage of innovative interface modalities.

The visitors are invited to interact with the installation by choosing keywords through the manipulation of three cubes. Each cube represents a class of keywords (people, action, emotion). As the videos are cut into segments and tagged with the same keywords, the system will automatically choose the most relevant video according to the position of the cubes. A Kinect-based system allows us to track the two closest visitors’ faces and infer on their interest for the played video. The more the visitors are interested, the higher is the probability to display this video to the next visitors, like in recommendation system on video broadcast platforms. In this way, the installation evolves with its public. Some segments will emerge from the others as the “most popular”.

The video presents the public exhibit which completed a one week workshop that took place in August 2014, at BüyükAyi atelier, Istanbul. This workshop has boosted up the physical realisation of the project. In one week, the interaction table has been rebuilt from a former one coming from a previous BüyükAyi project, all the equipment has been included in it and the software parts have been upgraded. For example, a lot of filtering has been added to avoid false marker detection or loss of people tracking in the Interest Tracker software. The public presentation brought about twenty people to the BüyükAyi place, most of them were from artists’ circles but we also welcomed some tourists intrigued by the event. In general, the installation intrigued the visitors and they spent a few minutes playing with it.

U MONS has set-up an improved and further developed version of the multimodal interface on site in Mons. It was first shown in January 2015 at the NUMedia Institute (see pictures below) where the visitors were cultural and industrial partners of Region Nord-Pas-De-Calais, Region Wallonne and Region Flandre. The installation with all the sources is freely available on github (http://github.com/Zarfab/SocialDocumentary). We hope other people will build their own and present other types of multimedia content.

This version is being adapted now to use content from Mundaneum in a number of high profile events planned in 2015 as part of Mons European Capital City of Culture:

  • As a demo with video, text and image content at 3rd U Mons open day (25/04/2015)
  • As a demo with the initial content about GeziPark at ACM TVX2015 (Brussels) demo session (4-5/06/2015)
  • As a one week public installation for the opening of mundaneum (Mons2015 event) (starting from 11/06/2015)
  • As a demo for the BigDataWeek 2015 (Mons2015 event) (dates to be defined)

Leave a Comment

You must be logged in to post a comment.