We need user models to represent the different kinds of interests people have in multimedia content. We investigated existing ontological models on the Web (largely from Linked Open Data – LOD) for representing user interests but none is ideal for a user model, as a consequence we created the LinkedTV User Model Ontology (LUMO). It combines two main issues:
- (1) it allows us to represent the mental model of the user as needed in a widespread multimedia domain, and
- (2) it provides an ontological bridge to existing LOD ontologies including those used in the semantic annotation work (see ‘Linking video to Web content’).
A LinkedTV User Model Editor (LUME) supports the initial manual creation of sample profiles. The profiles consist of primitive and complex concepts from the reference LUMO ontology and a degree of preference (weighting) for each concept (sample below).
Implicit user profiling
Since users will typically not take much time and effort to create their own profile, and even less so to continue to modify that profile constantly over time, it is important to consider how implicit approaches can be included to derive and evolve a user interests model. In LinkedTV, we consider two distinct forms of implicit profiling:
- Based on user attention tracking (when and where the user looks at the screen). We will use analysis of Kinect video streams to determine a viewer’s face direction with respect to the TV.
- Based on user interaction in the LinkedTV player (which objects are selected, which content is browsed to). We will use the GAIN tracking component for this.
Preference learning methods are applied to modify the user model based on the collected implicit actions of the user, contributing to the development of the EasyMiner system. The overall workflow of the user model is shown below.
On the basis of the user model and the semantic annotations of the media content derived in the LinkedTV workflow (see ‘Linking video to Web content’) we filter the video enrichments according to the users’ preferences and interests.
We have defined two initial approaches to concept and content filtering, the LinkedTV Semantic Filter (LSF) and f-PocketKRHyper (see the LinkedTV document (D4.3)). LSF is based on weighted semantic matching (path distance) while f-PocketKRHyper takes advantage of more complex user preferences to post-process the initial LSF filtering results by means of fuzzy logical reasoning.
Deliverables and presentations
Specification of user profiling and contextualisation
User profile schema and profile capturing
Content and concept filter v1
Tools and services
Daniel Stein, Fraunhofer IAIS