Downloads Publications and Journals
CUbRIK  Scientific Digest
 

 



histoGraph – A Visualization Tool for Collaborative Analysis of Historical Social Networks from Multimedia Collections

Award research paper presented to Session iV2014_1.2: Information Visualisation of iV 2014 , Paris, France, published in Proceedings of 18th International Conference Information Visualisation
By J.Novak, (European Institute for Participatory Media, Germany; University of Applied Sciences Stralsund, Germany); I.Micheel (European Institute for Participatory Media, Germany); L.Wieneke, (Centre Virtuel de la Connaissance sur l’Europe, Luxembourg); M. During (Centre Virtuel de la Connaissance sur l’Europe, Luxembourg ; University of North Carolina at Chapel Hill, NC, USA;); M.Melenhorst (Delft University of Technology,the Netherlands); J.Garcia Moron (Homeria Open Solutions S.L., Spain); C.Pasini, P.Fraternali, M.Tagliasacchi (Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Italy)

This paper describes the design and development of histoGraph, an interactive tool for explorative visualization and collaborative investigation of historical social networks from multimedia collections. Development of an interdisciplinary collaboration of computer scientists, historians, HCI researchers and interface designers, the tool aims at supporting historians in the discovery and historical analysis of relationships between people, places and events. A special focus is on the identification and interactive visualization of social relations from historical photo collections through a combination of automatic analysis and expert-based crowdsourcing. The tool design bridges the gap between established network analysis and visualization techniques and traditional hermeneutic research methods in historical research. It integrates visual exploration with hybrid social graph construction, hypothesis formulation and the consultation of digitized primary sources. A formative evaluation of the current prototype, developed as a domain-specific application for historians in the field of European integration points to opportunities and critical factors in applying this approach to support and further current research practices in digital humanities.
More


An Adaptive Teleportation Random Walk Model for Learning Social Tag Relevance

Full research paper presented to Social Media Session of SIGIR 2014 , 37th ACM Conference of the Special Interest Group On Information Retrieval, Gold Coast, Queensland, Australia, published in the 37th international ACM SIGIR conference on Research & development in information retrieval Pages 223-232
By X.Zhu, W.Nejdl, M.Georgescu (L3S Research Center, Hanover, Germany)

Social tags are known to be a valuable source of information for image retrieval and organization. However, contrary to the conventional document retrieval, rich tag frequency information in social sharing systems, such as Flickr, is not available, thus we cannot directly use the tag frequency (analogous to the term frequency in a document) to represent the relevance of tags. Many heuristic approaches have been proposed to address this problem, among which the well-known neighbor voting based approaches are the most effective methods. The basic assumption of these methods is that a tag is considered as relevant to the visual content of a target image if this tag is also used to annotate the visual neighbor images of the target image by lots of different users. The main limitation of these approaches is that they treat the voting power of each neighbor image either equally or simply based on its visual similarity. In this paper, we cast the social tag relevance learning problem as an adaptive teleportation random walk process on the voting graph. In particular, we model the relationships among images by constructing a voting graph, and then propose an adaptive teleportation random walk, in which a confidence factor is introduced to control the teleportation probability, on the voting graph. Through this process, direct and indirect relationships among images can be explored to cooperatively estimate the tag relevance. To quantify the performance of our approach, we compare it with state-of-the-art methods on two publicly available datasets (NUS-WIDE and MIR Flickr). The results indicate that our method achieves substantial performance gains on these datasets.
More


NicePic!: a system for extracting attractive photos from Flickr streams

Demonstration paper at SIGIR 2014 , 37th ACM Conference of the Special Interest Group On Information Retrieval, Gold Coast, Queensland, Australia, published in the 37th international ACM SIGIR conference on Research & development in information retrieval Pages 1259-1260
By S.Siersdorfer, S.Zerr, X.Zhu (L3S Research Center, Hanover, Germany), J. San Pedro (Telefonica Research, Barcelona, Spain), J. Hare (University of Southampton, United Kingdom)

A large number of images are continuously uploaded to popular photo sharing websites and online social communities. In this demonstration we show a novel application which automatically classifies images in a live photo stream according to their attractiveness for the community, based on a number of visual and textual features. The system effectively introduces an additional facet to browse and explore photo collections by highlighting the most attractive photographs and demoting the least attractive
More


Multi-Objective Optimization for Multimodal Visualization

Journal article for IEEE Transactions on Multimedia Volume:16 Issue:5, August 2014
By Kalamaras, I. (Dept. of Electr. & Electron. Eng., Imperial Coll. London, London, UK) Drosou, A. ; Tzovaras, D. (Centre for Research and Technology Hellas, Greece)

Using data visualization techniques can be of significant assistance in exploring multimedia databases. Data visualization is typically addressed as a unimodal learning task, where data are described with only one feature set, or modality. However, using multiple data modalities has been proved to increase the performance of learning methods. In this paper a novel approach for exploiting the multiple available modalities for visualization is proposed, motivated by the field of multi-objective optimization. Initially, each modality is considered separately. A graph of the dissimilarities among the data and the corresponding minimum spanning tree are formed. The suitability of a particular data placement is quantified using multiple cost functions, one for each modality. The utilized cost functions are defined in terms of graph aesthetic measures, computed for the unimodal minimum spanning trees. The cost functions are then used as the multiple objectives of a multi-objective optimization problem. Solving the problem results in a set of Pareto optimal placements, which represent different trade-offs among the various objectives. Experimental evaluation shows that the proposed method outperforms current multimodal visualization methods both in discovering more visualizations and in producing ones which are more aesthetically pleasing and easily perceivable.
More


Building the social graph of the History of European Integration: A pipeline for the Integration of Human and Machine Computation

Short paper presented to session 4 of DH 2014 Conference in Lausanne, Switzerland, published in Proceedings of the 2014 ACM conference on Web science, Pages 251-252
By L.Wieneke, G.Sillaume,M. Düring (Centre Virtuel de la Connaissance sur l’Europe, Luxembourg); C.Pasini, P.Fraternali, M.Tagliasacchi (Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Italy); M.Melenhorst (Delft University of Technology,the Netherlands); J.Novak, I.Micheel,E.Harloff (European Institute for Participatory Media, Germany); J.Garcia Moron (Homeria Open Solutions S.L., Spain); C.Lallemand (Public Research Centre Henri Tudor, Luxembourg); V.Croce, M.Lazzaro, F.Nucci (Engineering Ingegneria Informatica S.p.a., Italy)

The integration of human expertise and machine computation enables a new class of applications with significant potential for the digital humanities. So far this potential remains largely untapped due to the severe requirements of such projects: The implementation and integration of advanced algorithms requires specialized know-how and the final users from the humanities are challenged with defining unprecedented tasks for methods which haven’t emerged yet. The FP7-funded research project CUbRIK implements and integrates research in computer science, the design of human-computation tasks, data visualization, social engineering and the humanities. In the proposed presentation we would like to showcase one of CUbRIK’s case studies, the demo of the History of Europe application. The application introduces an effective interface to access collections of historical sources and to discover links among and entities within them. Upon completion CUbRIK will offer an innovative approach to human-enhanced time-aware multimedia search by synthesizing research in computer science, crowdsourcing and gamification. We will conclude the presentation with an outlook on the future development of the application.
More


Ars longa, vita brevis: Analysing the Duration of Trending Topics in Twitter Using Wikipedia

presented to poster session at WEBSci 2014 , ACM Web Science 2014 Conference in Cologne, Germany, published in Proceedings of the 2014 ACM conference on Web science, Pages 251-252
By Tuan Tran, M.Georgescu, X.Zhu, N.Kanhabua (L3S Research Center, Hannover, Germany)

The analysis of trending topics in Twitter is a goldmine for a variety of studies and applications. However, the contents of topics vary greatly from daily routines to major public events, enduring from a few hours to weeks or months. It is thus helpful to distinguish trending topics related to real-world events with those originated within virtual communities. In this paper, we analyse trending topics in Twitter using Wikipedia as reference for studying the provenance of trending topics. We show that among different factors, the duration of a trending topic characterizes exogenous Twitter trending topics better than endogenous ones.
More



Analysing and Mining Comments and Comments Ratings on the Social Web 

Published on   ACM Transactions on the Web (TWEB) , Volume 8 Issue 3, June 2014 Article No. 17 ,
By  S.Siersdorfer, S.Chelaru, W.Nejdl (L3S Research Center, Hannover, Germany); J.San Pedro (Telefonica Research, Barcelona, Spain); Ismail Sengor Altingovde (Middle East Technical University, Ankara, Turkey)

An analysis of the social video sharing platform YouTube and the news aggregator Yahoo! News reveals the presence of vast amounts of community feedback through comments for published videos and news stories, as well as through metaratings for these comments. This article presents an in-depth study of commenting and comment rating behavior on a sample of more than 10 million user comments on YouTube and Yahoo! News. In this study, comment ratings are considered first-class citizens. Their dependencies with textual content, thread structure of comments, and associated content (e.g., videos and their metadata) are analyzed to obtain a comprehensive understanding of the community commenting behavior. Furthermore, this article explores the applicability of machine learning and data mining to detect acceptance of comments by the community, comments likely to trigger discussions, controversial and polarizing content, and users exhibiting offensive commenting behavior. Results from this study have potential application in guiding the design of community-oriented online discussion platforms.
More


Provenance in Open Data Entity-centric aggregation

presented to poster session at ProvenanceWeek 2014 , Cologne, Germany
By F.Giunghiglia and Moaz Reyad (DISI, University of Trento, Italy)

Recently an increasing number of open data catalogs appear on the Web. These catalogs contain data that represents real world entities and their attributes. Data can be imported from several catalogs to build web services; hence there is a need to trace the source of each entity and attribute value in a way that handles also the possible conflicts between attribute values coming from overlapping sources. For open data, source tracing requires capturing both the provenance of the attribute values and the identity links between entities. Moreover, resolving the conflicts manually becomes harder with the increasing size of data. We propose a source tracing module that extends any existing import process by making it tracing-aware. The source tracing module contains three tools: authority, provenance and evidence. Authority provides rules for overriding attribute values, provenance specifies the source of an attribute value and evidence provides identity links between entities
More


Aggregation of Crowdsourced Labels Based on Worker History

presented to WIMS 2014 , Thessaloniki, Greece, published in Proceedings of the 4th International Conference on Web Intelligence, Mining and Semantics (WIMS14) Article No. 37
By M.Georgescu, Xiaofei Zhu (L3S Research Center, Leibniz Universität Hannover, Germany)

Using crowdsourcing for gathering labels can be beneficial for supervised machine learning, if done in the right way. Crowdsourcing is more cost-effective and faster than employing experts for labeling the items needed as training examples. Unfortunately, the crowd produced labels are not always of a comparable quality. Therefore, different methods could be employed in order to assure label quality. One of them is redundancy, by gathering more than one label per item, from different assessors. In this paper we introduce a novel method for aggregating multiple crowdsourced binary labels, taking into account the worker's history and how well the worker agrees with the aggregated label. According to previously solved tasks, the worker expertise, or the confidence we have in his labels can be assessed. The computation of the aggregated crowd label is mutually reinforced by the assessment of the worker confidence. Besides a method for computing a hard nominal aggregated label, we also propose a soft label as an indicator of how much the labelers agree and how strong their labels are. Furthermore, we investigate whether or not worker confidence should depend on the provided label, whether discriminating between positive and negative answer quality can be beneficial. We evaluate our method on multiple datasets, covering different domains and label gathering strategies. Moreover, we compare against other state of the art methods, showing the effectiveness of our proposed approach.
More


When in Doubt Ask the Crowd: Employing Crowdsourcing for Active Learning

presented to WIMS 2014 , Thessaloniki, Greece, published in Proceedings of the 4th International Conference on Web Intelligence, Mining and Semantics (WIMS14) Article No. 12
By M.Georgescu, Dang Duc Pham, Claudiu S. Firan, Ujwal Gadiraju, W.Nejdl (L3S Research Center, Leibniz Universität Hannover, Germany)

Crowdsourcing has become ubiquitous in machine learning as a cost effective method to gather training labels. In this paper we examine the challenges that appear when employing crowdsourcing for active learning, in an integrated environment where an automatic method and human labelers work together towards improving their performance at a certain task. By using Active Learning techniques on crowd-labeled data, we optimize the performance of the automatic method towards better accuracy, while keeping the costs low by gathering data on demand. In order to verify our proposed methods, we apply them to the task of deduplication of publications in a digital library by examining metadata. We investigate the problems created by noisy labels produced by the crowd and explore methods to aggregate them. We analyze how different automatic methods are affected by the quantity and quality of the allocated resources as well as the instance selection strategies for each active learning round, aiming towards attaining a balance between cost and performance.
More


A Novel Probabilistic Framework to Broaden the Context in Query Recommendation Systems

research paper presented to special session for Multimodal Recommendation Systems and their application to tourism at SETN 2014, 8th Hellenic Conference on Artificial IntelligenceIoannina, Greece

By Dimitrios Giakoumis, Dimitrios Tzovaras (Information Technologies Institute, Centre for Research and Technology Hellas, Greece)

This paper presents a novel probabilistic framework for broadening the notion of context in web search query recommendation systems. In the relevant literature, query suggestion is typically conducted based on past user actions of the current session, mostly related to query submission. Our proposed framework regards user context in a broader way, consisting of a series of further parameters that express it more thoroughly, such as spatial and temporal ones. Therefore, query recommendation is performed herein by considering the appropriateness of each candidate query suggestion, given this broadened context. Experimental evaluation showed that our proposed framework, utilizing spatiotemporal contextual features, is capable to increase query recommendation performance, compared to state-of-art methods such as co-occurence, adjacency and Variable-length Markov Models (VMM). Due to its generic nature, our framework can operate on the basis of further features expressing the user context than the ones studied in the present work, e.g. affect-related, toward further advancing web search query recommendation
More


On the application of game mechanics in information retrieval

presented to GamifIR 2014 , Amsterdam, The Netherlands, published in Proceedings of the First International Workshop on Gamification for Information Retrieval Pages 7-11
By L.Galli, P.Fraternali (Politecnico di Milano, Italy); A.Bozzon (Delft University of Technology, Netherlands)

The exponential growth of digital generated content in the form of audio, video and complex data structures calls for novel methods and tools able to cope with the limitation of automated analysis techniques. Gamification, the process of using game design methodologies and game mechanics to enhance traditional applications, is a promising tool that can help to increase the active involvement of humans in the Information Retrieval processes. This work contributes to the emerging research field of Gamification in Information Retrieval by providing an overview on: 1) the fundamental elements of a game; 2) the major game mechanics that have been applied in traditional games and gamication techniques; and 3) an overview of the possible adoption of such techniques in a typical IR scenario. The goal is to lay a path for the adoption of these new tools in IR systems, focusing on their application to the traditional building blocks of the query and content analysis processes.
More, Poster


Moody closet: exploring intriguing new views on wardrobe recommendation

Demonstration Paper at GamifIR 2014 workshop at ECIR2014, Amsterdam, The Netherlands, published in Proceedings of the First International Workshop on Gamification for Information Retrieval Pages 61-62
By Dumeljic, M.Larson, A.Bozzon (Delft University of Technology, Netherlands)

This paper introduces Moody Closet, a mobile application for the management of a personal wardrobe with a personalized outfit recommender. To provide incentive for the users to add content and express their preferences, the system provides an easy and enjoyable interaction, which delivers new perspectives on their closets. In particular, we focus on the mood of the wearer, which is considered to be an intriguing trigger capable of prompting the contribution of information needed to feed a recommendation system. An exploratory study with a small set of users provides an initial demonstration that the concept has the potential to fascinate users and motivate them to contribute. We demonstrate a working prototype which showcases the addition of content and the triggers provided to motivate this process.
More


People recognition using gamified ambiguous feedback

presented to GamifIR 2014 , Amsterdam, The Netherlands, published in Proceedings of the First International Workshop on Gamification for Information Retrieval Pages 22-26
By M.Brenner, N.Mirza,E.Izquierdo (Queen Mary University of London, UK)

We present a semi-supervised approach to recognize faces or people while incorporating crowd-sourced and gamified feedback to iteratively improve recognition accuracy. Unlike traditional approaches which are often limited to explicit feedback, we model ambiguous feedback information that we implicitly gather through a crowd that plays a game. We devise a graph-based recognition approach that incorporates such ambiguous feedback to jointly recognize people across an entire dataset. Multiple experiments demonstrate the effectiveness of our gamified feedback approach.
More


Learning to rank for joy

short paper in poster session at WWW Companion 2014, Seoul, Korea,  by Claudia Orellana-Rodriguez, and Wolfgang Nejdl (L3S Research Center Hannover, Germany), Ernesto Diaz-Aviles (IBM Research Dublin, Ireland), Ismail Sengor Altingovde (Middle East Technical University, Ankara, Turkey).

User-generated content is a growing source of valuable information and its analysis can lead to a better understanding of the users needs and trends. In this paper, we leverage user feedback about YouTube videos for the task of affective video ranking. To this end, we follow a learning to rank approach, which allows us to compare the performance of different sets of features when the ranking task goes beyond mere relevance and requires an affective understanding of the videos. Our results show that, while basic video features, such as title and tags, lead to effective rankings in an affective-less setup, they do not perform as good when dealing with an affective ranking task
More, Poster


Multimodal Detection, Retrieval and Classification of Social Events in Web Photo Collections-

presented to SEWM 2014 , Glasgow, United Kingdom.
by Markus Brenner and Ebroul Izquierdo (Queen Mary University of London, UK)

We present a framework to detect or cluster social events in web photo collections, retrieve associated photos and classify these photos according to event types. Compared to traditional approaches that often consider only textual or visual features without the notion of social events, our approach jointly utilizes both features while also incorporating other event-related contextual cues like date and time, location and usernames. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our combined constraint-based clustering and classification model.
More


VideoJot: A Multifunctional Video Annotation Tool

presented to ICMR2014, the annual ACM International Conference on Multimedia Retrieval, Demonstration Track, published in Proceedings of International Conference on Multimedia Retrieval Page 534.
By M.Riegler, M.Lux (Klagenfurt University, Klagenfurt, Austria); V.Charvillat, A. Carlier (University of Toulouse, Toulouse, France); R.Vliegendhart, M. Larson (Delft University of Technology, Delft, The Netherlands).

Videos are becoming more and more a tool of communication. There are how-to videos, people are discussing actions of others based on their recorded performance, e.g., in soccer, or they simply record videos of great moments and show them to friends and family. In this paper we focus on very specific how-to videos and present a novel, web based annotation tool, that combines (i) zoom, (ii) drawing, and (iii) temporal social bookmarking in video streams. Moreover, we present a short study on the usefulness of the tool to communicate general concepts of a specific video game based on a captured game session
More


Robust aggregation of GWAP tracks for local image annotation

Short paper presented to ICMR2014, the annual ACM International Conference on Multimedia Retrieval, session Demo and Poster, published in Proceedings of International Conference on Multimedia Retrieval Page 403.
By C. Bernaschina, P. Fraternali, L. Galli, D. Martinenghi, M. Tagliasacchi (Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano, Italy)

The possibility of assigning labels to localized regions in an image enables flexible image retrieval paradigms. However, the process of automatically segmenting and tagging images is notoriously hard, due to the presence of occlusions, noise, challenging illumination conditions, background clutter, etc. For this reason, human computation has recently emerged as a viable alternative when computer vision algorithms fail to provide a satisfactory answer. For example, Games with a purpose (GWAP) represent a powerful crowdsourcing mechanism to collect implicit annotations from human players. In this paper we consider the problem of aggregating the gaming tracks collected by a GWAP we developed to solve challenging instances of image segmentation problems. In particular we consider the existence of malicious players, who might try to fool the rules of the game to achieve higher rewards. The proposed solution can automatically estimate the reliability of human players, thus identifying cheaters. This information is exploited to aggregate the gaming tracks, thus significantly improving the image segmentation result and the quality of local image annotations.
More, Poster



Achievement Systems Explained

Book Chapter dedicated to the research made in CUbRIK in Trends and Applications of Serious Gaming and Social Media, pagg 25-50, Publisher: Springer Verlag Series: Gaming Media and Social Effects
By  L.Galli and P.Fraternali (Politecnico di Milano)

In the chapter of Achievement Systems Explained, Galli and Fraternali provide an insight on achievements, their purposes and the way in which they have evolved, and illustrate a taxonomy of possible achievements along with a set of guidelines to be followed when developing them. Finally, Galli and Fraternali introduce a model that can be used to describe all the existing systems in order to try to put the basis for an open platform capable of integrating existing gaming communities.
More



Fashion 10000: An Enriched Social Image Dataset for Fashion and Clothing

(presented in the Dataset Track Session II at MMSys 2014 Dataset, published in Proceedings of the 5th ACM Multimedia Systems Conference Pages 41-46 )
By B.Loni, Lei Yen Cheung, M.Riegler, A.Bozzon and M. Larson (Delft University of Technology, Netherlands); L.Gottlieb (International Computer Science Institute, Berkeley, CA)

In this work, we present a new social image dataset related to the fashion and clothing domain. The dataset contains more than 32000 images, their context and social metadata. Furthermore the dataset is enriched with several types of annotations collected from the Amazon Mechanical Turk (AMT) crowdsourcing platform, which can serve as ground truth for various content analysis algorithms. This dataset has been successfully used at the Crowdsourcing task of the 2013 MediaEval Multimedia Benchmarking initiative. The dataset contributes to several research areas such as Crowdsourcing, multimedia content and context analysis as well as hybrid human/automatic approaches. In this paper, the dataset is described in detail and the dataset collection strategy, statistics, applications of dataset and its contribution to MediaEval 2013 is discussed.
More


Div400: A Social Image Retrieval Result Diversification Dataset
(presented in the Dataset Track Session II at MMSys 2014 Dataset, published in Proceedings of the 5th ACM Multimedia Systems Conference pagg 29-34. )
By B.Ionescu (LAPI, University Politehnica of Bucharest, Romania); Anca-Livia Radu, M.Menéndez (DISI, University of Trento, Italy); H.Müller (HES-SO, Sierre, Switzerland); A.Popescu (CEA-LIST, France); B.Loni (Delft University of Technology).

In this paper we propose a new dataset, Div400, that was designed to support shared evaluation in different areas of social media photo retrieval, e.g., machine analysis (re-ranking, machine learning), human-based computation (crowdsourcing) or hybrid approaches (relevance feedback, machine-crowd integration). Div400 comes with associated relevance and diversity assessments performed by human annotators. 396 landmark locations are represented via 43,418 Flickr photos and metadata, Wikipedia pages and content descriptors for text and visual modalities. To facilitate distribution, only Creative Commons content was included in the dataset. The proposed dataset was validated during the 2013 Retrieving Diverse Social Images Task at the MediaEval Benchmarking Initiative for Multimedia Evaluation.
More


Towards Understanding the Implications of Social Role Manipulation in Online Tasks

research paper presented to Co-creating and Identity Making Workshop at CSCW 2014,  ACM International Conference on  Computer Supported Cooperative Work and Social Computing
By Mieke H. R. Leyssen (Centrum Wiskunde & Informatica, The Netherlands); M.Larson (Delft University of Technology, The Netherlands)

This paper provides an initial discussion of the ethical issues arising when people are asked to assume a role and, from the perspective of that role, asked to carry out an online task. We identify the following considerations: (a) People’s responses when playing a role can reveal personal information about themselves. (b) When people are asked to review the contributions of others who have a particular role, their behavior might indicate how they feel about these roles in their own life. (c) It is difficult to explain to people what they reveal about themselves when reporting their perception of the views of others. (d) People’s own view might change when they play a certain role for an extended period of time.
More


Assessing the impact of image manipulation on users' perceptions of deception

presented to IS&T/SPIE2014 , International Conference on Electronic Imaging , session Quality of Experience: User Experience in a Social Context, published in Proceedings of SPIE 9014, Human Vision and Electronic Imaging XIX, 90140Y (February 25, 2014);
By V.Conotter, Duc-Tien Dang-Nguyen, G.Boato, M.Menéndez(DISI, University of Trento) and Martha Larson (Delft University of Technology, The Netherlands)

Generally, we expect images to be an honest reflection of reality. However, this assumption is undermined by the new image editing technology, which allows for easy manipulation and distortion of digital contents. Our understanding of the implications related to the use of a manipulated data is lagging behind. In this paper we propose to exploit crowdsourcing tools in order to analyze the impact of different types of manipulation on users’ perceptions of deception. Our goal is to gain significant insights about how different types of manipulations impact users’ perceptions and how the context in which a modified image is used influences human perception of image deceptiveness. Through an extensive crowdsourcing user study, we aim at demonstrating that the problem of predicting user-perceived deception can be approached by automatic methods. Analysis of results collected on Amazon Mechanical Turk platform highlights how deception is related to the level of modifications applied to the image and to the context within modified pictures are used. To the best of our knowledge, this work represents the first attempt to address to the image editing debate using automatic approaches and going beyond investigation of forgeries.
More


A Hybrid Machine-Crowd Approach to Photo Retrieval Result Diversification

presented to MMM2014, 20th International Conference on Multimedia Modeling, oral presentation session of Interactive Retrieval
By Anca-Livia Radu (DISI, University of Trento, LAPI, University “Politehnica” of Bucharest); B.Ionescu (University “Politehnica” of Bucharest), M. Menéndez, J.Stöttinger, F.Giunchiglia and A. De Angeli (DISI, University of Trento)

In this paper we address the issue of optimizing the actual social photo retrieval technology in terms of users’ requirements. Typical users are interested in taking possession of accurately relevant-to-the-query and non-redundant images so they can build a correct exhaustive perception over the query. We propose to tackle this issue by combining two approaches previously considered nonoverlapping: machine image analysis for a pre-filtering of the initial query results followed by crowd-sourcing for a final refinement. In this mechanism, the machine part plays the role of reducing the time and resource consumption allowing better crowd-sourcing results. The machine technique ensures representativeness in images by performing a re-ranking of all images according to the most common image in the initial noisy set; additionally, diversity is ensured by clustering the images and selecting the best ranked images among the most representative in each cluster. Further, the crowd-sourcing part enforces both representativeness and diversity in images, objectives that are, to a certain extent, out of reach by solely the automated machine technique. The mechanism was validated on more than 25,000 photos retrieved from several common social media platforms, proving the efficiency of this approach.
More


Joint People Recognition across Photo Collections Using Sparse Markov Random Fields

presented to MMM 201420th International Conference on Multimedia Modeling, poster and demonstration session
By Markus Brenner and Ebroul Izquierdo (School of EECS, Queen Mary University of London, UK)

ABSTRACT: We show how to jointly recognize people across an entire photo collection while considering the specifies of personal photos that often depict multiple people. We devise and explore a sparse but efficient graph design based on a second-order Markov Random Field, and that utilizes a distance-based face description method. Experiments on two datasets demonstrate and validate the effectiveness of our probabilistic approach compared to traditional methods
More


Analyzing, Detecting, and Exploiting Sentiment in Web Queries
(published on ACM Transactions on the Web (TWEB) Volume 8 Issue 1, December 2013 Article No. 6)
By S.Chelaru, S.Siersdorfer, W.Nejdl (L3S Research Center); Ismail Sengor Altingovde (Middle East Technical University)

The Web contains an increasing amount of biased and opinionated documents on politics, products, and polarizing events. In this article, we present an indepth analysis of Web search queries for controversial topics, focusing on query sentiment. To this end, we conduct extensive user assessments and discriminative term analyses, as well as a sentiment analysis using the SentiWordNet thesaurus, a lexical resource containing sentiment annotations. Furthermore, in order to detect the sentiment expressed in queries, we build different classifiers based on query texts, query result titles, and snippets. We demonstrate the virtue of query sentiment detection in two different use cases.
First, we define a query recommendation scenario that employs sentiment detection of results to recommend additional queries for polarized queries issued by search engine users. The second application scenario is controversial topic discovery, where query sentiment classifiers are employed to discover previously unknown topics that trigger both highly positive and negative opinions among the users of a search engine. For both use cases, the results of our evaluations on real-world data are promising and show the viability and potential of query sentiment analysis in practical scenarios.

More


Building the social graph of the History of European Integration. A pipeline for Humanist-Machine Interaction in the Digital Humanities
(full research paper presented at HistoInformatics 2013  and DHd2014- published on Social Informatics- Lecture Notes in Computer Science Volume 8359, 2014, pp 86-99) 
By L.Wieneke, M.Düring,G.Sillaume (Centre Virtuel de la Connaissance sur l’Europe);C.Lallemand (Public Research Centre Henri Tudor); V.Croce, M.Lazzaro, F.Nucci (Engineering Ingegneria Informatica SpA); C.Pasini, P.Fraternali, M.Tagliasacchi (Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano); M.Melenhorst (Delft University of Technology);J.Novak, I.Micheel, E.Harloff (European Institute for Participatory Media); J.Garcia Moron (Homeria Open Solutions S.L.)

The breadth and scale of multimedia archives provides a tremendous potential for historical research that hasn’t been fully tapped up to know. In this paper we want to discuss the approach taken by the History of Europe application, a demonstrator for the integration of human and machine computation that combines the power of face recognition technology with two distinctively different crowd-sourcing approaches to compute co-occurrences of persons in historical image sets. These co-occurrences are turned into a social graph that connects persons with each other and positions them, through information about the date and location of recording, in time and space. The resulting visualization of the graph as well as analytical tools can help historians to find new impulses for research and to un-earth previously unknown relationships. As such the integration of human expertise and machine computation enables a new class of applications for the exploration of multimedia archives with significant potential for the digital humanities.
MoreVideo , Poster, Presentation


Content-based Tag Propagation and Tensor Factorization for Personalized Item Recommendation based on Social Tagging (article on ACM Transactions on Interactive Intelligent Systems,Volume 3 Issue 4, January 2014, Article No. 26)
By D.Rafailidis, A. Axenopoulos, S. Manolopoulou, and P. Daras (CERTH, Greece)  and  J. Etzold (Fulda University of Applied Sciences, Germany) 


In this paper, a novel method for personalized item recommendation based on social tagging is presented.
The proposed approach comprises a content-based tag propagation method, to address the sparsity and “cold start” problems, which often occur in social tagging systems and decrease the quality of recommendations. The proposed method exploits (a) the content of items and (b) users’ tag assignments through a relevance feedback mechanism, in order to automatically identify the optimal number of content-based and conceptually similar items.
The relevance degrees between users, tags, and conceptually similar items are calculated, in order to ensure accurate tag propagation and consequently to address the issue of “learning tag relevance”. Moreover, the ternary relation among users, tags and items is preserved by performing tag propagation in the form of triplets based on users’ personal preferences and “cold start” degree. The latent associations among users, tags and items are revealed based on a tensor factorization model, in order to build personalized item recommendations. In our experiments with real world social data, we show the superiority of the proposed approach over other state-of-the-art methods, since several problems in social tagging systems are successfully tackled.Finally, we present the recommendation methodology in the context of the multimodal engine of I-SEARCH.
More


Textual and Content-Based Search in Repositories of Web Application Models
(ACM Transactions on the Web (TWEB) Volume 8 Issue 2, Article No. 11, March 2014)
By Bojana Bislimovska, Marco Brambilla, Piero Fraternali (Politecnico di Milano) , Alessandro Bozzon (Delft University of Technology)  

Model-Driven Engineering relies on collections of models, which are the primary artifacts for software development. To enable knowledge sharing and reuse, models need to be managed within repositories, where they can be retrieved upon users queries. This paper examines two different techniques for indexing and searching model repositories, with a focus on Web development projects encoded in a Domain Specific Language. Keyword-based and content-based search (also known as query-by-example) are contrasted, with respect to the architecture of the system, the processing of models and queries, and the way in which metamodel knowledge can be exploited to improve search. A thorough experimental evaluation is conducted to examine what parameter configurations lead to better accuracy and to offer an insight in what queries are addressed best by each system.
More


Using Crowdsourcing to Capture Complexity in Human Interpretations of Multimedia Content
(Advances in Computer Vision and Pattern Recognition 2014, pp 229-269, - Ionescu, B. et al. Springer, Mar 2014)
By Martha Larson, Mark Melenhorst (Delft University of Technology, The Netherlands), María Menéndez (DISI, University of Trento, Italy), Peng Xu (Delft University of Technology, The Netherlands)

Large-scale crowdsourcing platforms are a key tool allowing researchers in the area of multimedia content analysis to gain insight into how users interpret social multimedia. The goal of this article is to support this process in a practical manner that opens the path for productive exploitation of complex human interpretations of multimedia content within multimedia systems. We first discuss in detail the nature of complexity in human interpretations of multimedia, and why we, as researchers, should look outward to the crowd, rather than inward to ourselves, to determine what users consider important about the content of images and videos. Then, we present strategies and insights from our own experience in designing tasks for crowdworkers. Our techniques are useful to researchers interested in eliciting information about the elements and aspects of multimedia that are important in the contexts in which humans use social multimedia. 
More




How Useful are Social Features for Learning to Rank YouTube Videos? 
(article on World Wide Web September 2014, Volume 17, Issue 5, pp 997-1025)
By  S. Chelaru, C. Orellana and  I.S. Altingovde (L3S Research Center, Germany). 
A vast amount of social feedback expressed via ratings (i.e., likes and dislikes) and comments is available for the multimedia content shared through Web 2.0 platforms. However, the potential of such social features associated with shared content still remains unexplored in the context of information retrieval. In this paper, we first study the social features that are associated with the top-ranked videos retrieved from the YouTube video sharing site for the real user queries. Our analysis considers both raw and derived social features. Next, we investigate the effectiveness of each such feature for video retrieval and the correlation between the features. Finally, we investigate the impact of the social features on the video retrieval effectiveness using state-of-the-art learning to rank approaches. In order to identify the most effective features, we adopt a new feature selection strategy based on the Maximal Marginal Relevance (MMR) method, as well as utilizing an existing strategy. In our experiments, we treat popular and rare queries separately and annotate 4,969 and 4,949 query-video pairs from each query type, respectively. Our findings reveal that incorporating social features is a promising approach for improving the retrieval performance for both types of queries.
More



A data driven approach for Social Event Detection 
(full research  paper presented at MediaEval 2013, workshop)
By  Dimitrios Rafailidis, Theodoros Semertzidis, Michalis Lazaridis, Petros Daras (CERTH), Theodoros Semertzidis (CERTH and University of Thessaloniki, Greece), Michael G. Strintzis (University of Thessaloniki, Greece) .

In this paper, we present a data-driven approach for challenge 1 of the MediaEval 2013 Social Event Detection Task. Our proposed approach consists of the following steps: (a) initialization based on the images’ spatio-temporal information; (b) computation of clusters’ intercorrelations; and (c) the final clusters’ generation. In the initialization step, the images that have both geolocation and time information are clustered analogously, where few“anchored”clusters are generated, while the rest of images with no geolocation or time information are considered as singleton (one image) clusters. In the second step, all pairwise intercorrelations between the “anchored” and the singleton clusters are calculated with the help of an aggregated similarity measure based on the user, title, description tag, and visual information of images. In the final step, the “anchored” and singleton clusters derived by the initialization step are merged based on the calculated intercorrelations of the second step to generate the final clusters. Our best run achieves a score of 0.5701, 0.8739 and 0.5592 for F1-Measure, NMI and Divergence (F1), respectively.
 More


MediaEval 2013: Social event detection, retrieval and classification in Collaborative Photo Collections (full research  paper presented at MediaEval 2013, workshop)
By  Markus Brenner, Ebroul Izquierdo (Queen Mary University of London, United Kingdom)

We present a framework to detect social events, retrieve associated photos and classify the photos according to event types in collaborative photo collections as part of the MediaEval 2013 benchmarks. We incorporate various contextual cues using both a constraint-based clustering model and a classification model. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.
 More
 

Retrieving Diverse Social Images at MediaEval 2013: Objectives, Dataset and Evaluation"
(workshop paper presented at MediaEval 2013, workshop)
By  Bogdan Ionescu (LAPI, University Politehnica of Bucharest, Romania), María Menéndez (DISI, University of Trento, Italy), Henning Müller (HES-SO, Sierre, Switzerland), Adrian Popescu (CEA-LIST, France).
This paper provides an overview of the Retrieving Diverse Social Images task that is organized as part of the MediaEval 2013 Benchmarking Initiative for Multimedia Evaluation. The task addresses the problem of result diversification in the context of social photo retrieval. We present the task challenges, the proposed data set and ground truth, the required participant runs and the evaluation metrics.
 More


Crowdsourcing for Social Multimedia at MediaEval 2013: Challenges, Data set, and Evaluation
(workshop paper presented at MediaEval 2013, workshop)
By  by Babak Loni, Martha Larson, Alessandro Bozzon ((Delft University of Technology, The Netherlands), Luke Gottlieb (International Computer Science Institute, Berkeley, CA, USA).

This paper provides an overview of the Crowdsourcing for Multimedia Task at MediaEval 2013 multimedia benchmarking initiative. The main goal of this task is to assess the potential of hybrid human/conventional computation techniques to generate accurate labels for social multimedia content. The task data are fashion-related images, collected from the Web-based photo sharing platform Flickr. Each image is accompanied by a) its metadata (e.g., title, description,and tags), and b) a set of ‘basic human labels’ collected from human annotators using a microtask with a basic quality control mechanism that is run on the Amazon Mechanical Turk crowdsourcing platform. The labels reflect whether or not the image depicts fashion, and whether or not the image matches its ‘category’ (i.e., the fashion-related query that returned the image from Flickr). The ‘basic human labels’ were collected such that their noise levels would be characteristic of data gathered from crowdsourcing workers without using highly sophisticated quality control. The task asks participants to predict high-quality labels, either by aggregating the ‘basic human labels’ or by combining them with the context (i.e., the metadata) and/or the content (i.e., visual features) of the image. .
 More


L3S @ MediaEval 2013 Crowdsourcing for Social Multimedia Task
(full research paper presented at MediaEval 2013, workshop)
By  Mihai Georgescu, Xiaofei Zhu (L3S Research Center, Leibniz Universität Hannover, Germany)

In this paper we present results of our initial research on aggregating noisy crowdsourced labels, by using a modified version of the EM algorithm introduced in A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied Statistics]. We propose different methods of estimating the worker confidence, a measure that indicates how well the worker is performing the task, and of integrating it in the computation of the aggregated label. Furthermore, we introduce a novel method of computing the worker confidence by using the soft aggregated labels. In order to assess the effectiveness of our proposed methods, we experiment on the MediaEval 2013 Crowdsourcing for Social Multimedia Task dataset.
 More


How Do We Deep-Link? Leveraging User-Contributed Time-Links for Non-Linear Video Access
(presented at ACM MM 2013, Main Conference Poster Session PS1)
By  R.Vliegendhart, B.Loni, M.Larson and A.Hanjalic (Multimedia Information Retrieval Lab, Delft University of Technology, The Netherlands)

This paper studies a new way of accessing videos in a non-linear fashion. Existing non-linear access methods allow users to jump into videos at points that depict specific visual concepts or that are likely to elicit affective reactions. We believe that deep-link comments, which occur unprompted on social video sharing platforms, offer a new opportunity beyond existing methods. With deep-link comments, viewers express themselves about a particular moment in a video by including a time-code. Deep-link comments are special because they reflect viewer perceptions of noteworthiness, that include, but extend beyond depicted conceptual content and induced affective reactions. Based on deep-link comments collected from YouTube, we develop a Viewer Expressive Reaction Variety (VERV) taxonomy that captures how viewers deep-link. We validate the taxonomy with a user study on a crowdsourcing platform and discuss how it extends conventional relevance criteria. We carry out experiments which show that deep-link comments can be automatically filtered and sorted into VERV categories. 
More, Poster

Strategies for user generated content and crowdsourcing in museums and cultural heritage (workshop paper for Digital Heritage 2013)
by Lars Wieneke (CVCE), Susan Hazan (The Israel Museum, Jerusalem), Nikolaos Maniatis, Ad Pollé, Marie-Hélène Serra, Christine Sauter, Stuart Dunn, James Brusuelas, Erwin Verbruggen, Roei Amit, Marion Dupeyrat and Christian Bajomi

User-generated content has become part and parcel of the mainstream Internet experience. Services like YouTube, Flickr or the Wikipedia provide platforms that encourage, enable and build on the creation of millions of users. Today, museums and memory institutions actively explore the inherent potential of user-generated content both to collect and share content with and from their audiences but also as a new tool for museum mediation. This workshop aims at bringing together the dispersed knowledge of practitioners from the digital heritage community as well as researchers from neighbouring disciplines to exchange best practices in engaging audiences, managing contributions and creating sustainable platforms for user-generated content. Presentation
 

Disco: Human and Machine Learning in Games workshop  (workshop paper for HCOMP2013)
By Markus Krause (Leibniz University Hannover, Germany), François Bry(Ludwig-Maximilian University Munich, Germany), Mihai Georgescu (L3S Research Center, Leibniz University Hannover, Germany)

Exploiting the playfulness of games has been extremely successful in bringing humans “in the loop” to solve complex computational tasks that would otherwise be hardly tractable. Although many proposals and systems after this paradigm have been developed, deployed, and tested, the relationship between play and human computation still deserves more investigations. Most work in human computation focuses on the ability for the machine to exploit, or learn from, humans. The workshop has a slightly different focus: the exploration of extending “I learn” (“disco” in Latin) to machines and humans alike. Games hold tremendous potential for discovery related to human and machine computation because of the intrinsic relation between play and learning. Extending and building upon the focus of past workshops on games and human computation Disco aims at exploring the intersection of entertainment, learning and human computation. More
 

Modeling Crowdsourcing scenarios in Socially-Enabled Human Computation Applications (published on  Journal of Data Semantics, Springer Berlin Heidelberg)
By  A.Bozzon, P.Fraternali, L.Galli and R.Karam (Politecnico di Milano, Italy)

User models have been defined since the '80s, mainly for the purpose of building context-based, user-adaptive applications. However, the advent of social networked media, serious games, and crowdsourcing/human computation platforms calls for a more pervasive notion of user model, capable of representing the multiple facets of social users and performers, including their social ties, interests, capabilities, activity history, and topical affinities. In this paper, we define a comprehensive model able to cater for all the aspects relevant for applications involving social networks and human computation; we capitalize on existing social user models and content description models, enhancing them with novel models for human computation and gaming activities representation. Finally, we report on our experiences in adopting the proposed model in the design and implementation of three socially-enabled human computation platforms.
More


Collective Action and Human Computation: From crowd-workers to social collectives
(article to appear on Handbook of Human Computation, Springer pp 43-63)
By J.Novak (European Institute for Participatory Media, Germany)

This chapter discusses how human computation can be conceptualized as a specific method within a broader context of collaborative systems. It considers how the theory of collective action and existing models of computer-supported collaboration can inform the design of new approaches to human computation. Accordingly, it proposes a conceptual design framework for collaborative human computation and illustrates its application to a prototypical design of an application integrating human computation with collective action.
More


Multimedia Indexing, Search and Retrieval in Large Databases of Social Networks
(article published on Social Media Retrieval, Springer pp 43-63)
By T.Semertzidis (Aristotle University of Thessaloniki,Greece; Centre For Research and Technology Hellas, Greece), D.Rafailidis and E.Tiakas (Centre For Research and Technology Hellas, Greece), M. G.Strintzis (Aristotle University of Thessaloniki,Greece; Centre For Research and Technology Hellas, Greece), P.Daras (Centre for Research and Technology Hellas, Greece)

Social networks are changing the way multimedia content is shared on the web, by allowing users to upload their photos, videos, and audio content, produced by any means of digital recorders such as mobile/smart-phones, and web/digital cameras. This plethora of content created the need for finding the desired media in the social media universe. Moreover, the diversity of the available content, inspired users to demand and formulate more complicated queries. In the social media era, multimedia content search is promoted to a fundamental feature towards efficient search inside social multimedia streams, content classification, context and event based indexing. In this chapter an overview of multimedia indexing and searching algorithms, following the data growth curve is presented in detail. The chapter is thematically structured in two parts. In the first part pure multimedia content retrieval issues are presented, while in the second part, the social aspects and new, interesting views on multimedia retrieval in the large social media databases are discussed.
More, Demonstrator



Imposing a semantic schema for the detection of potential mistakes in knowledge resources
(presented at On The Move ODBASE 2013 , Session Semantic Information Management)
By  V.Maltese (University of Trento, Italy)

Nowadays, there is a pressing need for very accurate, up-to-date and diversity-aware knowledge resources. As their maintenance is very expensive, we argue that the only affordable way to address this is by complementing automatic with manual checks. This paper presents an approach, based on the notion of semantic schema, which aims to minimize human intervention as it allows the automatic identification of potentially faulty parts of a knowledge resource which need manual checks. Our evaluation showed promising results.
More


Gender-aided People Recognition in Photo Collections (presented at IEEE MMSP 2013, Session OS:Retrieval)
By  M.Brenner and E.Izquierdo (Queen Mary University of London, UK)

We show how to recognize people based on their faces in Consumer Photo Collections while also incorporating context in the form of gender information. We devise and explore a unified framework that has a graphical model along a distancebased face description method at its core. We jointly recognize people across an entire photo collection to also consider the specifics of photos that depict multiple people. Experiments on two datasets demonstrate and validate the effectiveness of our probabilistic approach compared to traditional methods that do not consider gender information.
More


Combining human and machine computation for the digital humanities (presented at IEEE MMSP 2013, Demo Session)
By  L.Wieneke, G.Sillaume, C.Lallemand (Centre Virtuel de la Connaissance sur l’Europe, Luxembourg); J.Novak, I.Micheel (European Institute for Participatory Media, Germany);J. Garcia Morón (Homeria, Spain);M.Lazzaro and V.Croce (Engineering Ingegeria Informatica, Italy)

The History of Europe (HoE) demo showcases the potential of the FP7 IST funded research project CUbRIK in creating a new tool for digital humanities researchers through human-enhanced time-aware multimedia search. Based on the co-occurrence of persons in historical images a social graph is constructed, which becomes in itself a point of departure for new research questions in the humanities. The tight integration of machine and human computation enables a new perspective on the audio-visual heritage of the European Integration process.
Presentation  


Multimedia Search + Human Computation and Crowd Search Two Chapters dedicated to the research made in CUbRIK in Web Information Retrieval Book, Publisher: Springer Verlag Series: Data-Centric Systems and Applications
By  S.Ceri, A.Bozzon, M.Brambilla, E.Della Valle, P.Fraternali and S. Quarteroni. (Politecnico di Milano)

Chapter 13: Multimedia Search
The Web is progressively becoming a multimedia content delivery platform. This trend poses severe challenges to the information retrieval theories, techniques and tools. This chapter defines the problem of multimedia information retrieval with its challenges and application areas, overviews its major technical issues, proposes a reference architecture unifying the aspects of content processing and querying, exemplifies a next-generation platform for multimedia search, and concludes by showing the close ties between multi-domain search and multi- modal/multimedia search.

Chapter 15: Human Computation and Crowd Search

Human Computation (HC) is the discipline that aims at harmonizing the contribution of humans and computers in the resolution of complex tasks. The general principle of HC is to use computer and network architectures to organize the distributed allocation of work to a crowd of human performers, who contribute their skill in solving problems where algorithms fail or produce uncertain output, like object recognition in images or text translation. HC solutions assume a variety of forms, from crowdsourcing labor markets, to data collection or early alerting mobile applications, to games with a purpose and crowd search. This chapter overviews a few exemplary applications, discusses a conceptual framework that abstracts the common aspects of the existing approaches, classifies the dimensions that characterize a HC solution, and highlights some open research questions and projects addressing them.


From Knowledge Organisation to Knowledge Representation (presented at ISKO_UK 2013, Session 3B: Ontologies combine with other tools)
By  F.Giunchiglia(University of Trento), B.Dutta (DRTC - Indian Statistical Institute, Bangalore, India) and V.Maltese (University of Trento)

So far, within the Library and Information Science (LIS) community, Knowledge Organization (KO) has developed its own very successful solutions to document search, allowing for the classification and search of millions of books. However, current KO solutions are limited in expressivity as they only support queries by document properties, e.g., by title, author and subject. In parallel, within the Artificial Intelligence and Semantic Web communities, Knowledge Representation (KR), has developed very powerful end expressive techniques which, via the use of ontologies, support queries by any entity property (e.g., the properties of the entities described in a document). However, KR has not scaled yet to the level of KO, mainly because of the lack of a precise and scalable entity specification methodology. In this paper we present DERA, a new methodology, inspired by the faceted approach, as introduced in KO, that retains all the advantages of KR and compensates for the limitations of KO. DERA guarantees at the same time quality, extensibility, scalability and effectiveness in search
More


A semantic schema for GeoNames (presented at Inspire 2013, Parallel Session Semantics I)
By  V.Maltese and F.Farazi (University of Trento)

As part of a broader strategy towards supporting semantic interoperability in geospatial applications, in this paper we present a semantic schema we designed for GeoNames and the qualitative improvements we obtained by enforcing it on the data. 
More


Social Events and Social Ties (presented at ICMR 2013, Special Session: Social Events in Web Multimedia)
By  J.Paniagua, I.Tankoyeu, J.Stöttinger and F.Giunchiglia (University of Trento)

This paper is based upon an approach for automatic detection of personal events in on-line personal photo collections and proposes a powerful exploitation of these events: We compose social events out of personal events and then automatically reveal interpersonal ties. Trying to tame the stream of big data in social networks we solely rely on image meta-data of time and space. We validate our assumptions in the wild using 1.8 million public images of more than 4100 users. The proposed approach has three main steps: (i) personal event detection using individual, unsorted photo collections, in which we make use of the spatio-temporal context embedded in digital photos to detect event boundaries within the collection; (ii) social event detection for which we use a tailored similarity measurement between personal events of different users; and (iii) an analysis of event co-participation to propagate social connections. Experiments validate that the fully automated approach is able to accurately detect 78.76% of social events and reconstruct the interpersonal ties of a user with a verified true positive rate of 45%. This rate is probably much higher: Since most interpersonal ties are undefined in the universe of social networks, our experimental ground-truth of course remains fragmentary.
More



Recognizing people by face and body in Photo Collections (presented at FG2013, Session Face Technology Applications)
By  M.Brenner, E.Izquierdo (Queen Mary University of London, United Kingdom)

We show how to detect and recognize people based on their faces and bodies in Consumer Photo Collections. We devise a graphical model that incorporates multiple contextual cues to discriminate faces, upper and lower bodies, and ultimately, individuals without relying on faces. For efficiency, we only consider body features when faces are not discriminative enough. Experiments on two datasets demonstrate the effectiveness of our probabilistic approach.  
More



Multifeature Analysis and Semantic Context Learning for Image Classification article on ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), Volume 9 Issue 2, May 2013 Article No. 12
By  Q. Zhang and E.Izquierdo (Queen Mary, University of London)

This article introduces an image classification approach in which the semantic context of images and multiple low-level visual features are jointly exploited. The context consists of a set of semantic terms defining the classes to be associated to unclassified images. Initially, a multiobjective optimization technique is used to define a multifeature fusion model for each semantic class. Then, a Bayesian learning procedure is applied to derive a context model representing relationships among semantic classes. Finally, this context model is used to infer object classes within images. Selected results from a comprehensive experimental evaluation are reported to show the effectiveness of the proposed approaches. 
More


Generating Contextualized Sentiment Lexica based on Latent Topics and User Ratings
(long research paper  presented at Hipertext 2013)
By  Ralf Krestel, Stefan Siersdorfer (L3S Research Center)

Sentiment lexica are useful for analyzing opinions in Web collections, for domain-dependent sentiment classification, and as sub-components of recommender systems. In this paper, we present a strategy for automatically generating topic-dependent lexica from large corpora of review articles by exploiting accompanying user ratings. Our approach combines text segmentation, discriminative feature analysis techniques, and latent topic extraction to infer the polarity of n-grams in a topical context. Our experiments on rating prediction demonstrate a substantial performance improvement in comparison with existing state-of-the-art sentiment lexica. 
More


Reactive Crowdsourcing
(full research paper  presented at WWW 2013)
By  Alessandro Bozzon, Marco Brambilla,Stefano Ceri and Andrea Mauri (Politecnico di Milano)

An essential aspect for building effective crowdsourcing computations is the ability of "controlling the crowd", i.e. of dynamically adapting the behaviour of the crowdsourcing systems as response to the quantity and quality of completed tasks or to the availability and reliability of performers. Most crowdsourcing systems only provide limited and prede fined controls; in contrast, we present an approach to crowdsourcing which provides fine-level, powerful and flexible controls. We model each crowdsourcing application as composition of elementary task types and we progressively transform these high level specifi cations into the features of a reactive execution environment that supports task planning, assignment and completion as well as performer monitoring and exclusion. Controls are speci fied as active rules on top of data structures which are derived from the model of the application; rules can be added, dropped or modi fied, thus guaranteeing maximal flexibility with limited eff ort. We also report on our prototype platform that implements the proposed framework and we show the results of our experimentations with di fferent rule sets, demonstrating how simple changes to the rules can substantially a ffect time, e ffort and quality involved in crowdsourcing activities. 
More, Presentation Slides


Mining Emotions in Short Films: User Comments or Crowdsourcing?
(poster presented at WWW 2013)
By  Claudia Orellana-Rodriguez, Ernesto Diaz-Aviles, and Wolfgang Nejdl (L3S Research Center)

Short films are regarded as an alternative form of artistic creation, and they express, in a few minutes, a whole gamma of diff erent emotions oriented to impact the audience and communicate a story. In this paper, we exploit a multi-modal sentiment analysis approach to extract emotions in short films, based on the film criticism expressed through social comments from the video-sharing platform YouTube. We go beyond the traditional polarity detection (i.e., positive/negative), and extract, for each analyzed film, four opposing pairs of primary emotions: joy-sadness, anger-fear, trust-disgust, and anticipation-surprise.We found that YouTube comments are a valuable source of information for automatic emotion detection when compared to human analysis elicited via crowdsourcing. 
MorePoster


Temporal Summarization of Event-Related Updates in Wikipedia
(demonstrator presented at WWW 2013)
By  Mihai Georgescu, Dang Duc Pham, Nattiya Kanhabua Sergej Zerr, Stefan Siersdorfer, and Wolfgang Nejdl (L3S Research Center)

Wikipedia is a free multilingual online encyclopedia covering a wide range of general and speci fic knowledge. Its content is continuously maintained up-to-date and extended by a supporting community. In many cases, real-world events influence the collaborative editing of Wikipedia articles of the involved or a ffected entities. In this paper, we present Wikipedia Event Reporter, a web-based system that supports the entity-centric, temporal analytics of event-related information in Wikipedia by analyzing the whole history of article updates. For a given entity, the system first identifi es peaks of update activities for the entity using burst detection and automatically extracts event-related updates using a machine-learning approach. Further, the system determines distinct events through the clustering of updates by exploiting diff erent types of information such as update time, textual similarity, and the position of the updates within an article. Finally, the system generates the meaningful temporal summarization of event-related updates and automatically annotates the identified events in a timeline.
More, Presentation Video

 

The Open Platform for Personal Lifelogging: The eLifeLog Architecture (poster presented in Works-in-Progress session at CHI 2013)
By  Pil Ho Kim and F.Giunchiglia (DISI, University of Trento)

Lifelogging is a complex application domain of multimedia management. This makes it challenging for people to keep their personal lifelogs under their control. Our work aims to provide people with an open platform, named eLifeLog, that would work in user's private cloud to start archiving their valuable memories and experiences under the hood. eLifeLog has a number of distinct features that di erentiate it from legacy CMS (Content Management System) products or related works: (1) It is specialized for personal lifelogging, (2) it embeds an event-based uni ed data representation to handle heterogenous timestamped streams, and (3) it is open to the public with the complete source code for personal use and for accelerating lifelogging research collaboration.  
More

 

Event-driven retrieval in collaborative photo collections (presented to at WIAMIS 2013)
By  By M.Brenner and E.Izquierdo (Queen Mary University of London, United Kingdom)

We present an approach to retrieve photos relating to social events in collaborative photo collections. Compared to traditional approaches that typically consider only the visual features of photos as a source of information, we incorporate multiple additional contextual cues like date and time, location and usernames to improve retrieval performance. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach. 
More

 

People recognition in ambiguously labeled Photo Collections (presented in the session FO13: face and people recognition at ICME 2013)
By  By M.Brenner and E.Izquierdo (Queen Mary University of London, United Kingdom)

We show how to recognize people based on their faces in Consumer Photo Collections while also incorporating context in the form of ambiguous labels. Such labels can be assigned to single photos (depicting multiple people) as well as to entire sets of photos (e.g. relating to events). To achieve this, we devise a unified framework that has a graphical model along a distance-based face description method at its core. We evaluate our probabilistic approach by performing experiments on two datasets, one of which includes around 5000 face appearances spanning nearly ten years 
More

 

Building social graphs from images through expert-based crowdsourcing (presented at SoHuman 2013)
By  M. Dionisio, P. Fraternali, D. Martinenghi, C. Pasini, M. Tagliasacchi and S. Zagorac, (Politecnico di Milano, Italy), E.Harloff, I. Micheel and J.Novak (European Institute for Participatory Media, Germany)

The extraction of semantic information from multimedia content represents a challenging problem. Despite the continuous refinement of automatic tools, the quality and completeness of the results is not always satisfactory. To overcome this limitation, the vision of the CUbRIK project is to provide a multimedia search and exploration platform that seamlessly integrate human tasks and algorithms. In this paper, as a concrete example, we illustrate the design of a multimedia content processing pipeline that aims at extracting evidence of social relationships from the analysis of a photo collection covering the main events and people that shaped the history of Europe after World War II. We discuss the issues faced by generalpurpose crowdsourcing and automatic face detection/recognition algorithms in determining the identities of people portrayed in the photo collection. Hence, we illustrate the design of a system that tackles the uncertainty of the results produced by face detection/recognition with expert-based crowdsourcing. 
More

 

Exploiting User Generated Content for Mountain Peak Detection (presented at SoHuman 2013)
By  R. Fedorov, D.Martinenghi, M. Tagliasacchi and A.Castelletti (Politecnico di Milano, Italy)

We present a system for the classification of mountain panoramas from user-generated photographs followed by identification and extraction of mountain peaks from those panoramas. We have developed an automatic technique that, given as input a geo-tagged photograph, estimates its FOV (Field Of View) and the direction of the camera using a matching algorithm on the photograph edge maps and a rendered view of the mountain silhouettes that should be seen from the observer’s point of view. The extraction algorithm then identifies the mountain peaks present in the photograph and their profiles. We discuss possible applications in social fields like photographs peaks tagging on social portals, augmented reality on mobile devices when viewing a mountain panorama, and generation of collective intelligence systems (such as environmental models) from massive social media collections (e.g. snow water availability maps based on mountain peaks states extracted from photographs hosting services). 
More, Presentation slides, Demonstrator

 

MSIDX: Multi-Sort Indexing for Efficient Content-based Image Search and Retrieval  (published on Multimedia IEEE Transactions Volume PP Issue 99)
By  E.Tiakas, D.Rafailidis, A.Dimou and P.Daras, (Information Technologies Institute, Centre for Research and Technology Hellas)

In this paper, a novel approximate indexing scheme for efficient content-based image search and retrieval is presented, called Multi-Sort Indexing (MSIDX). The proposed scheme analyzes high dimensional image descriptor vectors, by employing the value cardinalities of their dimensions. The dimensions value cardinalities, an inherent characteristic of descriptor vectors, are the number of discrete values in the dimensions. As expected, value cardinalities significantly vary, due to the existence of several extraction methods. Moreover, different quantization and normalization techniques used in the extraction process, have a strong impact on the dimensions value cardinalities. Since dimensions with high value cardinalities have more discriminative power, a multiple sort algorithm is used to reorder the descriptors dimensions according to their value cardinalities, in order to increase the probability of two similar images to lie within a close constant range. The expected bounds of the constant range are defined in detail, following deterministic and probabilistic analyses. The proposed scheme is fully suitable (a) for real-time indexing of images, and (b) for searching and retrieving relevant images with an efficient query processing algorithm. In our experiments with five real datasets, we show the superiority of the proposed approach against hashing methods, also suitable for approximate similarity search. 
More , the database

 

Extracting Event-Related Information from Article Updates in Wikipedia (presented at ECIR 2013)
By  Mihai Georgescu, Nattiya Kanhabua, Daniel Krause, Wolfgang Nejdl, and Stefan Siersdorfer (L3S Research Center)

Wikipedia is widely considered the largest and most up-to date online encyclopedia, with its content being continuously maintained by a supporting community. In many cases, real-life events like new scientific findings, resignations, deaths, or catastrophes serve as triggers for collaborative editing of articles about a ffected entities such as persons or countries. In this paper, we conduct an in-depth analysis of event-related updates in Wikipedia by examining di fferent indicators for events including language, meta annotations, and update bursts. We then study how these indicators can be employed for automatically detecting event related updates. Our experiments on event extraction, clustering, and summarization show promising results towards generating entity-speci fic news tickers and timelines.
More

 

Recommending High Utility Query via Session-Flow Graph (presented at ECIR 2013)
By  Xiaofei Zhu,Jiafeng Guo, Xueqi Cheng, Yanyan Lan (Institute of Computing Technology, Chinese Academy of Sciences, BeiJing, China, and Wolfgang Nejdl (L3S Research Center)

Query recommendation is an integral part of modern search engines that helps users find their information needs. Traditional query recommendation methods usually focus on recommending users relevant queries, which attempt to find alternative queries with close search intent to the original query. Whereas the ultimate goal of query recommendation is to assist users to accomplish their search task successfully, while not just find relevant queries in spite of they can sometimes return useful search results. To better achieve the ultimate goal of query recommendation, a more reasonable way is to recommend users high utility queries, i.e., queries that can return more useful information. In this paper, we propose a novel utility query recommendation approach based on absorbing random walk on the session-flow graph, which can learn queries’ utility by simultaneously modeling both users’ reformulation behaviors and click behaviors. Extensively experiments were conducted on real query logs, and the results show that our method significantly outperforms the state-of-the-art methods under the evaluation metric QRR and MRD.
More

 

Mining People's Appearances To Improve Recognition in Photo Collections (presented at MMM2013, 19th International Conference on Multimedia Modelling) , published by Springer in Advances in Multimedia Modeling, Lecture Notes in Computer Science , Volume 7732, 2013, pp 185-195
By  M.Brenner and E.Izquierdo (Queen Mary University of London)

We show how to recognize people in Consumer Photo Collections by employing a graphical model together with a distance-based face description method. To further improve recognition performance, we incorporate context in the form of social semantics. We devise an approach that has a data mining technique at its core to discover and incorporate patterns of groups of people frequently appearing together in photos. We demonstrate the e ect of our probabilistic approach through experiments on a dataset that spans nearly ten years.
More

 

Unifying rating-oriented and ranking-oriented collaborative filtering for improved recommendation (published on Information Sciences- Elsevier Volume 229 2013, , Pages 29–39 )
By  Y.Shi, M.Larson and A.Hanjalic, (Delft University of Technology, The Netherlands)

We propose a novel unified recommendation model, URM, which combines a rating-oriented collaborative filtering (CF) approach, i.e., probabilistic matrix factorization (PMF), and a ranking-oriented CF approach, i.e., list-wise learning-to-rank with matrix factorization (ListRank). The URM benefits from the rating-oriented perspective and the ranking-oriented perspective by sharing common latent features of users and items in PMF and ListRank. We present an efficient learning algorithm to solve the optimization problem for URM. The computational complexity of the algorithm is shown to be scalable, i.e., to be linear with the number of observed ratings in a given user-item rating matrix. The experimental evaluation is conducted on three public datasets with different scales, allowing validation of the scalability of the proposed URM. Our experiments show the proposed URM significantly outperforms other state-of-the-art recommendation approaches across different datasets and different conditions of user profiles. We also demonstrate that the primary contribution to improve recommendation performance is contributed by the ranking-oriented component, while the rating-oriented component is responsible for a significant enhancement .
More

 

Generating Visual Summaries of Geographic Areas Using Community-Contributed Images (published on IEEE Transactions on Multimedia, vol.15, no.4 2013, pp.921,932 )
By  S. Rudinac, A. Hanjalic, M.Larson (Delft University of Technology, The Netherlands)

In this paper, we present a novel approach for automatic visual summarization of a geographic area that exploits user-contributed images and related explicit and implicit metadata collected from popular content-sharing websites. By means of this approach, we search for a limited number of representative but diverse images to represent the area within a certain radius around a specific location. Our approach is based on the random walk with restarts over a graph that models relations between images, visual features extracted from them, associated text, as well as the information on the uploader and commentators. In addition to introducing a novel edge weighting mechanism, we propose in this paper a simple but effective scheme for selecting the most representative and diverse set of images based on the information derived from the graph. We also present a novel evaluation protocol, which does not require input of human annotators, but only exploits the geographical coordinates accompanying the images in order to reflect conditions on image sets that must necessarily be fulfilled in order for users to find them representative and diverse. Experiments performed on a collection of Flickr images, captured around 207 locations in Paris, demonstrate the effectiveness of our approach .

 

Top-k diversity queries over bounded regions   (published on ACM Transactions on Database Systems (TODS)  2013, article N.10 Volume 38 Issue 2
By  M.Tagliasacchi, I. Catallo, E. Ciceri, P. Fraternali and D. Martinenghi (Politecnico di Miano, Italy)

Top-k diversity queries over objects embedded in a low-dimensional vector space aim to retrieve the best k objects that are both relevant to given user's criteria and well distributed over a designated region. An interesting case is provided by spatial Web objects, which are produced in great quantity by location-based services that let users attach content to places and are found also in domains like trip planning, news analysis, and real estate. In this article we present a technique for addressing such queries that, unlike existing methods for diversified top-k queries, does not require accessing and scanning all relevant objects in order to find the best k results. Our Space Partitioning and Probing (SPP) algorithm works by progressively exploring the vector space, while keeping track of the already seen objects and of their relevance and position. The goal is to provide a good quality result set in terms of both relevance and diversity. We assess quality by using as a baseline the result set computed by MMR, one of the most popular diversification algorithms, while minimizing the number of accessed objects. In order to do so, SPP exploits score-based and distance-based access methods, which are available, for instance, in most geo-referenced Web data sources. Experiments with both synthetic and real data show that SPP produces results that are relevant and spatially well distributed, while significantly reducing the number of accessed objects and incurring a very low computational overhead .
More

 

Fashion-focused Creative Commons Social dataset (presented at ACM Multimedia Systems Conference MMSys 2013)
By  M. Menendez (University of Trento),  D. Martinenghi and L. Galli (Politecnico di Milano), C. Massari (Innovation Engineering), I. Altingovde and M. Georgescu (L3S Research Center/University of Hannover), M. Larson, M. Melenhorst, B. Loni and R. Vliegendhart (Delft University of Technology)

In this work, we present a fashion-focused Creative Commons dataset, which is designed to contain a mix of general images as well as a large component of images that are focused on fashion (i.e., relevant to particular clothing items or fashion accessories) The dataset contains 4810 images and related metadata. Furthermore, a ground truth on image’s tags is presented. Ground truth generation for large-scale datasets is a necessary but expensive task. Traditional expert based approaches have become an expensive and non-scalable solution. For this reason, we turn to crowdsourcing techniques in order to collect ground truth labels, in particular we make use of the commercial crowdsourcing platform, Amazon Mechanical Turk (AMT). Two different groups of annotators (i.e., trusted annotators known to the authors and crowdsourcing workers on AMT) participated in the ground truth creation. Annotation agreement between the two groups is analyzed. Applications of the dataset in different contexts are discussed. This dataset contributes to research areas such as crowdsourcing for multimedia, multimedia content analysis, and design of systems that can elicit fashion preferences from users.
Presentation Video,

This Dataset is made publicly available to the research community. Data collection is limited to images that users have uploaded to Flickr with a Creative Commons license. The license allows the use of the images by persons other than the photographer.  All licenses require Attribution (i.e., the source of the image be cited when the image is used), which implies that the uploader explicitly requires that the image be associated with his/her online profile.

 

Blip10000: A social Video Dataset containing SPUG Content for Tagging and Retrieval (presented at ACM Multimedia Systems Conference MMSys 2013)
By  S.Schmiedeke and T.Sikora (Technische Universität Berlin, Germany),  Peng Xu, C.Kofler and M.Larson (Delft University of Technology, The Netherlands), I.Ferrané (University of Toulouse, France), M.Eskevich and G.J.F.Jones (Dublin City University, Ireland), Y.Estève (Language and Speech Technology team, LIUM, Le Mans, France) and L.Lamel (Spoken Language Processing Group, LIMSI/Vocapia, France)

The increasing amount of digital multimedia content available is inspiring potential new types of user interaction with video data. Users want to easily find the content by searching and browsing. For this reason, techniques are needed that allow automatic categorisation, searching the content and linking to related information. In this work, we present a dataset that contains comprehensive semi-professional user generated (SPUG) content, including audiovisual content, user-contributed metadata, automatic speech recognition transcripts, automatic shot boundary files, and social information for multiple `social levels'. We describe the principal characteristics of this dataset and present results that have been achieved on different tasks.
Presentation Video

This 
Dataset is made publicly available to the research community. Data collection is limited to images that users have uploaded to Flickr with a Creative Commons license. The license allows the use of the images by persons other than the photographer.  All licenses require Attribution (i.e., the source of the image be cited when the image is used), which implies that the uploader explicitly requires that the image be associated with his/her online profile.

 

Cache-Based Query Processing for Search Engines (article for ACM Transactions on the Web Volume 6 Issue 4, November 2012)
By B. Barla Cambazoglu, Ismail Sengor Altingovde, Rifat Ozcan, Özgür Ulusoy (L3S Research Center/University of Hannover)

In practice, a search engine may fail to serve a query due to various reasons such as hardware/network failures, excessive query load, lack of matching documents, or service contract limitations (e.g., the query rate limits for third-party users of a search service). In this kind of scenarios, where the backend search system is unable to generate answers to queries, approximate answers can be generated by exploiting the previously computed query results available in the result cache of the search engine. In this work, we propose two alternative strategies to implement this cache-based query processing idea. The first strategy aggregates the results of similar queries that are previously cached in order to create synthetic results for new queries. The second strategy forms an inverted index over the textual information (i.e., query terms and result snippets) present in the result cache and uses this index to answer new queries. Both approaches achieve reasonable result qualities compared to processing queries with an inverted index built on the collection.
More

 

Can Social Features Help Learning to Rank YouTube Videos (presented at WISE 2012)
By Sergiu Chelaru, Claudia Orellana-Rodriguez, Ismail Sengor Altingovde (L3S Research Center/University of Hannover)

We investigate the impact of social features (such as likes, dislikes, comments, etc.) on the effectiveness of video retrieval in YouTube video sharing system using state-of-the-art learning to rank approaches and a greedy feature selection algorithm. > Our experiments based on a dataset of 3,500 annotated query-video pairs reveal that social features are promising to improve the retrieval performance.
More

 

LikeLines: Collecting Timecode-level Feedback for Web Videos through User Interactions (presented at ACM MM12)
By Raynor Vliegendhart, Martha Larson, Alan Hanjalic (Delft University of Technology)

Conventional online video players do not make the inner structure of the video apparent, making it hard to jump straight to the interesting parts. Our LikeLines system provides its users with a navigable heat map of interesting regions for the videos they are watching. Its novelty lies in its combination of content analysis and both explicit and implicit user interactions. The system can be readily used and deployed to collect large amounts of interaction data needed for in-depth research on timecode-level feedback.
More , Beta Version, Demonstrator Video, Poster

 

Map to Humans and Reduce Error –Crowdsourcing for Deduplication Applied to Digital Libraries (short paper poster session, IR Track,  CIKM 2012)
By Mihai Georgescu, Dang Duc Pham, Claudiu S. Firan, Wolfgang Nejdl, Julien Gaugaz (L3S Research Center/University of Hannover)

Detecting duplicate entities, usually by examining metadata, has been the focus of much recent work. Several methods try to identify duplicate entities, while focusing either on accuracy or on efficiency and speed - with still no perfect solution. We propose a combined layered approach for duplicate detection with the main advantage of using Crowdsourcing as a training and feedback mechanism. By using Active Learning techniques on human provided examples, we fine tune our algorithm toward better duplicate detection accuracy. We keep the training cost low by gathering training data on demand for borderline cases or for inconclusive assessments. We apply our simple and powerful methods to an online publication search system: First, we perform a coarse duplicate detection relying on publication signatures in real time. Then, a second automatic step compares duplicate candidates and increases accuracy while adjusting based on both feedback from our online users and from Crowdsourcing platforms. Our approach shows an improvement of 14% over the untrained setting  and is at only 4% difference to the human assessors in accuracy. 
PosterMore

 

Characterizing and Handling Queries with Very Few or No Results (presented at CIKM 2012)
By Ismail Sengor Altingovde (L3S Research Center/University of Hannover), Roi Blanco and B. Barla Cambazoglu (Yahoo Research); Rifat Ozcan, Erdem Sarigil and Ozgur Ulusoy (Bilkent University)

Despite the continuous efforts to improve the web search quality, a non-negligible fraction of user queries end up with very few or even no matching results in leading web search engines. In this work, we provide a detailed characterization of such queries based on an analysis of a real-life query log. Our experimental setup allows us to characterize the queries with few/no results and compare the mechanisms employed by the major search engines in handling them.
More

 

Efficient Jaccard-based Diversity Analysis of Large Document Collection (presented at CIKM 2012)
By Fan Deng, Stefan Siersdorfer , Sergej Zerr (L3S Research Center/University of Hannover)

We propose two efficient algorithms for exploring topic diversity in large document corpora such as user generated content on the social web, bibliographic data, or other web repositories. Analyzing diversity is useful for obtaining insights into knowledge evolution, trends, periodicities, and topic heterogeneity of such collections. Calculating diversity statistics requires averaging over the similarity of all object pairs, which, for large corpora, is prohibitive from a computational point of view. Our proposed algorithms overcome the quadratic complexity of the average pair-wise similarity computation, and allow for constant time (depending on dataset properties) or linear time approximation with probabilistic guarantees. Our theoretical findings are verified on synthetic and real-world data sets. As applications, we show examples of diversity-based studies on large samples from corpora such as the social photo sharing site Flickr, the DBLP bibliography, and US Census data.
More, Presentation slides

 

PicAlert!: A System for Privacy-Aware Image Classification and Retrieval (Demo session, IR Track, presented at CIKM 2012)
By Sergej Zerr,Stefan Siersdorfer, Jonathon Hare (L3S Research Center/University of Hannover)

Photo publishing in Social Networks and other Web2.0 applications has become very popular due to the pervasive availability of cheap digital cameras, powerful batch upload tools and a huge amount of storage space. A portion of uploaded images are of a highly sensitive nature, disclosing many details of the users' private life. We have developed a web service which can detect private images within a user's photo stream and provide support in making privacy decisions in the sharing context. In addition, we present a privacy-oriented image search application which automatically identifies potentially sensitive images in the result set and separates them from the remaining pictures.
PosterMore

 

One of These thing is not like the other: Crowdsourcing semantic similarity of Multimedia Files(presented at ICT Open 2012)
By R.Vliegendhart , M.Larson and J.Powelse (Delft University of Technology, The Netherlands)

Problem: What constitutes a near duplicate? For example: Are these two files the same? Why (not)?
More

 

Taking the Pulse of Political Emotions in Latin America Based on Social Web Streams (presented at LA-WEB 2012)
By Diaz-Aviles, Claudia Orellana-Rodriguez, Wolfgang Nejdl (L3S Research Center/University of Hannover)

Social media services have become increasingly popular and their penetration is worldwide. Micro-blogging services, such as Twitter, allow users to express themselves, share their emotions and discuss their daily life affairs in real-time, covering a variety of different points of view and opinions, including political and event-related topics such as immigration, economic issues, tax policy or election campaigns. On the other hand, traditional methods tracking public opinion still heavily rely upon opinion polls, which are usually limited to small sample sizes and can incur in significant costs in terms of time and money. In this paper, we leverage state-of-the-art techniques of sentiment analysis for real-time political emotion tracking. In particular, we analyze mentions of personal names of 18 presidents in Latin America, and measure each political figure’s effect in the emotions reflected on the social web.
More

 

Representativeness and Diversity in Photos via Crowd-Sourced Media Analysis (presented at AMR 2012)
By Anca-Livia Radu, Julian Stöttinger, Bogdan Ionescu, María Menéndez and Fausto Giunchiglia (University of Trento)

In this paper we address the problem of user-adapted image retrieval. First, we provide a survey of the performance of the existing social media retrieval platforms and highlight their limitations. In this context, we propose a hybrid, two step, machine and human automated media analysis approach. It aims to improve retrieval relevance by selecting a small number of representative and diverse images from a noisy set of candidate images (e.g. the case of Internet media). In the machine analysis step, to ensure representativeness, images are re-ranked according to the similarity to the "most common" image in the set. Further, to ensure also the diversity of the results, images are clustered and the best ranked images among the most representative in each cluster are retained. The human analysis step aims to bridge further inherent descriptor semantic gap. The retained images are further re ned via crowd-sourcing which adapts the results to human. The method was validated in the context of the retrieval of images with monuments using a data set of more than 30.000 images retrieved from various social image search platforms.
More

 

Impact of Regionalization on Performance of Web Search Engine Result Caches (presented at SPIRE 2012)
By B. Barla Cambazoglu (Yahoo Research), Ismail Sengor Altingovde (L3S Research Center/University of Hannover)

Large-scale web search engines are known to maintain caches that store the results of previously issued queries. They are also known to customize their search results in different forms to improve the relevance of their results to a particular group of users. In this paper, we show that the regionalization of search results decreases the hit rates attained by a result cache. As a remedy, we investigate result prefetching strategies that aim to recover the hit rate sacri ced to search result regionalization. Our results indicate that prefetching achieves a reasonable increase in the result cache hit rate under regionalization of search results.
More

 

Life Logging practice for Human behaviour modelling  (presented at IEEE SMC 2012)
By Pil Ho Kim and F.Giunchiglia (DISI, University of Trento)

With the fast progress of mobile computing, personal lifelogging gets easier taking a little effort for automation. However data-recalling services to help a user locate or remind the event in her lifelogs still lacks supports in major aspects due to the difficulty in creating a useful environment for users. This is because a collection of lifelog streams needs high-level behavior analysis techniques to create the meaningful summary of person’s daily life. The challenging problem is that such knowledge depends on the types of data proprietarily created by the sensors where a complex knowledge on human activity patterns is not easy to apply for real-life event modeling and detection. This paper addresses many aspects of related lifelog research topics and technical challenges. We also studied the way to first handle disparate sensor data and showed the case study to put them on the table for lifelog data analysis and exploration.
More

 

Achievement Systems Explained (presented at SGSC 2012)
By Piero Fraternali, Luca Galli (Politecnico di Milano)

In today’s gaming world, the word “Achievements”, even if rooted in the gaming history, has become extremely popular. The spread of broadband connections and the introduction of multiplayer interactions as core components of a videogame have brought to life to a number of social platforms like Xbox Live!, Playstation Network, Steam and Kongregate, in which the players can track their progress along different game titles and compete among each others. Unfortunately, even if such platforms share similar features, the way in which they manage the aspects of user profiling and statistic tracking is different, leaving the architectural and development aspects of an achievement system tied to the implementation of each vendor. This paper provides an insight on Achievements, their purposes and the way in which they have evolved. A taxonomy of possible achievements is shown along with a set of guidelines to be followed when developing them. Finally a model that can be used to describe all the existing systems is introduced in order to try to put the basis for an open platform capable of integrating existing gaming communities.
More , Presentation Slides, Presentation Video

 

QMUL @ MediaEval 2012: Social Event Detection in Collaborative Photo Collections (presented at MediaEval 2012)
By Markus Brenner, Ebroul Izquierdo (Queen Mary University of London)

We present an approach to detect social events and retrieve associated photos in collaboratively annotated photo collections as part of the MediaEval 2012 Benchmark. We combine data of various modalities from annotated photos as well as from external data sources within a framework that has a classification model at its core. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.
More

 

Search and Hyperlinking Task at MediaEval 2012 (presented at MediaEval 2012)
By M. Eskevich, G. J.F. Jones, S. Chen (Dublin City University, Ireland), R.Aly, R.Ordelman (University of Twente, The Netherlands) and M. Larson (Delft University of Technology, The Netherlands)

The Search and Hyperlinking Task was one of the Brave New Tasks at MediaEval 2012. The Task consisted of two subtasks which focused on search and linking in retrieval from a collection of semi-professional video content. These tasks followed up on research carried out within the MediaEval 2011 Rich Speech Retrieval (RSR) Task and the VideoCLEF 2009 Linking Task.
More

 

Crowdsourcing Social Fashion Images: a Pilot Study (poster presented at MediaEval 2012)
By M.Menéndez (University of Trento, Italy); B.Loni,R.Vliegendhart, M.Melenhorstand M.Larson(Delft University of Technology, The Netherlands; C.Massari (Innovation Engineering,Italy; D.Martinenghi,L.Galli,M.Tagliasacchi and P.Fraternali (Politecnico di Milano, Italy)

As part of the requirements from industrial and technical partners, a collection of fashion items needs to be gathered for further use in the vertical domains. This data can be used for several tasks related to social fashion such as:- Multimedia Content Analysis tasks including: recognizing different types of fashion items and predicting user appeal of fashion images -“Multimedia crowdsourcing” tasks: Algorithms for combining many noisy human decisions into a single decision.
Poster

 

Modeling End-Users as Contributors in Human Computation Applications (presented at MEDI 2012)
By Roula Karam, Piero Fraternali, Alessandro Bozzon and Luca Galli (Politecnico di Milano)

User models have been defined since the ’80s, mainly for the purpose of building context-based, user-adaptive applications. However, the advent of social networked media, serious games, and crowdsourcing platforms calls for a more pervasive notion of user model, capable of representing the multiple facets of a social user, including his social ties, capabilities, activity history, and topical affinities. In this paper, we overview several user models proposed recently to address the platform-independent representation of users embedded in a social context, and discuss the features of the CUbRIK user model, which is designed to support multi-platform human computation applications where users are called as collaborators in the resolution of complex tasks found in the multimedia information retrieval field.
More

 

Indexing Large Online Multimedia Repositories Using Semantic Expansion and Visual Analysis (article for IEEE Multimedia vol 19 Issue 3 page 53-61)
By Sevillano Xavier (La Salle—Universitat Ramon Lull); Piatrik Tomas, Chandramouli Krishna, Zhang Qianni and Izquierdo Ebroul (Queen Mary University of London)

The proposed framework automatically predicts user tags for online videos from their visual features and associated textual metadata, which is semantically expanded using complementary textual resources.
More

 

Context-based people recognition in consumer photo collections (presented at SSMS 2012)
By Markus Brenner (Queen Mary University of London)

Resolve identities of people primarily by their faces Incorporate rich contextual cues of personal photo collections where few individual people frequently appear together Perform recognition by considering all contextual information at the same time (unlike traditional approaches that usually train a classifier and then predict identities independently.
More, Poster

 

The CUbRIK Project Human-enhanced time-aware multimedia search (presented at SSMS 2012)
By Ilio Catallo, Eleonora Ciceri (Politecnico di Milano)

Identifying occurrences of trademark logos in a video collection through keyword-based queries.Special case of the classic problem of object recognition.
More

 

SAM: A tool for the semi-automatic mapping and enrichment of ontologies (presented at OnToContent 2012)
By Vincenzo Maltese, Bayzid Ashik Hossain (University of Trento)

Ontologies are fundamental tools used with different purposes and with different modalities in different areas and communities. To guarantee the right level of quality, the most widely used ontologies are man-made. However, developing and maintaining them turns out to be extremely time-consuming. For this reason, there are approaches aiming at their automatic construction where ontologies are incrementally extended by extracting and integrating knowledge from existing sources. However, these approaches tend to reach an accuracy that, according to the application they need to serve, cannot be always considered satisfactory. Therefore, when a higher accuracy is necessary, manual or semi-automatic approaches are still preferable. In this paper we present a technique and a corresponding tool, that we called SAM (semi-automatic mapper), for the semi-automatic enrichment of an ontology through the mapping of an external source to the target ontology. As proved by our evaluation, the tool allows saving around 50% of the time required by purely manual approaches.
More

 

Swarming to Rank for Recommender Systems (presented at RecSys 2012)
By Ernesto Diaz-Aviles, Mihai Georgescu, Wolfgang Nejdl (L3S Research Center / University of Hannover)

Recommender systems make product suggestions that are tailored to the user’s individual needs and represent powerful means to combat information overload. In this paper, we focus on the item prediction task of Recommender Systems and present SwarmRankCF, a method to automatically optimize the performance quality of recommender systems using a Swarm Intelligence perspective. Our approach, which is well-founded in a Particle Swarm Optimization framework, learns a ranking function by optimizing the combination of unique characteristics (i.e., features) of users, items and their interactions. In particular, we build feature vectors from a factorization of the user-item interaction matrix, and directly optimize Mean Average Precision metric in order to learn a linear ranking model for personalized recommendations. Our experimental evaluation, on a real world online radio dataset, indicates that our approach is able to find ranking functions that significantly improve the performance of the system for the Top-N recommendation task.
More, Poster

 

A Draw-and-Guess Game to Segment Images (presented at SoHuman 2012)
By Luca Galli, Piero Fraternali, Davide Martinenghi, Marco Tagliasacchi (Politecnico di Milano) and Jasminko Novak (European Institut for Participatory Media)

Human Computation is defined as the integration of human tasks and automated algorithms to achieve superior quality in complex tasks like multimedia content analysis. This paper discusses a scenario in which human computation is used to segment timestamped fashion images for mining trends based on visual features of garments (e.g., color and texture) and attributes of portrayed subjects (e.g., gender and age). State-of-the-art algorithms for body part detection and feature extraction can produce low quality results when parts of the body are occluded and when dealing with complex human poses. In such cases, these algorithms could benefit from the assistance of human agents. In order to jointly leverage the potential of crowds and image analysis algorithms, a game with a purpose (GWAP) is proposed, whereby players can help segment images for which specialized algorithms have failed, so as to improve the extraction of color and texture features of garments and their association with the features of the subject wearing themMore
Presentation Slides, Demonstrator

 

Diversification for Multi-domain Result Sets (presented at ICWE 2012)
By Alessandro Bozzon, Marco Brambilla, Piero Fraternali and Marco Tagliasacchi (Politecnico di Milano)

Multi-domain search answers to queries spanning multiple entities, like "Find a hotel in Milan, close to a concert venue, a museum and a good restaurant" by producing ranked sets of entity combinations that maximize relevance, measured by a function expressing the user's preferences. Due to the combinatorial nature of results, good entity instances (e.g., five stars hotels) tend to appear repeatedly in top-ranked combinations. To improve the quality of the result set, it is important to balance relevance with diversity, which promotes different, yet almost equally relevant, entities in the top-k combinations. This paper explores two different notions of diversity for multi-domain result sets, compares experimentally alternative algorithms for the trade-off between relevance and diversity, and performs a user study for evaluating the utility of diversification in multi-domain queries.
More

 

Lifelog Data Model and Management: Study on Research Challenges (article for  IJCISIM Volume 5 (2013) pp.115-125 )
By Pil Ho Kim, Fausto Giunchiglia (University of Trento)

Utilizing a computer to manage an enormous amount of information like lifelogs needs concrete digitized data models on information sources and their connections. For lifelogging, we need to model one’s life in a way that a computer can translate and manage information where many research efforts are still needed to close the gap between real life models and computerized data models. This work studies a fundamental lifelog data modeling method from a digitized information perspective that translates real life events into a composition of digitized and timestamped data streams. It should be noted that a variety of events occurred in one’s real life can’t be fully captured by limited numbers and types of sensors. It is also impractical to ask a user to manually tag entire events and their minute detail relations. Thus we aim to develop the lifelog management system architecture and service structures for people to facilitate mapping a sequence of sensor streams with real life activities. Technically we focus on time series data modeling and management as the first step toward lifelog data fusion and complex event detection.
More

 

Large Scale Sketch Based Image Retrieval Using Patch Hashing (presented at ISVC 2012)
By Konstantinos Bozas and Ebroul Izquierdo (Queen Mary University of London)

This paper introduces a hashing based framework that facilitates sketch based image retrieval in large image databases. Instead of exporting a single visual descriptor for every image, an overlapping spatial grid is utilised to generate a pool of patches. We rank similarities between a hand drawn sketch and the natural images in a database through a voting process where near duplicate in terms of shape and structure patches arbitrate for the result. Patch similarity is efficiently estimated with a hashing algorithm. A reverse index structure built on the hashing keys ensures the scalability of our scheme and at the same time allows for real time reranking on query updates. Experiments in a publicly available benchmark dataset demonstrate the superiority of our approach.
More

 

Efficient Diversification of Top-k Queries over Bounded Regions (presented at SEBD 2012)
By Piero Fraternali, Davide Martinenghi, and Marco Tagliasacchi (Politecnico di Milano)

This paper reports on recent findings regarding diversity queries over objects embedded in a low-dimensional vector space. Among the many contexts of interest, we mention spatial Web objects, which are abundant in location-based services that let users attach content to places. Typical queries aim at retrieving the best set of relevant objects that are well distributed over a region of interest. Existing methods for answering diversified top-k queries are too costly, as they evaluate diversity by accessing and scanning all relevant objects, even if only a small subset thereof is needed. Our proposal, named SPP, is an algorithm that, while finding exactly the same result as MMR (one of the most popular diversification algorithms), does not require retrieving all the relevant objects and, indeed, minimizes the number of accessed objects. Experiments confirm that SPP saves a significant amount of accesses while incurring a very low computational overhead.
More

 

(Unseen) Event Recognition via Semantic Compositionality (presented at CVPR 2012)
By Julian Stöttinger,Jasper R. R. Uijlings, Anand K. Pandey, Nicu Sebe and Fausto Giunchiglia (University of Trento)

Since high-level events in images (e.g. “dinner”, “motorcycle stunt”) are not directly correlated with their visual appearance, low-level visual features do not carry enough semantics to classify such events satisfactorily. This paper explores a fully compositional approach for event based image retrieval which is able to overcome this shortcoming. Furthermore, the approach is fully scalable in both adding new events and new primitives. Using the Pascal VOC 2007 dataset, our contributions are the following: (i) We apply the Faceted Analysis-Synthesis Theory (FAST) to build a hierarchy of 228 high-level events. (ii) We show that rulebased classifiers are better suited for compositional recognition of events than SVMs. In addition, rule-based classifiers provide semantically meaningful event descriptions which help bridging the semantic gap. (iii) We demonstrate that compositionality enables unseen event recognition: we can use rules learned from non-visual cues, together with object detectors to get reasonable performance on unseen event categories.
More

 

Social Event Detection and Retrieval in Collaborative Photo Collection (presented at ICMR 2012)
By Markus Brenner and Ebroul Izquierdo (Queen Mary University of London)

In this paper, we present an approach to detect social events and retrieve associated photos in collaboratively annotated photo collections. We combine data of various modalities such as time, location, and textual and visual features within a framework that has a classification model at its core. Compared to traditional approaches that mainly consider the photos only as a source of information, we also incorporate external information from datasets and online web services to further improve the performance. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.
More

 

Indexing Media by Personal Events (presented at ICMR 2012)
By Javier Paniagua, Ivan Tankoyeu, Julian Stöttinger; Fausto Giunchiglia (University of Trento)

We are addressing the problem of organizing and indexing one’s personal media. Recent approaches of media indexing use events as media aggregators, but do not fully consider the context in which the media asset has been produced and do not take the personal perspective of the user into account. To this end, we propose a new paradigm for the automated indexing of social media based on the the notion of personal events. We reveal both personal habits of a user by analyzing the patterns of capturing images in space and time, while we also improve the understanding of photos over the years by learning the user’s personal behavior. Our fully automatic and computationally inexpensive approach outperforms the state of the art in event-based media indexing. Moreover, we aim to push two main ideas to the problem: (1) We automatically assign the events to routine locations and non-routine locations. This gives the basic nature of events. (2) We hierarchically arrange events at non-routine locations until a routine location is reached again and the round trip is complete. This highly coincides with the given ground-truth at large scale experiments on Picasaweb. We provide experimental validation on a data-set crawled from Picasaweb which consists of about 42,000 photos taken by 5 users in a time period of 37 years, outperforming the state-of-the-art significantly.
More

 

Community Management Systems for Social Deliberation and Action (presented at COOP 2012)
By Alessandro Bozzon, Marco Brambilla, Piero Fraternali, Andrea Mauri (Politecnico di Milano); Stefano Butti, Matteo Silva (WebRatio)

In this paper we discuss the use of social media as a platform for constructing Community Management Systems, defined as tools for identifying, profiling and working with communities of interest, for both large scale deliberation and action. We motivate the notion of Community Management System, present its architecture, discuss application scenarios, identify some of the research problems at the base of its foundation, survey relevant results from the research community, and highlight the building blocks that we are constructing to implement and put to work a such system.
More

 

Top-k Bounded Diversification (presented at SIGMOD 2012)
By Piero Fraternali Davide Martinenghi Marco Tagliasacchi (Politecnico di Milano)

This paper investigates diversity queries over objects embedded in a low-dimensional vector space. An interesting case is provided by spatial Web objects, which are produced in great quantity by location-based services that let users attach content to places, and arise also in trip planning, news analysis, and real estate scenarios. The targeted queries aim at retrieving the best set of objects relevant to given user criteria and well distributed over a region of interest. Such queries are a particular case of diversified top-k queries, for which existing methods are too costly, as they evaluate diversity by accessing and scanning all relevant objects, even if only a small subset is needed. We therefore introduce Space Partitioning and Probing (SPP), an algorithm that minimizes the number of accessed objects while finding exactly the same result as MMR, the most popular diversification algorithm. SPP belongs to a family of algorithms that rely only on score-based and distance-based access methods, which are available in most geo-referenced Web data sources, and do not require retrieving all the relevant objects. Experiments show that SPP significantly reduces the number of accessed objects while incurring a very low computational overhead.
More

 

Sparse Color Interest Points for Image Retrieval and Object Categorization (article for IEEE transactions on Image Processing Volume 21, Issue:5 Pages 2681-2692) 
By Julian Stottinger, Nicu Sebe(University of Trento); Theo Gevers (University of Amsterdam and Universitat Autonoma de Barcelona); Allan Hanbury (Vienna University of Technology)

Interest point detection is an important research area in the field of image processing and computer vision. In particular, image retrieval and object categorization heavily rely on interest point detection from which local image descriptors are computed for image matching. In general, interest points are based on luminance, and color has been largely ignored. However, the use of color increases the distinctiveness of interest points. The use of color may therefore provide selective search reducing the total number of interest points used for image matching. This paper proposes color interest points for sparse image representation. To reduce the sensitivity to varying imaging conditions, light-invariant interest points are introduced. Color statistics based on occurrence probability lead to color boosted points, which are obtained through saliency-based feature selection. Furthermore, a principal component analysis-based scale selectionmethod is proposed, which gives a robust scale estimation per interest point. From large-scale experiments, it is shown that the proposed color interest point detector has higher repeatability than a luminance-based one. Furthermore, in the context of image retrieval, a reduced and predictable number of color features show an increase in performance compared to state-of-the-art interest points. Finally, in the context of object recognition, for the Pascal VOC 2007 challenge, our method gives comparable performance to state-of-the-art methods using only a small fraction of the features, reducing the computing time considerably.
More

 

Human-enhanced time-aware multimedia search: the CUbRIK Project (presented at WWW2012)
By co-authors in CUbRIK

The CUbRIK Project is an Integrated Project of the 7th Frame- work Programme that aims at contributing to the multimedia search domain by opening the architecture of multimedia search engines to the integration of open source and third party content annotation and query processing components, and by exploiting the contribution of humans and communities in all the phases of multimedia search, from content processing to query processing and relevance feedback processing. The CUBRIK presentation will showcase the architectural concept and scientific background of the project and demonstrate an initial scenario of human-enhanced content and query processing pipeline.
More

 

Answering Search Queries with CrowdSearcher (presented at WWW2012)
By Alessandro Bozzon, Marco Brambilla, Stefano Ceri (Politecnico di Milano)

Web users are increasingly relying on social interaction to complete and validate the results of their search activities. While search systems are superior machines to get world-wide information, the opinions collected within friends and expert/local communities can ultimately determine our decisions: human curiosity and creativity is often capable of going much beyond the capabilities of search systems in scouting “interesting” results, or suggesting new, unexpected search directions. Such personalized interaction occurs in most times aside of the search systems and processes, possibly instrumented and mediated by a social network; when such interaction is completed and users resort to the use of search systems, they do it through new queries, loosely related to the previous search or to the social interaction. In this paper we propose CrowdSearcher, a novel search paradigm that embodies crowds as first-class sources for the information seeking process. CrowdSearcher aims at filling the gap between generalized search systems, which operate upon world-wide information - including facts and recommendations as crawled and indexed by computerized systems – with social systems, capable of interacting with real people, in real time, to capture their opinions, suggestions, emotions. The technical contribution of this paper is the discussion of a model and architecture for integrating computerized search with human interaction, by showing how search systems can drive and encapsulate social systems. In particular we show how social platforms, such as Facebook, LinkedIn and Twitter, can be used for crowdsourcing search-related tasks; we demonstrate our approach with several prototypes and we report on our experiment upon real user communities.
More

 

CrowdSearch: Crowdsourcing Web search [workshop profile  CrowdSearch 2012]
By Piero Fraternali, Stefano Ceri (Politecnico di Milano); Ricardo Baeza-Yates (Yahoo! Research Spain); Fausto Giunchiglia (University of Trento)

Link analysis, that has shaped Web search technology in the last decade, can be seen as a massive mining of crowd- secured reputation associated with pages. With the exponential increase of social engagement, link analysis is now complemented by other kinds of crowd-generated information, such as multimedia content, recommendations, tweets and tags, and each person can ask for information or advices from dedicated sites. With the growth of online presence, we expect questions to be directly routed to informed crowds. At the same time, many kinds of tasks - either directly used for search or indirectly used for enriching content to make it more searchable - are explicitly crowd-sourced, possibly under the format of games. Many such tasks can be used to craft information, e.g. by naming and tagging data objects and by solving representational ambiguities and con icts, thereby enhancing the scope of searchable objects. Thus, social engagement is empowering and reshaping the search of Web information.
More

 

Discovering user perceptions of semantic similarity in near-duplicate multimedia files (presented at CrowdSearch 2012)
By R. Vliegendhart, M. Larson, and J. Pouwelse (Delft University of Technology)

We address the problem of discovering new notions of user perceived similarity between near-duplicate multimedia files. We focus on file-sharing, since in this setting, users have a well-developed understanding of the available content, but what constitutes a near-duplicate is nonetheless nontrivial. We elicited judgments of semantic similarity by implementing triadic elicitation as a crowdsourcing task and ran it on Amazon Mechanical Turk. We categorized the judgments and arrived at 44 different dimensions of semantic similarity perceived by users. These discovered dimensions can be used for clustering items in search result lists. The challenge in performing elicitations in this way is to ensure that workers are encouraged to answer seriously and remain engaged.
More

 

A Model Driven Approach for Crowdsourcing Search (presented at CrowdSearch 2012)
By Alessandro Bozzon, Marco Brambilla, Andrea Mauri (Politecnico di Milano)

Even though search systems are very efficient in retrieving world-wide information, they can not capture some peculiar aspects and features of user needs, such as subjective opinions and recommendations, or information that require local or domain specific expertise. In this kind of scenario, the human opinion provided by an expert or knowledgeable user can be more useful than any factual information retrieved by a search engine. In this paper we propose a model-driven approach for the specification of crowd-search tasks, i.e. activities where real people - in real time - take part to the generalized search process that involve search engines. In particular we define two models: the "Query TaskModel", representing the meta- model of the query that is submitted to the crowd and the associated answers; and the "User Interaction Model", which shows how the user can interact with the query model to fulfill her needs. Our solution allows for a top-down design approach, from the crowd-search task design, down to the crowd answering system design. Our approach also grants automatic code generation thus leading to quick prototyping of search applications based on human responses collected over social networking or crowdsourcing platforms.
More

 

A Framework for Crowdsourced Multimedia Processing and Querying (presented at CrowdSearch 2012)
By Alessandro Bozzon, Ilio Catallo, Eleonora Ciceri, Piero Fraternali, Davide Martinenghi, Marco Tagliasacchi (Politecnico di Milano)

This paper introduces a conceptual and architectural framework for addressing the design, execution and verification of tasks by a crowd of performers. The proposed framework is substantiated by an ongoing application to a problem of trademark logo detection in video collections. Preliminary results show that the contribution of crowds can improve the recall of state-of-the-art traditional algorithms, with no loss in terms of precision. However, task-to-executor matching, as expected, has an important influence on the task performance.
More

 

On the Size of Full Element-Indexes for XML Keyword Search (presented at ECIR 2012)
By Duygu Atilgan (Bilkent University, Ankara), Ismail Sengor Altingovde (L3S Research Center, University of Hannover), Özgür Ulusoy (Bilkent University, Ankara)

We show that a full element-index can be as space-efficient as a direct index with Dewey ids, after compression using typical techniques
More

 

In Praise of Laziness: A Lazy Strategy for Web Information Extraction (presented at ECIR 2012)
By Rifat Ozcan(Bilkent University, Ankara), Ismail Sengor Altingovde (L3S Research Center, University of Hannover), Özgür Ulusoy (Bilkent University, Ankara)

A large number of Web information extraction algorithms are based on machine learning techniques. For such extraction algorithms, we propose employing a lazy learning strategy to build a specialized model for each test instance to improve the extraction accuracy and avoid the disadvantages of constructing a single general model.
More

 

Social and behavioural media access  (presented at SBNMA 2011)
By Naeem Ramzan(Queen Mary University of London), Peng Cui, Fei Wang, Shiqiang Yang

Social media has received substantial attention in last five years due to its focal point on analysis and relationships among persons and on the models and repercussions of these relationships. In the meantime, with the fast augment in production and distribution of multimedia content, effectively integrating context and content for multimedia mining, management, indexing and retrieval on the Internet has become an evident and difficult problem. The interest in these areas is demonstrated by a significantly increasing number of publications each year. In this paper, we give an overview of the key theoretical and empirical advances in the current decade related to social media retrieval by considering not only the multimedia content analysis but also the behavioural analysis of the users. We also discuss the significant challenges involved in the adaptation of existing multimedia content analysis techniques for interactive content sharing in social and P2P networks.
More

 

Search Computing: Business Areas, Research and Socio-Economic Challenge (presented at Search Computing and Social Media workshop 2011)
By co-authors in CUbRIK and CHORUS+

Search has become an important and necessary component of many diverse ICT applications. A large number of business and application areas such as: a) the Web, b) mobile devices and applications, c) social networks and social media, and d) enterprise data access and organization, depend on the efficiency and availability of search techniques being able to process and retrieve heterogeneous and dispersed data. Such techniques are directly related to a number of research topics and challenges ranging from multimodal analysis and indexing to affective computing and human aspects as well as to various socio-economic challenges including business models, open innovation, legal and ethical issues. The area covering all these issues is also known as “Search Computing”. The objective of this document is to provide an overview of the business areas, the research challenges and the socio-economic aspects related to “Search Computing”.
More

A CUbRIK paper presented at SGSC 2012

Achievement Systems Explained

By: Luca Galli, Piero Fraternali(Politecnico di Milano)

Abstract: In today’s gaming world, the word “Achievements”, even if rooted in the gaming history, has become extremely popular. The spread of broadband connections and the introduction of multiplayer interactions as core components of a videogame have brought to life to a number of social platforms like Xbox Live!, Playstation Network, Steam and Kongregate, in which the players can track their progress along different game titles and compete among each others. Unfortunately, even if such platforms share similar features, the way in which they manage the aspects of user profiling and statistic tracking is different, leaving the architectural and development aspects of an achievement system tied to the implementation of each vendor. This paper provides an insight on Achievements, their purposes and the way in which they have evolved. A taxonomy of possible achievements is shown along with a set of guidelines to be followed when developing them. Finally a model that can be used to describe all the existing systems is introduced in order to try to put the basis for an open platform capable of integrating existing gaming communities.

Download Link:

Video:

CUbRIK Scientific Publications and Articles next to appear

CUbRIK has promoted visibility of its research activity with many scientific publications in the course of its 1st Year of operation, additional ones are next to appear as listed below:

  • 2012, October (presented at Medi 2012)
    Modeling End-Users as Contributors in Human Computation Applications

    By Roula Karam, Piero Fraternali, Alessandro Bozzon and Luca Galli (Politecnico di Milano)

  • 2012, October (presented at SGSC 2012)
    Achievement Systems Explained

    By Piero Fraternali, Luca Galli (Politecnico di Milano

  • 2012, October (presented at SPIRE 2012)
    Impact of Regionalization on Performance of Web Search Engine Result Caches

    By B. Barla Cambazoglu (Yahoo Research), Ismail Sengor Altingovde (L3S Research Center/University of Hannover)

  • 2012, October (to be presented at AMR 2012)
    Representativeness and Diversity in Photos via Crowd-Sourced Media Analysis

    By Anca-Livia Radu, Julian Stöttinger, Bogdan Ionescu, María Menéndez and Fausto Giunchiglia (University of Trento)

  • 2012, October (to be presented at LA-WEB 2012)
    Taking the Pulse of Political Emotions in Latin America Based on Social Web Streams

    By Diaz-Aviles, Claudia Orellana-Rodriguez, Wolfgang Nejdl (L3S Research Center/University of Hannover)

  • 2012, November (to be presented at CIKM 2012)
    Map to Humans and Reduce Error –Crowdsourcing for Deduplication Applied to Digital Libraries
    By Mihai Georgescu, Dang Duc Pham, Claudiu S. Firan, Wolfgang Nejdl, Julien Gaugaz (L3S Research Center/University of Hannover)
    Characterizing and Handling Queries with Very Few or No Results
    By Ismail Sengor Altingovde (L3S Research Center/University of Hannover), Roi Blanco and B. Barla Cambazoglu (Yahoo Research); Rifat Ozcan, Erdem Sarigil and Ozgur Ulusoy (Bilkent University)
    Efficient Jaccard-based Diversity Analysis of Large Document Collection
    By Fan Deng, Stefan Siersdorfer , Sergej Zerr (L3S Research Center/University of Hannover)
    PicAlert!: A System for Privacy-Aware Image Classification and Retrieval

    By Sergej Zerr,Stefan Siersdorfer, Jonathon Hare (L3S Research Center/University of Hannover)

  • 2012, November (to be presented at WISE 2012)
    Can Social Features Help Learning to Rank YouTube Videos

    By Sergiu Chelaru, Claudia Orellana-Rodriguez, Ismail Sengor Altingovde (L3S Research Center/University of Hannover)

  • for publication on IEEE Multimedia vol 19 Issue 3 page 53-61,
    Indexing Large Online Multimedia Repositories Using Semantic Expansion and Visual Analysis

    By Sevillano Xavier (La Salle—Universitat Ramon Lull); Piatrik Tomas, Chandramouli Krishna, Zhang Qianni and Izquierdo Ebroul (Queen Mary University of London)

  • for publication on TWEB 2012, ACM Transactions on the Web
    Cache-Based Query Processing for Search Engines

    By B. Barla Cambazoglu (Yahoo Research), I.S. Altingövde(L3S Research Center/University of Hannover) , R. Ozcan and Ö. Ulusoy (Bilkent University)

A CUbRIK Paper presented at Medi 2012

Modeling End-Users as Contributors in Human Computation Applications

By: Roula Karam, Piero Fraternali, Alessandro Bozzon and Luca Galli (Politecnico di Milano)

Abstract: User models have been de ned since the '80s, mainly for the purpose of building context-based, user-adaptive applications. However, the advent of social networked media, serious games, and crowdsourcing platforms calls for a more pervasive notion of user model, capable of representing the multiple facets of a social user, including his social ties, capabilities, activity history, and topical anities. In this paper, we overview several user models proposed recently to address the platform independent representation of users embedded in a social context, and discuss the features of the CUbRIK user model, which is designed to support multi-platform human computation applications where users are called as collaborators in the resolution of complex tasks found in the multimedia information retrieval fi eld.

Download Link: proceedings of the 2nd International Conference on Model and Data Engineering, MEDI 2012, held in Poitiers, France, in October 2012

CUbRIK Year 1: Scientific Publications and Articles 
  • 2012, September (presented at SSMS 2012)
    Context-based people recognition in consumer photo collections

    By Markus Brenner (Queen Mary University of London) 
    The CUbRIK Project Human-enhanced time-aware multimedia search
    By Ilio Catallo, Eleonora Ciceri (Politecnico di Milano)


  • 2012, September (presented at OnToContent 2012)
    SAM: A tool for the semi-automatic mapping and enrichment of ontologies

    By Vincenzo Maltese, Bayzid Ashik Hossain (University of Trento)


  • 2012, September (presented at RecSys 2012)
    Swarming to Rank for Recommender Systems

    By Ernesto Diaz-Aviles, Mihai Georgescu, Wolfgang Nejdl (L3S Research Center / University of Hannover)


  • 2012, September (presented at SoHuman 2012)
    A Draw-and-Guess Game to Segment Images

    By Luca Galli, Piero Fraternali, Davide Martinenghi, Marco Tagliasacchi (Politecnico di Milano) and Jasminko Novak (European Institut for Participatory Media)


  • 2012, July (presented at ICWE 2012)
    Diversification for Multi-domain Result Sets

    By Alessandro Bozzon, Marco Brambilla, Piero Fraternali and Marco Tagliasacchi (Politecnico di Milano)


  • 2012, July (article on IJCISIM)
    Lifelog Data Model and Management: Study on Research Challenges

    By Pil Ho Kim, Fausto Giunchiglia (University of Trento)


  • 2012, July (presented at ISVC 2012)
    Large Scale Sketch Based Image Retrieval Using Patch Hashing

    By Konstantinos Bozas and Ebroul Izquierdo (Queen Mary University of London)


  • 2012, June (presented at CVCR 2012)
    (Unseen) Event Recognition via Semantic Compositionality

    By Julian Stöttinger,Jasper R. R. Uijlings, Anand K. Pandey, Nicu Sebe and Fausto Giunchiglia (University of Trento)


  • 2012, June (presented at ICMR 2012)
    Social Event Detection and Retrieval in Collaborative Photo Collection

    By Markus Brenner and Ebroul Izquierdo (Queen Mary University of London)
    Social Indexing Media by Personal Events

    By Javier Paniagua, Ivan Tankoyeu, Julian Stöttinger; Fausto Giunchiglia (University of Trento)


  • 2012, May (presented at COOP 2012)
    Community Management Systems for Social Deliberation and Action

    By Alessandro Bozzon, Marco Brambilla, Piero Fraternali, Andrea Mauri (Politecnico di Milano); Stefano Butti, Matteo Silva (WebRatio)


  • 2012, May (presented at SIGMOD 2012)
    Top-k Bounded Diversification

    By Piero Fraternali Davide Martinenghi Marco Tagliasacchi (Politecnico di Milano)


  • 2012, May (article on IEEE Transactions on Image Processing)
    Sparse Color Interest Points for Image Retrieval and Object Categorization

    By Julian Stottinger, Nicu Sebe(University of Trento); Theo Gevers (University of Amsterdam and Universitat Autonoma de Barcelona); Allan Hanbury (Vienna University of Technology)


  • 2012, April (presented at WWW 2012)
    Human-enhanced time-aware multimedia search: the CUbRIK Project

    By co-authors in CUbRIK
    Answering Search Queries with CrowdSearcher

    By Alessandro Bozzon, Marco Brambilla, Stefano Ceri (Politecnico di Milano)


  • 2012, April (presented at CrowdSearch 2012)
    CrowdSearch: Crowdsourcing Web search [A WWW 2012 Workshop]

    By Piero Fraternali, Stefano Ceri (Politecnico di Milano); Ricardo Baeza-Yates (Yahoo! Research Spain); Fausto Giunchiglia (University of Trento)
    Discovering user perceptions of semantic similarity in near-duplicate multimedia files

    By R. Vliegendhart, M. Larson, and J. Pouwelse (Delft University of Technology)
    A Model Driven Approach for Crowdsourcing Search

    By Alessandro Bozzon, Marco Brambilla, Andrea Mauri (Politecnico di Milano)
    A Framework for Crowdsourced Multimedia Processing and Querying

    By Alessandro Bozzon, Ilio Catallo, Eleonora Ciceri, Piero Fraternali, Davide Martinenghi, Marco Tagliasacchi (Politecnico di Milano)


  • 2012, April (presented at ECIR 2012)
    On the Size of Full Element-Indexes for XML Keyword Search

    By Duygu Atilgan (Bilkent University, Ankara), Ismail Sengor Altingovde (L3S Research Center, University of Hannover), Özgür Ulusoy (Bilkent University, Ankara)
    In Praise of Laziness: A Lazy Strategy for Web Information Extraction

    By Rifat Ozcan(Bilkent University, Ankara), Ismail Sengor Altingovde (L3S Research Center, University of Hannover), Özgür Ulusoy (Bilkent University, Ankara)


  • 2011, December (presented at SBNMA 2011)
    Social and behavioural media access

    By Naeem Ramzan(Queen Mary University of London), Peng Cui, Fei Wang, Shiqiang Yang


  • 2011, September 
    Search Computing: Business Areas, Research and Socio-Economic Challenge

    White Paper, by co-authors in CUbRIK and CHORUS+


More Articles...

Page 1 of 5

<< Start < Prev 1 2 3 4 5 Next > End >>