Downloads 
Publications and Journals
Publications and Journals
CUbRIK Scientific Digest
Generating Contextualized Sentiment Lexica based on Latent Topics and User Ratings (long research paper presented at Hipertext 2013)
By Ralf Krestel, Stefan Siersdorfer (L3S Research Center)
Sentiment lexica are useful for analyzing opinions in Web collections, for domain-dependent sentiment classification, and as sub-components of recommender systems. In this paper, we present a strategy for automatically generating topic-dependent lexica from large corpora of review articles by exploiting accompanying user ratings. Our approach combines text segmentation, discriminative feature analysis techniques, and latent topic extraction to infer the polarity of n-grams in a topical context. Our experiments on rating prediction demonstrate a substantial performance improvement in comparison with existing state-of-the-art sentiment lexica.
More
More
Reactive Crowdsourcing (full research paper presented at WWW 2013)
By Alessandro Bozzon, Marco Brambilla,Stefano Ceri and Andrea Mauri (Politecnico di Milano)
An essential aspect for building effective crowdsourcing computations is the ability of "controlling the crowd", i.e. of
dynamically adapting the behaviour of the crowdsourcing
systems as response to the quantity and quality of completed tasks or to the availability and reliability of performers. Most crowdsourcing systems only provide limited and
prede fined controls; in contrast, we present an approach to
crowdsourcing which provides fine-level, powerful and
flexible controls. We model each crowdsourcing application as
composition of elementary task types and we progressively
transform these high level specifi cations into the features of a
reactive execution environment that supports task planning,
assignment and completion as well as performer monitoring
and exclusion. Controls are speci fied as active rules on top
of data structures which are derived from the model of the
application; rules can be added, dropped or modi fied, thus
guaranteeing maximal
flexibility with limited eff ort.
We also report on our prototype platform that implements
the proposed framework and we show the results of our experimentations with di fferent rule sets, demonstrating how
simple changes to the rules can substantially a ffect time,
e ffort and quality involved in crowdsourcing activities. More
Mining Emotions in Short Films:
User Comments or Crowdsourcing? (poster presented at WWW 2013)
By Claudia Orellana-Rodriguez, Ernesto Diaz-Aviles, and Wolfgang Nejdl
(L3S Research Center)
Short films are regarded as an alternative form of artistic creation, and they express, in a few minutes, a whole
gamma of diff erent emotions oriented to impact the audience and communicate a story. In this paper, we exploit a
multi-modal sentiment analysis approach to extract emotions in short films, based on the film criticism expressed through social comments from the video-sharing platform
YouTube. We go beyond the traditional polarity detection
(i.e., positive/negative), and extract, for each analyzed film,
four opposing pairs of primary emotions: joy-sadness, anger-fear, trust-disgust, and anticipation-surprise.We found that
YouTube comments are a valuable source of information for
automatic emotion detection when compared to human analysis elicited via crowdsourcing. More
Temporal Summarization of Event-Related Updates
in Wikipedia (demonstrator presented at WWW 2013)
By Mihai Georgescu, Dang Duc Pham, Nattiya Kanhabua
Sergej Zerr, Stefan Siersdorfer, and Wolfgang Nejdl
(L3S Research Center)
Wikipedia is a free multilingual online encyclopedia covering a wide range of general and speci fic knowledge. Its content is continuously maintained up-to-date and extended by
a supporting community. In many cases, real-world events
influence the collaborative editing of Wikipedia articles of
the involved or a ffected entities. In this paper, we present
Wikipedia Event Reporter, a web-based system that supports the entity-centric, temporal analytics of event-related information in Wikipedia by analyzing the whole history of
article updates. For a given entity, the system first identifi es
peaks of update activities for the entity using burst detection and automatically extracts event-related updates using a machine-learning approach. Further, the system determines distinct events through the clustering of updates by exploiting diff erent types of information such as update time,
textual similarity, and the position of the updates within an article. Finally, the system generates the meaningful temporal summarization of event-related updates and automatically annotates the identified events in a timeline. More
The Open Platform for Personal
Lifelogging: The eLifeLog
Architecture (poster presented in Works-in-Progress session at CHI 2013)
By Pil Ho Kim and F.Giunchiglia (DISI, University of Trento)
Lifelogging is a complex application domain of multimedia
management. This makes it challenging for people to
keep their personal lifelogs under their control. Our work
aims to provide people with an open platform, named
eLifeLog, that would work in user's private cloud to start
archiving their valuable memories and experiences under
the hood. eLifeLog has a number of distinct features that
di erentiate it from legacy CMS (Content Management
System) products or related works: (1) It is specialized for
personal lifelogging, (2) it embeds an event-based uni ed
data representation to handle heterogenous timestamped
streams, and (3) it is open to the public with the
complete source code for personal use and for accelerating
lifelogging research collaboration.
Building social graphs from images through
expert-based crowdsourcing (presented at SoHuman 2013)
By M. Dionisio, P. Fraternali, D. Martinenghi,
C. Pasini, M. Tagliasacchi and S. Zagorac, (Politecnico di Milano, Italy), E.Harloff, I. Micheel and J.Novak (European Institute for Participatory Media, Germany)
The extraction of semantic information from multimedia
content represents a challenging problem. Despite the
continuous refinement of automatic tools, the quality and completeness
of the results is not always satisfactory. To overcome
this limitation, the vision of the CUbRIK project is to provide
a multimedia search and exploration platform that seamlessly
integrate human tasks and algorithms. In this paper, as a
concrete example, we illustrate the design of a multimedia content
processing pipeline that aims at extracting evidence of social
relationships from the analysis of a photo collection covering
the main events and people that shaped the history of Europe
after World War II. We discuss the issues faced by generalpurpose
crowdsourcing and automatic face detection/recognition
algorithms in determining the identities of people portrayed in
the photo collection. Hence, we illustrate the design of a system
that tackles the uncertainty of the results produced by face
detection/recognition with expert-based crowdsourcing.
Exploiting User Generated Content for Mountain Peak
Detection (presented at SoHuman 2013)
By R. Fedorov, D.Martinenghi, M. Tagliasacchi and A.Castelletti (Politecnico di Milano, Italy)
We present a system for the classification of mountain
panoramas from user-generated photographs followed by
identification and extraction of mountain peaks from those
panoramas. We have developed an automatic technique that,
given as input a geo-tagged photograph, estimates its FOV
(Field Of View) and the direction of the camera using a
matching algorithm on the photograph edge maps and a rendered
view of the mountain silhouettes that should be seen
from the observer’s point of view. The extraction algorithm
then identifies the mountain peaks present in the photograph
and their profiles. We discuss possible applications in social
fields like photographs peaks tagging on social portals,
augmented reality on mobile devices when viewing a mountain
panorama, and generation of collective intelligence systems
(such as environmental models) from massive social
media collections (e.g. snow water availability maps based
on mountain peaks states extracted from photographs hosting
services).
MSIDX: Multi-Sort Indexing for Efficient Content-based Image Search and Retrieval (published on Multimedia IEEE Transactions Volume PP Issue 99)
By E.Tiakas,
D.Rafailidis, A.Dimou and P.Daras, (Information Technologies Institute, Centre for Research and Technology Hellas)
In this paper, a novel approximate indexing scheme for efficient content-based image search and retrieval is presented, called Multi-Sort Indexing (MSIDX). The proposed scheme analyzes high dimensional image descriptor vectors, by employing the value cardinalities of their dimensions. The dimensions value cardinalities, an inherent characteristic of descriptor vectors, are the number of discrete values in the dimensions. As expected, value cardinalities significantly vary, due to the existence of several extraction methods. Moreover, different quantization and normalization techniques used in the extraction process, have a strong impact on the dimensions value cardinalities. Since dimensions with high value cardinalities have more discriminative power, a multiple sort algorithm is used to reorder the descriptors dimensions according to their value cardinalities, in order to increase the probability of two similar images to lie within a close constant range. The expected bounds of the constant range are defined in detail, following deterministic and probabilistic analyses. The proposed scheme is fully suitable (a) for real-time indexing of images, and (b) for searching and retrieving relevant images with an efficient query processing algorithm. In our experiments with five real datasets, we show the superiority of the proposed approach against hashing methods, also suitable for approximate similarity search.
Extracting Event-Related Information from
Article Updates in Wikipedia (presented at ECIR 2013)
By Mihai Georgescu, Nattiya Kanhabua, Daniel Krause, Wolfgang Nejdl, and
Stefan Siersdorfer (L3S Research Center)
Wikipedia is widely considered the largest and most up-to date
online encyclopedia, with its content being continuously maintained
by a supporting community. In many cases, real-life events like new scientific
findings, resignations, deaths, or catastrophes serve as triggers for
collaborative editing of articles about a ffected entities such as persons or
countries. In this paper, we conduct an in-depth analysis of event-related
updates in Wikipedia by examining di fferent indicators for events including
language, meta annotations, and update bursts. We then study
how these indicators can be employed for automatically detecting event related
updates. Our experiments on event extraction, clustering, and
summarization show promising results towards generating entity-speci fic
news tickers and timelines.
Recommending High Utility Query
via Session-Flow Graph (presented at ECIR 2013)
By Xiaofei Zhu,Jiafeng Guo, Xueqi Cheng, Yanyan Lan (Institute of Computing Technology, Chinese Academy of Sciences, BeiJing, China, and Wolfgang Nejdl (L3S Research Center)
Query recommendation is an integral part of modern search
engines that helps users find their information needs. Traditional query
recommendation methods usually focus on recommending users relevant
queries, which attempt to find alternative queries with close search intent
to the original query. Whereas the ultimate goal of query recommendation
is to assist users to accomplish their search task successfully, while
not just find relevant queries in spite of they can sometimes return useful
search results. To better achieve the ultimate goal of query recommendation,
a more reasonable way is to recommend users high utility queries,
i.e., queries that can return more useful information. In this paper, we
propose a novel utility query recommendation approach based on absorbing
random walk on the session-flow graph, which can learn queries’ utility
by simultaneously modeling both users’ reformulation behaviors and
click behaviors. Extensively experiments were conducted on real query
logs, and the results show that our method significantly outperforms the
state-of-the-art methods under the evaluation metric QRR and MRD.
Mining People's Appearances To Improve
Recognition in Photo Collections (presented at MMM 2013, 19th International Conference on Multimedia Modelling)
By M.Brenner and E.Izquierdo (Queen Mary University of London)
We show how to recognize people in Consumer Photo Col-
lections by employing a graphical model together with a distance-based
face description method. To further improve recognition performance,
we incorporate context in the form of social semantics. We devise an
approach that has a data mining technique at its core to discover and
incorporate patterns of groups of people frequently appearing together in
photos. We demonstrate the e ect of our probabilistic approach through
experiments on a dataset that spans nearly ten years..
Multimedia Indexing, Search and Retrieval in Large Databases of Social Networks (published on
Social Media Retrieval Computer Communications and Networks 2013, pp 43-63)
By T.Semertzidis,
D. Rafailidis, E.Tiakas,M.G. Strintzis and P.Daras (Information Technologies Institute, Centre for Research and Technology Hellas)
Social networks are changing the way multimedia content is shared on the Web, by allowing users to upload their photos, videos, and audio content, produced by any means of digital recorders such as mobile/smartphones and Web/digital cameras. This plethora of content created the need for finding the desired media in the social media universe. Moreover, the diversity of the available content inspired users to demand and formulate more complicated queries. In the social media era, multimedia content search is promoted to a fundamental feature toward efficient search inside social multimedia streams, content classification, and context and event-based indexing. In this chapter, an overview of multimedia indexing and searching algorithms, following the data growth curve, is presented in detail. This chapter is thematically structured in two parts. In the first part, pure multimedia content retrieval issues are presented, while in the second part, the social aspects and new, interesting views on multimedia retrieval in the large social media databases are discussed.
Fashion-focused Creative Commons Social dataset (presented at ACM Multimedia Systems Conference MMSys 2013)
By M. Menendez (University of Trento), D. Martinenghi and L. Galli (Politecnico di Milano), C. Massari (Innovation Engineering), I. Altingovde and M. Georgescu (L3S Research Center/University of Hannover), M. Larson, M. Melenhorst, B. Loni and R. Vliegendhart (Delft University of Technology)
In this work, we present a fashion-focused Creative Commons dataset, which is designed to contain a mix of general images as well as a large component of images that are focused on fashion (i.e., relevant to particular clothing items or fashion accessories) The dataset contains 4810 images and related metadata. Furthermore, a ground truth on image’s tags is presented. Ground truth generation for large-scale datasets is a necessary but expensive task. Traditional expert based approaches have become an expensive and non-scalable solution. For this reason, we turn to crowdsourcing techniques in order to collect ground truth labels, in particular we make use of the commercial crowdsourcing platform, Amazon Mechanical Turk (AMT). Two different groups of annotators (i.e., trusted annotators known to the authors and crowdsourcing workers on AMT) participated in the ground truth creation. Annotation agreement between the two groups is analyzed. Applications of the dataset in different contexts are discussed. This dataset contributes to research areas such as crowdsourcing for multimedia, multimedia content analysis, and design of systems that can elicit fashion preferences from users.
Cache-Based Query Processing for Search Engines (article for ACM Transactions on the Web Volume 6 Issue 4, November 2012)
By B. Barla Cambazoglu, Ismail Sengor Altingovde, Rifat Ozcan, Özgür Ulusoy (L3S Research Center/University of Hannover)
In practice, a search engine may fail to serve a query due to various reasons such as hardware/network failures, excessive query load, lack of matching documents, or service contract limitations (e.g., the query rate limits for third-party users of a search service). In this kind of scenarios, where the backend search system is unable to generate answers to queries, approximate answers can be generated by exploiting the previously computed query results available in the result cache of the search engine. In this work, we propose two alternative strategies to implement this cache-based query processing idea. The first strategy aggregates the results of similar queries that are previously cached in order to create synthetic results for new queries. The second strategy forms an inverted index over the textual information (i.e., query terms and result snippets) present in the result cache and uses this index to answer new queries. Both approaches achieve reasonable result qualities compared to processing queries with an inverted index built on the collection.
More
More
Can Social Features Help Learning to Rank YouTube Videos (presented at WISE 2012)
By Sergiu Chelaru, Claudia Orellana-Rodriguez, Ismail Sengor Altingovde (L3S Research Center/University of Hannover)
We investigate the impact of social features (such as likes, dislikes, comments, etc.) on the effectiveness of video retrieval in YouTube video sharing system using state-of-the-art learning to rank approaches and a greedy feature selection algorithm. > Our experiments based on a dataset of 3,500 annotated query-video pairs reveal that social features are promising to improve the retrieval performance.
More
More
LikeLines: Collecting Timecode-level Feedback for Web Videos through User Interactions (presented at ACM MM12)
By Raynor Vliegendhart, Martha Larson, Alan Hanjalic (Delft University of Technology)
Conventional online video players do not make the inner structure of the video apparent, making it hard to jump straight to the interesting parts. Our LikeLines system provides its users with a navigable heat map of interesting regions for the videos they are watching. Its novelty lies in its combination of content analysis and both explicit and implicit user interactions. The system can be readily used and deployed to collect large amounts of interaction data needed for in-depth research on timecode-level feedback.
More
More
Map to Humans and Reduce Error –Crowdsourcing for Deduplication Applied to Digital Libraries (presented at CIKM 2012)
By Mihai Georgescu, Dang Duc Pham, Claudiu S. Firan, Wolfgang Nejdl, Julien Gaugaz (L3S Research Center/University of Hannover)
Detecting duplicate entities, usually by examining metadata, has been the focus of much recent work. Several methods try to identify duplicate entities, while focusing either on accuracy or on efficiency and speed - with still no perfect solution. We propose a combined layered approach for duplicate detection with the main advantage of using Crowdsourcing as a training and feedback mechanism. By using Active Learning techniques on human provided examples, we fine tune our algorithm toward better duplicate detection accuracy. We keep the training cost low by gathering training data on demand for borderline cases or for inconclusive assessments. We apply our simple and powerful methods to an online publication search system: First, we perform a coarse duplicate detection relying on publication signatures in real time. Then, a second automatic step compares duplicate candidates and increases accuracy while adjusting based on both feedback from our online users and from Crowdsourcing platforms. Our approach shows an improvement of 14% over the untrained setting and is at only 4% difference to the human assessors in accuracy.
More
More
Characterizing and Handling Queries with Very Few or No Results (presented at CIKM 2012)
By Ismail Sengor Altingovde (L3S Research Center/University of Hannover), Roi Blanco and B. Barla Cambazoglu (Yahoo Research); Rifat Ozcan, Erdem Sarigil and Ozgur Ulusoy (Bilkent University)
Despite the continuous efforts to improve the web search quality, a non-negligible fraction of user queries end up with very few or even no matching results in leading web search engines. In this work, we provide a detailed characterization of such queries based on an analysis of a real-life query log. Our experimental setup allows us to characterize the queries with few/no results and compare the mechanisms employed by the major search engines in handling them.
More
More
Efficient Jaccard-based Diversity Analysis of Large Document Collection (presented at CIKM 2012)
By Fan Deng, Stefan Siersdorfer , Sergej Zerr (L3S Research Center/University of Hannover)
We propose two efficient algorithms for exploring topic diversity in large document corpora such as user generated content on the social web, bibliographic data, or other web repositories. Analyzing diversity is useful for obtaining insights into knowledge evolution, trends, periodicities, and topic heterogeneity of such collections. Calculating diversity statistics requires averaging over the similarity of all object pairs, which, for large corpora, is prohibitive from a computational point of view. Our proposed algorithms overcome the quadratic complexity of the average pair-wise similarity computation, and allow for constant time (depending on dataset properties) or linear time approximation with probabilistic guarantees. Our theoretical findings are verified on synthetic and real-world data sets. As applications, we show examples of diversity-based studies on large samples from corpora such as the social photo sharing site Flickr, the DBLP bibliography, and US Census data.
More
More
PicAlert!: A System for Privacy-Aware Image Classification and Retrieval (presented at CIKM 2012)
By Sergej Zerr,Stefan Siersdorfer, Jonathon Hare (L3S Research Center/University of Hannover)
Photo publishing in Social Networks and other Web2.0 applications has become very popular due to the pervasive availability of cheap digital cameras, powerful batch upload tools and a huge amount of storage space. A portion of uploaded images are of a highly sensitive nature, disclosing many details of the users' private life. We have developed a web service which can detect private images within a user's photo stream and provide support in making privacy decisions in the sharing context. In addition, we present a privacy-oriented image search application which automatically identifies potentially sensitive images in the result set and separates them from the remaining pictures.
More
More
Taking the Pulse of Political Emotions in Latin America Based on Social Web Streams (presented at LA-WEB 2012)
By Diaz-Aviles, Claudia Orellana-Rodriguez, Wolfgang Nejdl (L3S Research Center/University of Hannover)
Social media services have become increasingly popular and their penetration is worldwide. Micro-blogging services, such as Twitter, allow users to express themselves, share their emotions and discuss their daily life affairs in real-time, covering a variety of different points of view and opinions, including political and event-related topics such as immigration, economic issues, tax policy or election campaigns. On the other hand, traditional methods tracking public opinion still heavily rely upon opinion polls, which are usually limited to small sample sizes and can incur in significant costs in terms of time and money. In this paper, we leverage state-of-the-art techniques of sentiment analysis for real-time political emotion tracking. In particular, we analyze mentions of personal names of 18 presidents in Latin America, and measure each political figure’s effect in the emotions reflected on the social web.
More
More
Representativeness and Diversity in Photos via Crowd-Sourced Media Analysis (presented at AMR 2012)
By Anca-Livia Radu, Julian Stöttinger, Bogdan Ionescu, María Menéndez and Fausto Giunchiglia (University of Trento)
In this paper we address the problem of user-adapted image retrieval. First, we provide a survey of the performance of the existing social media retrieval platforms and highlight their limitations. In this context, we propose a hybrid, two step, machine and human automated media analysis approach. It aims to improve retrieval relevance by selecting a small number of representative and diverse images from a noisy set of candidate images (e.g. the case of Internet media). In the machine analysis step, to ensure representativeness, images are re-ranked according to the similarity to the "most common" image in the set. Further, to ensure also the diversity of the results, images are clustered and the best ranked images among the most representative in each cluster are retained. The human analysis step aims to bridge further inherent descriptor semantic gap. The retained images are further re ned via crowd-sourcing which adapts the results to human. The method was validated in the context of the retrieval of images with monuments using a data set of more than 30.000 images retrieved from various social image search platforms.
More
More
Impact of Regionalization on Performance of Web Search Engine Result Caches (presented at SPIRE 2012)
By B. Barla Cambazoglu (Yahoo Research), Ismail Sengor Altingovde (L3S Research Center/University of Hannover)
Large-scale web search engines are known to maintain caches that store the results of previously issued queries. They are also known to customize their search results in di erent forms to improve the relevance of their results to a particular group of users. In this paper, we show that the regionalization of search results decreases the hit rates attained by a result cache. As a remedy, we investigate result prefetching strategies that aim to recover the hit rate sacri ced to search result regionalization. Our results indicate that prefetching achieves a reasonable increase in the result cache hit rate under regionalization of search results.
More
More
Life Logging practice for Human behaviour modelling (presented at IEEE SMC 2012)
By Pil Ho Kim and F.Giunchiglia (DISI, University of Trento)
With the fast progress of mobile computing, personal lifelogging gets easier taking a little effort for automation. However data-recalling services to help a user locate or remind the event in her lifelogs still lacks supports in major aspects due to the difficulty in creating a useful environment for users. This is because a collection of lifelog streams needs high-level behavior analysis techniques to create the meaningful summary of person’s daily life. The challenging problem is that such knowledge depends on the types of data proprietarily created by the sensors where a complex knowledge on human activity patterns is not easy to apply for real-life event modeling and detection. This paper addresses many aspects of related lifelog research topics and technical challenges. We also studied the way to first handle disparate sensor data and showed the case study to put them on the table for lifelog data analysis and exploration.
Achievement Systems Explained (presented at SGSC 2012)
By Piero Fraternali, Luca Galli (Politecnico di Milano)
In today’s gaming world, the word “Achievements”, even if rooted in the gaming history, has become extremely popular. The spread of broadband connections and the introduction of multiplayer interactions as core components of a videogame have brought to life to a number of social platforms like Xbox Live!, Playstation Network, Steam and Kongregate, in which the players can track their progress along different game titles and compete among each others. Unfortunately, even if such platforms share similar features, the way in which they manage the aspects of user profiling and statistic tracking is different, leaving the architectural and development aspects of an achievement system tied to the implementation of each vendor. This paper provides an insight on Achievements, their purposes and the way in which they have evolved. A taxonomy of possible achievements is shown along with a set of guidelines to be followed when developing them. Finally a model that can be used to describe all the existing systems is introduced in order to try to put the basis for an open platform capable of integrating existing gaming communities.
More
More
QMUL @ MediaEval 2012: Social Event Detection in Collaborative Photo Collections (presented at MediaEval 2012)
By Markus Brenner, Ebroul Izquierdo (Queen Mary University of London)
We present an approach to detect social events and retrieve associated photos in collaboratively annotated photo collections as part of the MediaEval 2012 Benchmark. We combine data of various modalities from annotated photos as well as from external data sources within a framework that has a classification model at its core. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.
More
More
Modeling End-Users as Contributors in Human Computation Applications (presented at MEDI 2012)
By Roula Karam, Piero Fraternali, Alessandro Bozzon and Luca Galli (Politecnico di Milano)
User models have been defined since the ’80s, mainly for the purpose of building context-based, user-adaptive applications. However, the advent of social networked media, serious games, and crowdsourcing platforms calls for a more pervasive notion of user model, capable of representing the multiple facets of a social user, including his social ties, capabilities, activity history, and topical affinities. In this paper, we overview several user models proposed recently to address the platform-independent representation of users embedded in a social context, and discuss the features of the CUbRIK user model, which is designed to support multi-platform human computation applications where users are called as collaborators in the resolution of complex tasks found in the multimedia information retrieval field.
More
More
Indexing Large Online Multimedia Repositories Using Semantic Expansion and Visual Analysis (article for IEEE Multimedia vol 19 Issue 3 page 53-61)
By Sevillano Xavier (La Salle—Universitat Ramon Lull); Piatrik Tomas, Chandramouli Krishna, Zhang Qianni and Izquierdo Ebroul (Queen Mary University of London)
The proposed framework automatically predicts user tags for online videos from their visual features and associated textual metadata, which is semantically expanded using complementary textual resources.
More
More
Context-based people recognition in consumer photo collections (presented at SSMS 2012)
By Markus Brenner (Queen Mary University of London)
Resolve identities of people primarily by their faces Incorporate rich contextual cues of personal photo collections where few individual people frequently appear together Perform recognition by considering all contextual information at the same time (unlike traditional approaches that usually train a classifier and then predict identities independently.
More
More
The CUbRIK Project Human-enhanced time-aware multimedia search (presented at SSMS 2012)
By Ilio Catallo, Eleonora Ciceri (Politecnico di Milano)
Identifying occurrences of trademark logos in a video collection through keyword-based queries.Special case of the classic problem of object recognition.
More
More
SAM: A tool for the semi-automatic mapping and enrichment of ontologies (presented at OnToContent 2012)
By Vincenzo Maltese, Bayzid Ashik Hossain (University of Trento)
Ontologies are fundamental tools used with different purposes and with different modalities in different areas and communities. To guarantee the right level of quality, the most widely used ontologies are man-made. However, developing and maintaining them turns out to be extremely time-consuming. For this reason, there are approaches aiming at their automatic construction where ontologies are incrementally extended by extracting and integrating knowledge from existing sources. However, these approaches tend to reach an accuracy that, according to the application they need to serve, cannot be always considered satisfactory. Therefore, when a higher accuracy is necessary, manual or semi-automatic approaches are still preferable. In this paper we present a technique and a corresponding tool, that we called SAM (semi-automatic mapper), for the semi-automatic enrichment of an ontology through the mapping of an external source to the target ontology. As proved by our evaluation, the tool allows saving around 50% of the time required by purely manual approaches.
More
More
Swarming to Rank for Recommender Systems (presented at RecSys 2012)
By Ernesto Diaz-Aviles, Mihai Georgescu, Wolfgang Nejdl (L3S Research Center / University of Hannover)
Recommender systems make product suggestions that are tailored to the user’s individual needs and represent powerful means to combat information overload. In this paper, we focus on the item prediction task of Recommender Systems and present SwarmRankCF, a method to automatically optimize the performance quality of recommender systems using a Swarm Intelligence perspective. Our approach, which is well-founded in a Particle Swarm Optimization framework, learns a ranking function by optimizing the combination of unique characteristics (i.e., features) of users, items and their interactions. In particular, we build feature vectors from a factorization of the user-item interaction matrix, and directly optimize Mean Average Precision metric in order to learn a linear ranking model for personalized recommendations. Our experimental evaluation, on a real world online radio dataset, indicates that our approach is able to find ranking functions that significantly improve the performance of the system for the Top-N recommendation task.
More
More
A Draw-and-Guess Game to Segment Images (presented at SoHuman 2012)
By Luca Galli, Piero Fraternali, Davide Martinenghi, Marco Tagliasacchi (Politecnico di Milano) and Jasminko Novak (European Institut for Participatory Media)
Human Computation is defined as the integration of human tasks and automated algorithms to achieve superior quality in complex tasks like multimedia content analysis. This paper discusses a scenario in which human computation is used to segment timestamped fashion images for mining trends based on visual features of garments (e.g., color and texture) and attributes of portrayed subjects (e.g., gender and age). State-of-the-art algorithms for body part detection and feature extraction can produce low quality results when parts of the body are occluded and when dealing with complex human poses. In such cases, these algorithms could benefit from the assistance of human agents. In order to jointly leverage the potential of crowds and image analysis algorithms, a game with a purpose (GWAP) is proposed, whereby players can help segment images for which specialized algorithms have failed, so as to improve the extraction of color and texture features of garments and their association with the features of the subject wearing them.
More
More
Diversification for Multi-domain Result Sets (presented at ICWE 2012)
By Alessandro Bozzon, Marco Brambilla, Piero Fraternali and Marco Tagliasacchi (Politecnico di Milano)
Multi-domain search answers to queries spanning multiple entities, like "Find a hotel in Milan, close to a concert venue, a museum and a good restaurant" by producing ranked sets of entity combinations that maximize relevance, measured by a function expressing the user's preferences. Due to the combinatorial nature of results, good entity instances (e.g., five stars hotels) tend to appear repeatedly in top-ranked combinations. To improve the quality of the result set, it is important to balance relevance with diversity, which promotes different, yet almost equally relevant, entities in the top-k combinations. This paper explores two different notions of diversity for multi-domain result sets, compares experimentally alternative algorithms for the trade-off between relevance and diversity, and performs a user study for evaluating the utility of diversification in multi-domain queries.
More
More
Lifelog Data Model and Management: Study on Research Challenges (article for IJCISIM Volume 5 (2013) pp.115-125 )
By Pil Ho Kim, Fausto Giunchiglia (University of Trento)
Utilizing a computer to manage an enormous amount of information like lifelogs needs concrete digitized data models on information sources and their connections. For lifelogging, we need to model one’s life in a way that a computer can translate and manage information where many research efforts are still needed to close the gap between real life models and computerized data models. This work studies a fundamental lifelog data modeling method from a digitized information perspective that translates real life events into a composition of digitized and timestamped data streams. It should be noted that a variety of events occurred in one’s real life can’t be fully captured by limited numbers and types of sensors. It is also impractical to ask a user to manually tag entire events and their minute detail relations. Thus we aim to develop the lifelog management system architecture and service structures for people to facilitate mapping a sequence of sensor streams with real life activities. Technically we focus on time series data modeling and management as the first step toward lifelog data fusion and complex event detection.
More
More
Large Scale Sketch Based Image Retrieval Using Patch Hashing (presented at ISVC 2012)
By Konstantinos Bozas and Ebroul Izquierdo (Queen Mary University of London)
This paper introduces a hashing based framework that facilitates sketch based image retrieval in large image databases. Instead of exporting a single visual descriptor for every image, an overlapping spatial grid is utilised to generate a pool of patches. We rank similarities between a hand drawn sketch and the natural images in a database through a voting process where near duplicate in terms of shape and structure patches arbitrate for the result. Patch similarity is efficiently estimated with a hashing algorithm. A reverse index structure built on the hashing keys ensures the scalability of our scheme and at the same time allows for real time reranking on query updates. Experiments in a publicly available benchmark dataset demonstrate the superiority of our approach.
More
More
Efficient Diversification of Top-k Queries over Bounded Regions (presented at SEBD 2012)
By Piero Fraternali, Davide Martinenghi, and Marco Tagliasacchi (Politecnico di Milano)
This paper reports on recent findings regarding diversity queries over objects embedded in a low-dimensional vector space. Among the many contexts of interest, we mention spatial Web objects, which are abundant in location-based services that let users attach content to places. Typical queries aim at retrieving the best set of relevant objects that are well distributed over a region of interest. Existing methods for answering diversified top-k queries are too costly, as they evaluate diversity by accessing and scanning all relevant objects, even if only a small subset thereof is needed. Our proposal, named SPP, is an algorithm that, while finding exactly the same result as MMR (one of the most popular diversification algorithms), does not require retrieving all the relevant objects and, indeed, minimizes the number of accessed objects. Experiments confirm that SPP saves a significant amount of accesses while incurring a very low computational overhead.
More
More
(Unseen) Event Recognition via Semantic Compositionality (presented at CVPR 2012)
By Julian Stöttinger,Jasper R. R. Uijlings, Anand K. Pandey, Nicu Sebe and Fausto Giunchiglia (University of Trento)
Since high-level events in images (e.g. “dinner”, “motorcycle stunt”) are not directly correlated with their visual appearance, low-level visual features do not carry enough semantics to classify such events satisfactorily. This paper explores a fully compositional approach for event based image retrieval which is able to overcome this shortcoming. Furthermore, the approach is fully scalable in both adding new events and new primitives. Using the Pascal VOC 2007 dataset, our contributions are the following: (i) We apply the Faceted Analysis-Synthesis Theory (FAST) to build a hierarchy of 228 high-level events. (ii) We show that rulebased classifiers are better suited for compositional recognition of events than SVMs. In addition, rule-based classifiers provide semantically meaningful event descriptions which help bridging the semantic gap. (iii) We demonstrate that compositionality enables unseen event recognition: we can use rules learned from non-visual cues, together with object detectors to get reasonable performance on unseen event categories.
More
More
Social Event Detection and Retrieval in Collaborative Photo Collection (presented at ICMR 2012)
By Markus Brenner and Ebroul Izquierdo (Queen Mary University of London)
In this paper, we present an approach to detect social events and retrieve associated photos in collaboratively annotated photo collections. We combine data of various modalities such as time, location, and textual and visual features within a framework that has a classification model at its core. Compared to traditional approaches that mainly consider the photos only as a source of information, we also incorporate external information from datasets and online web services to further improve the performance. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.
More
More
Indexing Media by Personal Events (presented at ICMR 2012)
By Javier Paniagua, Ivan Tankoyeu, Julian Stöttinger; Fausto Giunchiglia (University of Trento)
We are addressing the problem of organizing and indexing one’s personal media. Recent approaches of media indexing use events as media aggregators, but do not fully consider the context in which the media asset has been produced and do not take the personal perspective of the user into account. To this end, we propose a new paradigm for the automated indexing of social media based on the the notion of personal events. We reveal both personal habits of a user by analyzing the patterns of capturing images in space and time, while we also improve the understanding of photos over the years by learning the user’s personal behavior. Our fully automatic and computationally inexpensive approach outperforms the state of the art in event-based media indexing. Moreover, we aim to push two main ideas to the problem: (1) We automatically assign the events to routine locations and non-routine locations. This gives the basic nature of events. (2) We hierarchically arrange events at non-routine locations until a routine location is reached again and the round trip is complete. This highly coincides with the given ground-truth at large scale experiments on Picasaweb. We provide experimental validation on a data-set crawled from Picasaweb which consists of about 42,000 photos taken by 5 users in a time period of 37 years, outperforming the state-of-the-art significantly.
More
More
Community Management Systems for Social Deliberation and Action (presented at COOP 2012)
By Alessandro Bozzon, Marco Brambilla, Piero Fraternali, Andrea Mauri (Politecnico di Milano); Stefano Butti, Matteo Silva (WebRatio)
In this paper we discuss the use of social media as a platform for constructing Community Management Systems, defined as tools for identifying, profiling and working with communities of interest, for both large scale deliberation and action. We motivate the notion of Community Management System, present its architecture, discuss application scenarios, identify some of the research problems at the base of its foundation, survey relevant results from the research community, and highlight the building blocks that we are constructing to implement and put to work a such system.
More
More
Top-k Bounded Diversification (presented at SIGMOD 2012)
By Piero Fraternali Davide Martinenghi Marco Tagliasacchi (Politecnico di Milano)
This paper investigates diversity queries over objects embedded in a low-dimensional vector space. An interesting case is provided by spatial Web objects, which are produced in great quantity by location-based services that let users attach content to places, and arise also in trip planning, news analysis, and real estate scenarios. The targeted queries aim at retrieving the best set of objects relevant to given user criteria and well distributed over a region of interest. Such queries are a particular case of diversified top-k queries, for which existing methods are too costly, as they evaluate diversity by accessing and scanning all relevant objects, even if only a small subset is needed. We therefore introduce Space Partitioning and Probing (SPP), an algorithm that minimizes the number of accessed objects while finding exactly the same result as MMR, the most popular diversification algorithm. SPP belongs to a family of algorithms that rely only on score-based and distance-based access methods, which are available in most geo-referenced Web data sources, and do not require retrieving all the relevant objects. Experiments show that SPP significantly reduces the number of accessed objects while incurring a very low computational overhead.
More
More
Sparse Color Interest Points for Image Retrieval and Object Categorization (article for IEEE transactions on Image Processing Volume 21, Issue:5 Pages 2681-2692)
By Julian Stottinger, Nicu Sebe(University of Trento); Theo Gevers (University of Amsterdam and Universitat Autonoma de Barcelona); Allan Hanbury (Vienna University of Technology)
Interest point detection is an important research area in the field of image processing and computer vision. In particular, image retrieval and object categorization heavily rely on interest point detection from which local image descriptors are computed for image matching. In general, interest points are based on luminance, and color has been largely ignored. However, the use of color increases the distinctiveness of interest points. The use of color may therefore provide selective search reducing the total number of interest points used for image matching. This paper proposes color interest points for sparse image representation. To reduce the sensitivity to varying imaging conditions, light-invariant interest points are introduced. Color statistics based on occurrence probability lead to color boosted points, which are obtained through saliency-based feature selection. Furthermore, a principal component analysis-based scale selectionmethod is proposed, which gives a robust scale estimation per interest point. From large-scale experiments, it is shown that the proposed color interest point detector has higher repeatability than a luminance-based one. Furthermore, in the context of image retrieval, a reduced and predictable number of color features show an increase in performance compared to state-of-the-art interest points. Finally, in the context of object recognition, for the Pascal VOC 2007 challenge, our method gives comparable performance to state-of-the-artmethods using only a small fraction of the features, reducing the computing time considerably.
More
More
Human-enhanced time-aware multimedia search: the CUbRIK Project (presented at WWW2012)
By co-authors in CUbRIK
The CUbRIK Project is an Integrated Project of the 7th Frame- work Programme that aims at contributing to the multimedia search domain by opening the architecture of multimedia search engines to the integration of open source and third party content annotation and query processing components, and by exploiting the contribution of humans and communities in all the phases of multimedia search, from content processing to query processing and relevance feedback processing. The CUBRIK presentation will showcase the architectural concept and scientific background of the project and demonstrate an initial scenario of human-enhanced content and query processing pipeline.
More
More
Answering Search Queries with CrowdSearcher (presented at WWW2012)
By Alessandro Bozzon, Marco Brambilla, Stefano Ceri (Politecnico di Milano)
Web users are increasingly relying on social interaction to complete and validate the results of their search activities. While search systems are superior machines to get world-wide information, the opinions collected within friends and expert/local communities can ultimately determine our decisions: human curiosity and creativity is often capable of going much beyond the capabilities of search systems in scouting “interesting” results, or suggesting new, unexpected search directions. Such personalized interaction occurs in most times aside of the search systems and processes, possibly instrumented and mediated by a social network; when such interaction is completed and users resort to the use of search systems, they do it through new queries, loosely related to the previous search or to the social interaction. In this paper we propose CrowdSearcher, a novel search paradigm that embodies crowds as first-class sources for the information seeking process. CrowdSearcher aims at filling the gap between generalized search systems, which operate upon world-wide information - including facts and recommendations as crawled and indexed by computerized systems – with social systems, capable of interacting with real people, in real time, to capture their opinions, suggestions, emotions. The technical contribution of this paper is the discussion of a model and architecture for integrating computerized search with human interaction, by showing how search systems can drive and encapsulate social systems. In particular we show how social platforms, such as Facebook, LinkedIn and Twitter, can be used for crowdsourcing search-related tasks; we demonstrate our approach with several prototypes and we report on our experiment upon real user communities.
More
More
CrowdSearch: Crowdsourcing Web search [workshop profile CrowdSearch 2012]
By Piero Fraternali, Stefano Ceri (Politecnico di Milano); Ricardo Baeza-Yates (Yahoo! Research Spain); Fausto Giunchiglia (University of Trento)
Link analysis, that has shaped Web search technology in the last decade, can be seen as a massive mining of crowd- secured reputation associated with pages. With the exponential increase of social engagement, link analysis is now complemented by other kinds of crowd-generated information, such as multimedia content, recommendations, tweets and tags, and each person can ask for information or advices from dedicated sites. With the growth of online presence, we expect questions to be directly routed to informed crowds. At the same time, many kinds of tasks - either directly used for search or indirectly used for enriching content to make it more searchable - are explicitly crowd-sourced, possibly under the format of games. Many such tasks can be used to craft information, e.g. by naming and tagging data objects and by solving representational ambiguities and con icts, thereby enhancing the scope of searchable objects. Thus, social engagement is empowering and reshaping the search of Web information.
More
More
Discovering user perceptions of semantic similarity in near-duplicate multimedia files (presented at CrowdSearch 2012)
By R. Vliegendhart, M. Larson, and J. Pouwelse (Delft University of Technology)
We address the problem of discovering new notions of userperceived similarity between near-duplicate multimedia files. We focus on file-sharing, since in this setting, users have a well-developed understanding of the available content, but what constitutes a near-duplicate is nonetheless nontrivial. We elicited judgments of semantic similarity by implementing triadic elicitation as a crowdsourcing task and ran it on Amazon Mechanical Turk. We categorized the judgments and arrived at 44 different dimensions of semantic similarity perceived by users. These discovered dimensions can be used for clustering items in search result lists. The challenge in performing elicitations in this way is to ensure that workers are encouraged to answer seriously and remain engaged.
More
More
A Model Driven Approach for Crowdsourcing Search (presented at CrowdSearch 2012)
By Alessandro Bozzon, Marco Brambilla, Andrea Mauri (Politecnico di Milano)
Even though search systems are very efficient in retrieving world-wide information, they can not capture some peculiar aspects and features of user needs, such as subjective opinions and recommendations, or information that require local or domain specific expertise. In this kind of scenario, the human opinion provided by an expert or knowledgeable user can be more useful than any factual information retrieved by a search engine. In this paper we propose a model-driven approach for the specification of crowd-search tasks, i.e. activities where real people - in real time - take part to the generalized search process that involve search engines. In particular we define two models: the "Query TaskModel", representing the meta- model of the query that is submitted to the crowd and the associated answers; and the "User Interaction Model", which shows how the user can interact with the query model to fulfill her needs. Our solution allows for a top-down design approach, from the crowd-search task design, down to the crowd answering system design. Our approach also grants automatic code generation thus leading to quick prototyping of search applications based on human responses collected over social networking or crowdsourcing platforms.
More
More
A Framework for Crowdsourced Multimedia Processing and Querying (presented at CrowdSearch 2012)
By Alessandro Bozzon, Ilio Catallo, Eleonora Ciceri, Piero Fraternali, Davide Martinenghi, Marco Tagliasacchi (Politecnico di Milano)
This paper introduces a conceptual and architectural framework for addressing the design, execution and verification of tasks by a crowd of performers. The proposed framework is substantiated by an ongoing application to a problem of trademark logo detection in video collections. Preliminary results show that the contribution of crowds can improve the recall of state-of-the-art traditional algorithms, with no loss in terms of precision. However, task-to-executor matching, as expected, has an important influence on the task performance.
More
More
On the Size of Full Element-Indexes for XML Keyword Search (presented at ECIR 2012)
By Duygu Atilgan (Bilkent University, Ankara), Ismail Sengor Altingovde (L3S Research Center, University of Hannover), Özgür Ulusoy (Bilkent University, Ankara)
We show that a full element-index can be as space-efficient as a direct index with Dewey ids, after compression using typical techniques
More
More
In Praise of Laziness: A Lazy Strategy for Web Information Extraction (presented at ECIR 2012)
By Rifat Ozcan(Bilkent University, Ankara), Ismail Sengor Altingovde (L3S Research Center, University of Hannover), Özgür Ulusoy (Bilkent University, Ankara)
A large number of Web information extraction algorithms are based on machine learning techniques. For such extraction algorithms, we propose employing a lazy learning strategy to build a specialized model for each test instance to improve the extraction accuracy and avoid the disadvantages of constructing a single general model.
More
More
Social and behavioural media access (presented at SBNMA 2011)
By Naeem Ramzan(Queen Mary University of London), Peng Cui, Fei Wang, Shiqiang Yang
Social media has received substantial attention in last five years due to its focal point on analysis and relationships among persons and on the models and repercussions of these relationships. In the meantime, with the fast augment in production and distribution of multimedia content, effectively integrating context and content for multimedia mining, management, indexing and retrieval on the Internet has become an evident and difficult problem. The interest in these areas is demonstrated by a significantly increasing number of publications each year. In this paper, we give an overview of the key theoretical and empirical advances in the current decade related to social media retrieval by considering not only the multimedia content analysis but also the behavioural analysis of the users. We also discuss the significant challenges involved in the adaptation of existing multimedia content analysis techniques for interactive content sharing in social and P2P networks.
More
More
Search Computing: Business Areas, Research and Socio-Economic Challenge (presented at Search Computing and Social Media workshop 2011)
By co-authors in CUbRIK and CHORUS+
Search has become an important and necessary component of many diverse ICT applications. A large number of business and application areas such as: a) the Web, b) mobile devices and applications, c) social networks and social media, and d) enterprise data access and organization, depend on the efficiency and availability of search techniques being able to process and retrieve heterogeneous and dispersed data. Such techniques are directly related to a number of research topics and challenges ranging from multimodal analysis and indexing to affective computing and human aspects as well as to various socio-economic challenges including business models, open innovation, legal and ethical issues. The area covering all these issues is also known as “Search Computing”. The objective of this document is to provide an overview of the business areas, the research challenges and the socio-economic aspects related to “Search Computing”.
More
More
Last Updated on 29 May 2013
More Articles...
The CUbRIK project has received research funding from the European Union 
Back to Top