Multimedia search processing done in CUbRIK (across query content analysis and relevance feedback processing) benefit from:
CUbRIK incorporates human and social capabilities
CUbRIK works on enhancing the quality of search experience by augmenting the precision and the relevance of results when machine intelligence fails or is unable to remove uncertainty. The approach is not to emulate, but rather to incorporate human and social capabilities from feature extraction to search and validation of multimedia content: because we, the humans, still have a lot to teach machines about semantic understanding of multimedia content!
CUbRIK search-based applications are able to mix Crowd-based, GWAP-basedand pure-machine computation thanks to pipelines that describe programmable workflow of tasks allocated to executors (e.g. software components for data analysis, metadata indexing, search engines, presentation of results, or others) and able to combine different types of intelligence, according to the specific application requirements.
The mechanics of this relies on two specific Frameworks, part of the CUbRIK platform architecture:
CUbRIK for you
CUbRIK has proven the feasibility of its approach and the benefits of the integration of machine, human and social computation for multimedia search by showcasing reference ASSETS for the exploitation of its technologies.
METHODOLOGY assets: guidelines about how to embed GWAP, Gamification and Crowdsourcing
FRAMEWORKS assets: software infrastructures, ready to use for combining conventional computation with the power of the crowd-based processing, enablers of the development of this new class of multimedia search applications
COMPONENTS, ALGORITHMS, SERVICES and LIBRARIES: technologies used and developed in CUbRIK, reference functionalities that can be invoked in pipelines of this new class of multimedia search applications.
SERVICES and LIBRARIES
The inspiring principle of CUbRIK is about creating a “white-box” version of a multimedia content & query processing system.
The functionalities of query processing, content analysis and relevance feedback processing are unbundled into a set of search processing orchestrations (referred to as Pipelinesable to mix open source and third-party components, to instantiate algorithms and to aggregate Automatic Computation Jobs (automatic workflow or SMILA Workflow) and Human Computation (Crowd-enabled or GWAP-enabled)Tasks.
Metadata are extracted from media collections, using the software mix that best fits the need, and specific components can be included to process multimodal queries or to analyze user’s feedback in novel ways.
The architecture is an example of differential design: based on a SMILA underlying framework, modified to support programmable workflows and asynchronous task execution as it is required to mix automatic operations (Jobs) and human activities (Tasks) chained in a sequence. CUbRIK inherits and extends SMILA capabilities of easy integration of data source connectors, search engines, sophisticated analysis methods and other components by gaining scalability and reliability out-of-the-box.