|Day 1: Tutorials|
|Summary: 2 half-day Tutorials (each half-day is 3.5 hours)|
|09:00||Stefan Rüger||Multimedia information retrieval|
|11:00||Stefan Rüger||Multimedia information retrieval|
|14:00||Julio Gonzalo||Evaluation Metrics for Information Access|
|16:00||Julio Gonzalo||Evaluation Metrics for Information Access|
Day 2: Keynote, Paper Presentations & Competive Demo
|09:30||Keynote: Ralf Steinberger||Multilingual and cross-lingual news analysis in the Europe Media Monitor (EMM)|
|Session 1 - Semantic Search and Tools|
|11:15||Jagdev Bhogal and Andrew Macfarlane||Ontology Based Query Expansion with a Probabilistic Retrieval Model|
|11:45||Niraj Shrestha, Ivan Vulic and Marie-Francine Moens||An IR-Inspired Approach to Recovering Named Entity Tags in Broadcast News|
|12:15||Jose M Chenlo and David E Losada||A Machine Learning approach for Subjectivity Classification based on Positional and Discourse Features|
|MUMIA Interdisciplinary WG Meeting (14:00-15:20)|
|Stefanos Vrochidis||MULTISENSOR - Mining and Understanding of multilinguaL contenT for Intelligent Sentiment Enriched coNtext and Social Oriented inteRpretation|
|Sebastien Ferre||Scalable Query-based Faceted Search on top of SPARQL Endpoints for Guided and Expressive Semantic Search|
|Laurentiu Vasiliu||Decision Making in the Big Financial Data Age: an EU Start-up Perspective|
|Yaakov HaCohen-Kerner||Language Resources for Hebrew - Overview, Updates and Developments|
|MUMIA Management Committee Meeting (15:30-17:00)|
|17:00||End of Day 2|
Day 3: Paper Presentations & MUMIA Events
|Session 2 - Users and Search Tasks|
|09:30||Nils Pharo and Ragnar Nordlie||Using 'search transitions' to study searchers' investment of effort: experiences with client and server side logging|
|10:00||Fernando Loizides and George Buchanan||Towards a Framework for Human (Manual) Information Retrieval|
|10:30||Bissan Audeh, Philippe Beaune and Michel Beigbeder||Recall-Oriented Evaluation for Information Retrieval Systems|
|Session 3 - Domain Specific Search|
|11:15||Anastasia Moumtzidou, Stefanos Vrochidis and Ioannis Kompatsiaris||Concept extraction from patent images based on recursive hybrid classification|
|11:30||Linda Sara Maud Andersson, Mihai Lupu and Allan Hanbury||Domain adaptation of General Natural Language Processing tools for a Patent Claim Visualization system|
|12:00||German Hurtado Martin, Steven Schockaert, Chris Cornelis and Helga Naessens||An Exploratory Study on Content-Based Filtering of Call for Papers|
|12:30||Michail Salampasis and Allan Hanbury||A generalized framework for Integrated Professional Search Systems|
|MUMIA Interdisciplinary WG Meeting (15:30-16:30)|
|Andrea Zielinski||Ontology-based Semantic Search for Information Professionals – Ready for Use?|
|Mike Salampasis||PerFedPat: An Integrated Federated System for Patent Search|
|Henrik Strindberg (findwise.com)||Untapped resources!|
|MUMIA WG: WRAP-UP SESSION (16:30-17:00)|
|17:00||End of Day 3|
Tutorial: Multimedia information retrieval
Prof Stefan Rüger, The Open University, UK
The aims of the tutorial is to introduce a wide range of computerized content-based techniques for organizing digital multimedia documents and searching in them (metadata driven, piggy-back text retrieval, content-based retrieval, automated image annotation and fingerprinting); to make the participants familiar with the particular advantages, disadvantages and challenges of these techniques. This part of the tutorial considers video, image and music data under the perspective of querying, representing and retrieving.
- Outline challenges brought about by multimedia collections and their interface needs
- Describe the principle components of a multimedia information retrieval system and how they differ from other retrieval systems, most notably text information retrieval systems
- Describe and implement simple visual feature vectors commonly used in image search engines
- Explain the workings of an image search engine and relate this to other media types
Stefan Rüger joined The Open University’s Knowledge Media Institute in 2006 to take up a chair in Knowledge Media. Before that he was a Reader in Multimedia and Information Systems at the Department of Computing, Imperial College London, where he also held an EPSRC Advanced Research Fellowship (1999–2004). Stefan is a theoretical physicist by training (FU Berlin) and received his PhD in Computing in 1996 from TU Berlin for his work on artificial intelligence and, in particular, the theory of neural networks. Since then he has made a continuous journey from theory to its applications in multimedia retrieval. In 2009 he was awarded a Honorary Professorship (until 2014) at the University of Waikato for his work with the Greenstone Digital Library group and is currently a visiting fellow at Cranfield University, UK. Stefan chaired ECIR 2006 and ECIR 2010, co-chaired ICTIR 2009, was programme chair of IRFC 2010, programme co-chair of WI 2010, ECIR 2013, is currently programme co-chair of RuSSIR 2013 and WI 2013, and has been asked to be programme co-chair of ICMR 2014. He acts as associate editor for three journals and has been reviewing for 32 other Computing journals, 70 international conferences and 15 research funders including the Commission of the European Communities and the European Research Council. Stefan is a Fellow of the Higher Education Academy in the UK; a member of the EPSRC College, ACM and BCS; and of the BCS Specialist Group for Information Retrieval. During his academic career he and his team have authored over 100 scientific publications in the area of multimedia information retrieval.
Tutorial: Evaluation Metrics for Information Access
In this tutorial we will review and compare the most popular evaluation metrics for some of the most salient problems in Information Access,
covering two basic types of task: text organization problems (Retrieval, Clustering, Filtering) and text generation problems (Summarization and
Machine Translation). We will also discuss metric combination procedures.
For most of this problems there are many competing evaluation metrics in the literature, and in general no clear procedure to choose the most adequate in a specific task/scenario. In practice, the tendency is to choose the most popular metric (which has a snowball effect that tends to prefer the oldest metrics).
In addition, for many problems there are different quality aspects that are captured by different metrics (e.g. Precision and Recall) and, although Van Rijsbergen's F measure provides a way of combining and assigning relative weights to individual metrics, there is often a lack of clear criteria to assign relative weights between metrics. The need for metrics combination is of particular importance in text generation problems (summarization, translation), where there are many different criteria to compare system outputs with gold standard texts.
The tutorial will make a special emphasis on the specification of formal constraints for suitable metrics in each of the tasks, and on the systematic comparison of metrics according to how they satisfy such constraints. This comparison provides criteria to select the most adequate metric or set of metrics for each specific Information Access task. The last part of the tutorial will investigate the grand challenge of providing a unified view of evaluation metrics for document organization tasks.
Enrique Amigó (UNED, Madrid, Spain) is associate professor at UNED and member of the nlp.uned.es research group. He has published
several papers (in venues such as SIGIR, ACL, EMNLP, Journal of Artificial Intelligence Research, Information Retrieval journal, etc.) on
evaluation methodologies and metrics for Text Summarization, Machine Translation, Text Clustering, Document Filtering, etc.
Julio Gonzalo (UNED, Madrid, Spain) is head of nlp.uned.es, the UNED research group in Natural Language Processing and Information
Retrieval. He has recently been CLEF 2011 General Co-Chair, Area Chair for EACL 2012, ECIR 2012 and EMNLP 2010, co-organizer of the RepLab
2012/2013 Evaluation Campaigns for Online Reputation Management Systems and the three WePS evaluation campaigns on Web People Search systems.
His research interests include Cross-Language and Interactive Information Retrieval, Search Results Organization, Entity-Oriented and Semantic
Search, and Evaluation Methodologies and Metrics in Information Access.