lucene-openrelevance-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicola Ferro <>
Subject Call for Participation - DESIRE 2011 Workshop at CIKM - Data infrastructurEs for Supporting Information Retrieval Evaluation
Date Tue, 26 Jul 2011 06:29:19 GMT
** Apologies for cross postings **

Call for Participation

Data infrastructurEs for Supporting Information Retrieval Evaluation (DESIRE 2011) Workshop

Co-located with CIKM 2011, the 20th ACM Conference on Information and Knowledge Management

Friday 28th October 2011, Glasgow, UK

Early registration deadline: 31st August 2011


The objectives of the workshop are: to gather interested experts from the three areas of interest

to CIKM - databases, information retrieval, and knowledge management - and discuss how to
the problem of envisaging and designing evaluation  infrastructures able to store, manage,
make accessible the scientific data and knowledge of interest for advancing the evaluation
information retrieval and access tools.

The main outcome will be to produce a roadmap and initial best practices to guide the development

of the necessary evaluation infrastructures. 


- An Infrastructure fo Supporting the Evaluation of Interactive Information Retrieval
 Norbert Fuhr, University of Duisburg-Essen, Germany.

- Ontology-based data management
 Maurizio Lenzerini, Sapienza University of Rome, Italy.


- Evaluation with the VIRTUOSO platform
 Gérard Dupont, CASSIDIAN, Elancourt, France
 Gaël de Chalendar, CEA, Fontenay-aux-Roses, France 
 Khaled Kheliff, CASSIDIAN, Val de Reuil, France 
 Dmitri Voitsekhovitchy, CEA, Fontenay-aux-Roses, France 
 Géraud Canety, CEA, Fontenay-aux-Roses, France 

- A Lightweight Framework for Reproducible Parameter Sweeping in Information Retrieval
 Richard Eckart de Castilho, Ubiquitous Knowledge Processing (UKP) Lab, Technische Universität
Darmstadt. Darmstadt, Germany <br/>
 Iryna Gurevych, Ubiquitous Knowledge Processing (UKP) Lab, Technische Universität Darmstadt.
Darmstadt, Germany <br/><br/>

- Use Cases as a Component of Information Access Evaluation
 Jussi Karlgren, Swedish Institute of Computer Science, Sweden
 Anni Järvelin, Swedish Institute of Computer Science, Sweden
 Preben Hansen, Swedish Institute of Computer Science, Sweden
 Gunnar Eriksson, Swedish Institute of Computer Science, Sweden 

- PatOlympics - An Infrastructure for Interactive Evaluation of Patent Retrieval Tools
 Mihai Lupu, Vienna University of Technology, Austria

- Infrastructure and Workflow for the Formal Evaluation of Semantic Search Technologies
 Stuart N. Wrigley, Dept. Computer Science, University of Sheffield, UK
 Raúl García-Castro, Facultad de Informática Universidad Politécnica de Madrid, Spain
 Cássia Trojahn, INRIA & LIG, Montbonnot Saint Martin, France

- Principles for Robust Evaluation Infrastructure
 Justin Zobel, Department of Computer Science & Software Engineering, The University of
Melbourne, Australia 
 William Webber, Department of Computer Science & Software Engineering, The University
of Melbourne, Australia 
 Mark Sanderson, School of Computer Science & Information Technology, RMIT University,
 Alistair Moffat, Department of Computer Science & Software Engineering, The University
of Melbourne, Australia


- Early Registration deadline: 31 August 2011
- Workshop: 28 October 2011


- Stefano Ceri, Politecnico di Milano, Italy
- Maarten de Rijke, University of Amsterdam, The Netherlands
- Donna Harman, National Institute of Standards and Technology (NIST), USA
- Noriko Kando, National Institute of Informatics (NII), Japan
- Henning Müller, University of Applied Sciences Western Switzerland
- Shila Ofek-Koifman, IBM Haifa, Israel
- Christos Papatheodorou, Ionian University, Greece
- Fausto Rabitti, ISTI-CNR, Italy
- Mark Sanderson, RMIT University, Australia
- Timos Sellis, National Technical University of Athens, Greece
- Gerhard Weikum, Max-Planck-Institut für Informatik, Germany


- Maristella Agosti, University of Padua, Italy
- Nicola Ferro, University of Padua, Italy
- Costantino Thanos, ISTI-CNR, National Council of Research, Italy


- Nicola Ferro, ferro@dei.unipd.It


The Information Retrieval area has a strong and long tradition dating back to the 1960s in
and processing scientific data resulting from the experimental evaluation of search algorithms
search systems. This attitude towards evaluation has led to fast and continuous progress in
evolution of information retrieval systems and search engines. 

However, in order to make these data test collections understandable and usable they must
be endowed 
with some auxiliary information, i.e., provenance, quality, context, etc. Therefore, there
is a need 
for metadata models able to describe the main characteristics of evaluation data. In addition,
order to make distributed data collections accessible, sharable, and interoperable, there
is a need 
for advanced data infrastructures.

Nevertheless, the information retrieval area has barely explored and exploited the possibilities
managing, storing, and effectively accessing the scientific data produced during the evaluation

studies by making use of the methods typical of the database and knowledge management areas.

It is thus time for these three communities – information retrieval, databases, and 
knowledge management – to join efforts, meet, and cooperate to address the problem of envisaging
designing useful infrastructures able to coherently manage pertinent data collections and
of information, and so take concrete steps towards developing them.

Indeed, the information retrieval experts need to recognise this need, while the database
knowledge management experts need to understand the problem and work together to solve it
by using 
the methods and techniques specific to information management.

Therefore, the workshop will gather together experts from these three areas,  to encourage
them to 
recognise the urgency of addressing the problem in an integrated and coherent way, and to
efforts towards drawing a roadmap and suggesting best practices for an effective solution
of the 

The goal of the workshop is to understand how to make use of the expertise of the three scientific

areas in a cooperative way to avoid the duplication of efforts which may occur when addressing
problem separately in each specific area and to trigger synergies and joint actions on this

The workshop is intended to be a single full day workshop, consisting of: invited talks and
presentations during the morning; active discussion groups during the afternoon, engaging
participants in the topics which have emerged in the previous sessions in order to define
a roadmap 
and best practices.

View raw message