Description

The ICDAR2015 Competition on Keyword Spotting for Handwritten Documents is organised in the framework of the ICDAR 2015 competitions. The goal of this competition is to fairly compare and promote different approaches used in the field of Keyword Spotting (KWS).

KWS has been considered for both Speech and Text documents under very different points of view and applications targets. Perhaps the most important distinction is Query-by-Example/Query-by-String (QbE/QbS); i.e., whether the query is by giving a word image example, or just a character string. But many other distinctions are (very) relevant; among other: Traning-based/training-free; i.e., whether the KWS system is allowed or not to train on appropiate (annotated) images, and Segmentation-based/segmentation-free; i.e., whether the KWS is applied to full document (page) images or just to images of individual words (previously cropped from the original full images).

Clearly each of these flavours of the KWS problem statement has its own difficulty degree and application targets. For instance, QbS is mandatory for applications involving large-scale handwritten image indexing and search under the precision-recall tradeoff model. In this case, given the scale, it can be very advantageus to use training-based KWS. Other kind of applications involve assisting human transcribers by allowing them to find words in a documment which have a shape similar to a word or part of a word (perhaps one which the transcriber is not sure how to transcribe when it appears for the first time). In such applications, a training-free QbE system is most approriate.

In the present contest we aim at testing KWS systems, maybe developped under different points of view, under uniform data sets and benchmark assessment conditions. This is expected to shear light on the relative capabilities of different approaches and their appropriateness for the different kinds of applications.

The contest is divided into two tracks, depending on the requirements of the keyword spotting approaches. Each track consists of two optional assignments. Participants are able to submit solutions to one or both assignments, depending on the restrictions of their systems.

Track I: Training-free track

This track is aimed for KWS approaches that do not require any training data. The query keywords will be given as image examples (i.e. under the QbE paradigm) to be spotted in the document images. Participants in this track are able to complete two different assignments:

  1. Segmentation-based: The words in the test document images will be segmented and given as image patches to the participants. Participants will have to provide a list of matches between the test patches and query images, sorted by confidence.
  2. Segmentation-free: No segmentation will be given on the test document images. Participants will have to match the query images directly on the document images. They will provide a list of bounding boxes indicating the matches, sorted by confidence.

Track II: Training-based track

This track is aimed for KWS approaches that rely on training data to build their KWS models. Training data consists of pairs of text (line) images accompained with their corresponding transcripts. Participants will have to match a list of query keywords (presented as strings or image examples) across the set of document images. In this track, participants may submit solutions to any of these two assignments:

  1. Query-by-String: The keywords will be given as text strings. Participants will have to provide a list of bounding boxes, sorted by confidence, indicating the image regions where the query keywords are spotted.
  2. Query-by-Example: The keywords will be given as exemplar images. Participants will have to provide a list of bounding boxes, sorted by confidence, indicating the image regions where the query keywords are spotted.

Schedule

  • Jan 10, 2015 Competition opens, start of the registration period.
  • Jan 31, 2015 Baseline systems, training and validation data are made public.
  • Mar 31, 2015 Test data provided to the participants, as well as the submissions form. Registration deadline: No more participants will be admitted.
  • Apr 19, 2015. 23:59 UTC Deadline for submitting results on the test set, together with the description of the KWS systems used in the final submission.
  • Aug 23-26, 2015 Winners and final ranking of all teams will be made public at the ICDAR conference.

Organizers

Please, do not hesitate to write to any of us with your questions, concerns or comments about the competition.

  • Joan Puigcerver1, joapuipe_AT_prhlt_DOT_upv_DOT_es
  • Dr. Alejandro H. Toselli1, ahector_AT_prhlt_DOT_upv_DOT_es
  • Dr. Enrique Vidal1, evidal_AT_prhlt_DOT_upv_DOT_es
1Pattern Recognition and Human Language Technologies