ctakes-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Miller (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CTAKES-143) dictionary lookup iterates inefficiently
Date Mon, 04 Feb 2013 21:34:14 GMT
Tim Miller created CTAKES-143:

             Summary: dictionary lookup iterates inefficiently
                 Key: CTAKES-143
                 URL: https://issues.apache.org/jira/browse/CTAKES-143
             Project: cTAKES
          Issue Type: Improvement
          Components: ctakes-dictionary-lookup
            Reporter: Tim Miller

I noticed quadratic performance on large notes (in length of file) mainly due to dictionary
annotator.  Inside DictionaryLookupAnnotator, it iterates over "Lookup windows."  Inside this
iteration, it calls getLookupTokenIterator() in the LookupInitializer.  As implemented in
FirstTokenPermLookupInitializerImpl, this method will create a iterator over all BaseTokens,
pruning out certain subtypes.  This is then passed back to the DictionaryLookupAnnotator,
which will only then constrain the list to those tokens which overlap the lookup window. 
This iterating over every token in the document for each lookup window is extremely inefficient.
 This could be fixed by, e.g., doing the constraining first and then doing the pruning by

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message