incubator-ctakes-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Miller (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CTAKES-143) dictionary lookup iterates inefficiently
Date Mon, 04 Feb 2013 21:44:14 GMT

     [ https://issues.apache.org/jira/browse/CTAKES-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tim Miller updated CTAKES-143:
------------------------------

    Attachment: dict.diff
    
> dictionary lookup iterates inefficiently
> ----------------------------------------
>
>                 Key: CTAKES-143
>                 URL: https://issues.apache.org/jira/browse/CTAKES-143
>             Project: cTAKES
>          Issue Type: Improvement
>          Components: ctakes-dictionary-lookup
>            Reporter: Tim Miller
>         Attachments: dict.diff
>
>
> I noticed quadratic performance on large notes (in length of file) mainly due to dictionary
annotator.  Inside DictionaryLookupAnnotator, it iterates over "Lookup windows."  Inside this
iteration, it calls getLookupTokenIterator() in the LookupInitializer.  As implemented in
FirstTokenPermLookupInitializerImpl, this method will create a iterator over all BaseTokens,
pruning out certain subtypes.  This is then passed back to the DictionaryLookupAnnotator,
which will only then constrain the list to those tokens which overlap the lookup window. 
This iterating over every token in the document for each lookup window is extremely inefficient.
 This could be fixed by, e.g., doing the constraining first and then doing the pruning by
sub-types.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message