lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Commented: (LUCENE-2384) Reset zzBuffer in StandardTokenizerImpl* when lexer is reset.
Date Thu, 08 Apr 2010 11:22:36 GMT


Uwe Schindler commented on LUCENE-2384:

For JFlex this does not help as the Jflex-generated code always needs a Reader. This is special
here, the lexer will not need to load the whole document into the reader, it only needs sometimes
a large look forward/backwards buffer.

> Reset zzBuffer in StandardTokenizerImpl* when lexer is reset.
> -------------------------------------------------------------
>                 Key: LUCENE-2384
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Sub-task
>          Components: Analysis
>    Affects Versions: 3.0.1
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
> When indexing large documents, the lexer buffer may stay large forever. This sub-issue
resets the lexer buffer back to the default on reset(Reader).
> This is done on the enclosing issue.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message