lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stanislaw Osinski (JIRA)" <>
Subject [jira] Commented: (LUCENE-966) A faster JFlex-based replacement for StandardAnalyzer
Date Thu, 02 Aug 2007 12:10:52 GMT


Stanislaw Osinski commented on LUCENE-966:

To be precise -- I'm not 100% sure that this is a bug in JavaCC (I'll try to browse/ask on
their mailing list to find out), but it looks like the scanner generated by JavaCC does not
really strictly follow the grammar. I discovered this when you gave the examples of different
token produced by JFlex- and JavaCC-based scanners generated from equivalent grammars -- JFlex
seemed to behave "correctly", hence different tokens.

I was about to start trying to hack the grammar to produce results simiar to StandardAnalyzer
(based on a finite set of test cases :). I'll see how it compares performance-wise too.

> A faster JFlex-based replacement for StandardAnalyzer
> -----------------------------------------------------
>                 Key: LUCENE-966
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Stanislaw Osinski
>             Fix For: 2.3
>         Attachments:, jflex-analyzer-patch.txt, jflex-analyzer-r560135-patch.txt,
jflex-analyzer-r561292-patch.txt, jflex-analyzer-r561693-compatibility.txt
> JFlex ( can be used to generate a faster (up to several times) replacement
for StandardAnalyzer. Will add a patch and a simple benchmark code in a while.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message