lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3919) more thorough testing of analysis chains
Date Sun, 25 Mar 2012 22:40:27 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238002#comment-13238002
] 

Robert Muir commented on LUCENE-3919:
-------------------------------------

The CzechStemmer bug is easy, its because of a zero-length term from KeywordTokenizer :)
I'll commit a trivial fix and test for that.

The next time I ran the test, i got a new fail:
{noformat}
    [junit] TEST FAIL: useCharFilter=true text=⩀⪴⫈⪆⩞ ye ه
    [junit] Exception from random analyzer: tokenizer=class org.apache.lucene.analysis.ngram.NGramTokenizer
    [junit] filters=class org.apache.lucene.analysis.shingle.ShingleFilter
    [junit] NOTE: reproduce with: ant test -Dtestcase=TestRandomChains -Dtestmethod=testRandomChains
-Dtests.seed=104b56460756fb6:33a429fcfb5503db:-1d952b2910440c7d -Dargs="-Dfile.encoding=UTF-8"
   [junit] java.lang.AssertionError: endOffset must be >= startOffset
    [junit] java.lang.RuntimeException: java.lang.AssertionError: endOffset must be >=
startOffset
    [junit] 	at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:397)
{noformat}

This is gonna be fun...
                
> more thorough testing of analysis chains
> ----------------------------------------
>
>                 Key: LUCENE-3919
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3919
>             Project: Lucene - Java
>          Issue Type: Task
>          Components: modules/analysis
>    Affects Versions: 3.6, 4.0
>            Reporter: Robert Muir
>         Attachments: LUCENE-3919.patch
>
>
> In lucene we essentially test each analysis component separately. we also give some
> good testing to the example Analyzers we provide that combine them.
> But we don't test various combinations that are possible: which is bad because
> it doesnt test possibilities for custom analyzers (especially since lots of solr users
> etc define their own).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message