lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <li...@ehatchersolutions.com>
Subject TestRussianAnalyzer
Date Tue, 12 Aug 2003 10:34:03 GMT
Is anyone experiencing problems with TestRussianAnalyzer (besides me)  
when running the test-unit target?  I'm dumping my results below.

I'm going to do some build file revamping because the test stuff was a  
bit screwy - (the "test" target compiled, as did the the "test-unit"  
target, but to two different classes location!).  But before I commit  
my local changes, I wanted to see if its only me with this test failure.

The results below are from my local test changes to use  
getResourceAsStream to get to the test files, so its likely I'm doing  
something wrong.  I'm I18N-challenged.  I've changed the test like this:

     public void testUnicode() throws IOException
     {
         RussianAnalyzer ra = new  
RussianAnalyzer(RussianCharsets.UnicodeRussian);
         InputStream isTxt =  
this.getClass().getResourceAsStream("/org/apache/lucene/analysis/ru/ 
testUnicode.txt");
         InputStreamReader inWords =
             new InputStreamReader(
                 isTxt,
                 "Unicode");

         InputStream isHtm =  
this.getClass().getResourceAsStream("/org/apache/lucene/analysis/ru/ 
testUnicode.txt");
         InputStreamReader sampleUnicode =
             new InputStreamReader(
                 isHtm,
                 "Unicode");

         TokenStream in = ra.tokenStream("all", inWords);

         RussianLetterTokenizer sample =
             new RussianLetterTokenizer(
                 sampleUnicode,
                 RussianCharsets.UnicodeRussian);

...
    }


     [junit] Testsuite: org.apache.lucene.analysis.ru.TestRussianAnalyzer
     [junit] Tests run: 3, Failures: 1, Errors: 2, Time elapsed: 0.683  
sec

     [junit] Testcase:  
testUnicode(org.apache.lucene.analysis.ru.TestRussianAnalyzer):   FAILED
     [junit] Unicode expected:<?????> but was:<??????>
     [junit] junit.framework.ComparisonFailure: Unicode expected:<?????>  
but was:<??????>
     [junit]     at  
org.apache.lucene.analysis.ru.TestRussianAnalyzer.testUnicode(TestRussia 
nAnalyzer.java:114)
     [junit]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native  
Method)
     [junit]     at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav 
a:39)
     [junit]     at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor 
Impl.java:25)


     [junit] Testcase:  
testKOI8(org.apache.lucene.analysis.ru.TestRussianAnalyzer):       
Caused an ERROR
     [junit] Missing byte-order mark
     [junit] sun.io.MalformedInputException: Missing byte-order mark
     [junit]     at  
sun.io.ByteToCharUnicode.convert(ByteToCharUnicode.java:120)
     [junit]     at  
sun.nio.cs.StreamDecoder$ConverterSD.convertInto(StreamDecoder.java:245)
     [junit]     at  
sun.nio.cs.StreamDecoder$ConverterSD.implRead(StreamDecoder.java:295)
     [junit]     at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:180)
     [junit]     at  
java.io.InputStreamReader.read(InputStreamReader.java:167)
     [junit]     at java.io.Reader.read(Reader.java:100)
     [junit]     at  
org.apache.lucene.analysis.CharTokenizer.next(CharTokenizer.java:91)
     [junit]     at  
org.apache.lucene.analysis.ru.RussianLowerCaseFilter.next(RussianLowerCa 
seFilter.java:79)
     [junit]     at  
org.apache.lucene.analysis.StopFilter.next(StopFilter.java:93)
     [junit]     at  
org.apache.lucene.analysis.ru.RussianStemFilter.next(RussianStemFilter.j 
ava:89)
     [junit]     at  
org.apache.lucene.analysis.ru.TestRussianAnalyzer.testKOI8(TestRussianAn 
alyzer.java:146)
     [junit]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native  
Method)
     [junit]     at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav 
a:39)
     [junit]     at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor 
Impl.java:25)


     [junit] Testcase:  
test1251(org.apache.lucene.analysis.ru.TestRussianAnalyzer):       
Caused an ERROR
     [junit] Missing byte-order mark
     [junit] sun.io.MalformedInputException: Missing byte-order mark
     [junit]     at  
sun.io.ByteToCharUnicode.convert(ByteToCharUnicode.java:120)
     [junit]     at  
sun.nio.cs.StreamDecoder$ConverterSD.convertInto(StreamDecoder.java:245)
     [junit]     at  
sun.nio.cs.StreamDecoder$ConverterSD.implRead(StreamDecoder.java:295)
     [junit]     at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:180)
     [junit]     at  
java.io.InputStreamReader.read(InputStreamReader.java:167)
     [junit]     at java.io.Reader.read(Reader.java:100)
     [junit]     at  
org.apache.lucene.analysis.CharTokenizer.next(CharTokenizer.java:91)
     [junit]     at  
org.apache.lucene.analysis.ru.RussianLowerCaseFilter.next(RussianLowerCa 
seFilter.java:79)
     [junit]     at  
org.apache.lucene.analysis.StopFilter.next(StopFilter.java:93)
     [junit]     at  
org.apache.lucene.analysis.ru.RussianStemFilter.next(RussianStemFilter.j 
ava:89)
     [junit]     at  
org.apache.lucene.analysis.ru.TestRussianAnalyzer.test1251(TestRussianAn 
alyzer.java:183)
     [junit]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native  
Method)
     [junit]     at  
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav 
a:39)
     [junit]     at  
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessor 
Impl.java:25)


Mime
View raw message