lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Bamford <Chris.Bamf...@scalix.com>
Subject RE: New "Stream closed" exception with Java 6
Date Fri, 04 Sep 2009 10:34:09 GMT
Hi Grant

> Seems like something is closing your InputStreamReader out from under you. 
> Is there a concurrency issue, perhaps?

I'm coming to the same conclusion - there must be >1 threads accessing this index at the
same time.  Better go figure it out  ...  :-)

Thanks again

- Chris

Chris Bamford
Senior Development Engineer
Scalix
chris.bamford@scalix.com
Tel: +44 (0)1344 381814
www.scalix.com



----- Original Message -----
From: Grant Ingersoll <gsingers@apache.org>
Sent: Thu, 3/9/2009 2:59pm
To: java-user@lucene.apache.org
Subject: Re: New "Stream closed" exception with Java 6


On Sep 2, 2009, at 7:45 AM, Chris Bamford wrote:

> Hi Grant,
>
> I have now followed Daniel's advice and catch the exception with:
>
>    try {
>        indexWriter.addDocument(doc);

What does your Document/Field creation code look like?  In other  
words, how do you construct doc?  Seems like something is closing your  
InputStreamReader out from under you.  Is there a concurrency issue,  
perhaps?


>    } catch (CorruptIndexException ex) {
>        throw new IndexerException ("CorruptIndexException on doc: "  
> + doc.toString(), ex);
>    } catch (IOException ex) {
>        throw new IndexerException ("*IOException* on doc: " +  
> doc.toString(), ex);
>    }
>
> The result now appears as:
>
> 2009-09-02 10:14:24,014 ERROR [QueueManager]  
> [BatchUpdater.processMods:183] Failed to add content document(s) for  
> user 0bb00000daaa2864-9.0.61.271  
> com.scalix.index.api.IndexerException: *IOException* on doc:  
> Document<indexed,tokenized<return-path:christian.sholtz@scalix.com>  
> indexed,tokenized<received:from scalixmail.scalix.com (localhost  
> [127.0.0.1])
>        by scalixmail.scalix.com (8.13.6/8.13.6/SuSE Linux 0.8) with  
> ESMTP id m0F9545p008012;
>        Tue, 15 Jan 2008 04:05:05 -0500>  
> indexed,tokenized<received:from [10.13.1.51] (scalixmail.scalix.com  
> [172.16.0.9])
>    by scalixmail.scalix.com (Scalix SMTP Relay 11.3.0.11339)
>    via ESMTP; Tue, 15 Jan 2008 04:05:05 -0500 (EST)> indexed<CREATE- 
> DATE:20080115090228> indexed,tokenized<date:Tue Jan 15 04:02:28 EST  
> 2008>  indexed,tokenized<from:"Christian Sholtz" <christian.sholtz@scalix.com

> >> indexed,tokenized<to:SIN - Robert Smith <Robert.Smi
> th@sin.co.at>> indexed,tokenized<cc:Patrick Lauria  
> <patrick@scalix.com>> indexed<message-id: 
> 478C76A4.2090600@scalix.com> indexed<in-reply- 
> to:H000007b0031e9f3.1200380510.scalix.sin.co.at@MHS>  
> indexed<references:H000007b0031e9f3.1200380510.scalix.sin.co.at@MHS>  
> indexed,tokenized<subject:
> Re: AW: Lead Landwirtschaftskammer Kärnten -> Smith>  
> indexed,tokenized<autoforwarded:FALSE>  
> indexed,tokenized<importance:Normal>  
> indexed,tokenized<priority:Normal>  
> indexed,tokenized<sensitivity:Normal> indexed,tokenized<x-msmail- 
> priority:Medium> indexed,tokenized<x-priority:3> indexed<x-scalix- 
> autoreplied:FALSE> indexed,tokenized<user-agent:Thunderbird 2.0.0.9  
> (Windows/20071031)> indexed,tokenized<x-spam-status:No, score=-1.3  
> required=5.0 tests=AWL,BAYES_00,
>        SUBJECT_ENCODED_TWICE autolearn=no version=3.1.8>  
> indexed,tokenized<x-spam-checker-version:SpamAssassin 3.1.8  
> (2007-02-13) on
>        scalixmail.scalix.com> indexed,tokenized<x-mime- 
> autoconverted:from 8bit to quoted-printable by scalixmail.scalix.com  
> id m0F9545p00801
> 2> indexed,tokenized<mime-version:1.0> indexed<SIZE-KBYTES: 
> 0000000006> indexed<DELIVERY-DATE:20080115090509>  
> indexed,tokenized<x-scalix-internaldate:Tue Jan 15 04:05:09 EST  
> 2008> indexed<x-scalix-has-attach:FALSE> indexed,tokenized<content- 
> type:text/plain;
>        charset="ISO-8859-1";
>        format="flowed"> indexed,tokenized<content-transfer- 
> encoding:quoted-printable> indexed,tokenized<content- 
> disposition:inline> indexed<CONTENT-TYPE:text/plain>  
> indexed,tokenized<BODY:java.io.InputStreamReader@75b940> indexed<x- 
> scalix-class:IPM.Note> indexed<MSG-TYPE:email> stored/ 
> uncompressed,indexed<INDEXID:90010ac-4682aaad-478c7745-2ee4ab>>
>        at com.scalix.index.manager.BatchUpdater.performContentAdds 
> (BatchUpdater.java:394)
>        at com.scalix.index.manager.BatchUpdater.processMods 
> (BatchUpdater.java:179)
>        at com.scalix.index.manager.QueueManager.processQueue 
> (QueueManager.java:134)
>        at com.scalix.index.manager.QueueManager.run 
> (QueueManager.java:81)
>        at java.lang.Thread.run(Unknown Source)
> Caused by: java.io.IOException: Stream closed
>        at sun.nio.cs.StreamDecoder.ensureOpen(Unknown Source)
>        at sun.nio.cs.StreamDecoder.read(Unknown Source)
>        at java.io.InputStreamReader.read(Unknown Source)
>        at  
> org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill 
> (StandardTokenizerImpl.java:443)
>        at  
> org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken 
> (StandardTokenizerImpl.java:629)
>        at org.apache.lucene.analysis.standard.StandardTokenizer.next 
> (StandardTokenizer.java:140)
>        at org.apache.lucene.analysis.standard.StandardFilter.next 
> (StandardFilter.java:43)
>        at org.apache.lucene.analysis.LowerCaseFilter.next 
> (LowerCaseFilter.java:34)
>        at org.apache.lucene.analysis.snowball.SnowballFilter.next 
> (SnowballFilter.java:64)
>        at org.apache.lucene.index.DocInverterPerField.processFields 
> (DocInverterPerField.java:134)
>        at  
> org.apache.lucene.index.DocFieldConsumersPerField.processFields 
> (DocFieldConsumersPerField.java:36)
>        at  
> org.apache.lucene.index.DocFieldProcessorPerThread.processDocument 
> (DocFieldProcessorPerThread.java:234)
>        at org.apache.lucene.index.DocumentsWriter.updateDocument 
> (DocumentsWriter.java:765)
>        at org.apache.lucene.index.DocumentsWriter.addDocument 
> (DocumentsWriter.java:743)
>        at org.apache.lucene.index.IndexWriter.addDocument 
> (IndexWriter.java:1902)
>        at org.apache.lucene.index.IndexWriter.addDocument 
> (IndexWriter.java:1880)
>        at com.scalix.index.manager.BatchUpdater.performContentAdds 
> (BatchUpdater.java:390)
>        ... 4 more
>
> Does this help?
>
> Thanks,
>
> - Chris
>
>
>
> ----- Original Message -----
> From: Daniel Shane <shaned@LEXUM.UMontreal.CA>
> Sent: Tue, 1/9/2009 10:52pm
> To: java-user@lucene.apache.org
> Subject: Re: New "Stream closed" exception with Java 6
>
> I think you should do this instead (it will print the exception  
> message
> *and* the stack trace instead of only the message) :
>
> throw new IndexerException ("CorruptIndexException on doc: " +  
> doc.toString(), ex);
>
> Daniel Shane
>
> Chris Bamford wrote:
>> Hi Grant,
>>
>>
>>>> I think you code there needs to show the underlying exception,  
>>>> too, so
>>>> we can see that stack trace.
>>>>
>>
>> Ummm... isn't this code already doing that?  What am I missing?
>>
>>            try {
>>
>>                indexWriter.addDocument(doc);
>>
>>            } catch (CorruptIndexException ex) {
>>
>>                throw new IndexerException ("CorruptIndexException  
>> on doc: " + doc.toString() +
>>
>>                    " - " + ex.toString());
>>
>>            } catch (IOException ex) {
>>
>>                throw new IndexerException ("IOException on doc: " +  
>> doc.toString() +
>>
>>                    " - " + ex.toString());
>>
>>            }
>>
>> Thanks,
>>
>> - Chris
>>
>> Chris Bamford
>> Senior Development Engineer
>> Scalix
>> chris.bamford@scalix.com
>> Tel: +44 (0)1344 381814
>> www.scalix.com
>>
>>
>>
>> ----- Original Message -----
>> From: Grant Ingersoll <gsingers@apache.org>
>> Sent: Sat, 29/8/2009 12:00pm
>> To: java-user@lucene.apache.org
>> Subject: Re: New "Stream closed" exception with Java 6
>>
>> I think you code there needs to show the underlying exception, too,  
>> so
>> we can see that stack trace.
>>
>> -Grant
>>
>> On Aug 28, 2009, at 11:47 AM, Chris Bamford wrote:
>>
>>
>>> Hi Grant
>>>
>>> Please see attached.
>>>
>>> Thanks,
>>>
>>> - Chris
>>>
>>>
>>> ----- Original Message -----
>>> From: Grant Ingersoll <gsingers@apache.org>
>>> Sent: Fri, 28/8/2009 1:26pm
>>> To: java-user@lucene.apache.org
>>> Subject: Re: New "Stream closed" exception with Java 6
>>>
>>> Are there any log messages around that spot?  Did something happen  
>>> to
>>> the underlying Reader you are using?  Can you share that little  
>>> bit of
>>> indexing code?
>>>
>>> On Aug 27, 2009, at 10:11 AM, Chris Bamford wrote:
>>>
>>>
>>>> Hi,
>>>>
>>>> Since moving our app to Java 6 and Tomcat 6, we have started  
>>>> getting
>>>> occasional exceptions of the form:
>>>>
>>>> java.io.IOException: Stream closed
>>>>  at sun.nio.cs.StreamDecoder.ensureOpen(Unknown Source)
>>>>  at sun.nio.cs.StreamDecoder.read(Unknown Source)
>>>>  at java.io.InputStreamReader.read(Unknown Source)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene
>>>> .analysis
>>>> .standard.StandardTokenizerImpl.zzRefill 
>>>> (StandardTokenizerImpl.java:
>>>> 443)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene
>>>> .analysis
>>>> .standard
>>>> .StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:629)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene
>>>> .analysis.standard.StandardTokenizer.next(StandardTokenizer.java: 
>>>> 140)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene.analysis.standard.StandardFilter.next(StandardFilter.java: 
>>>> 43)
>>>>  at
>>>> org.apache.lucene.analysis.LowerCaseFilter.next 
>>>> (LowerCaseFilter.java:
>>>> 34)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene.analysis.snowball.SnowballFilter.next(SnowballFilter.java: 
>>>> 64)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene
>>>> .index.DocInverterPerField.processFields(DocInverterPerField.java:
>>>> 134)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene
>>>> .index
>>>> .DocFieldConsumersPerField
>>>> .processFields(DocFieldConsumersPerField.java:36)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene
>>>> .index
>>>> .DocFieldProcessorPerThread
>>>> .processDocument(DocFieldProcessorPerThread.java:234)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:
>>>> 765)
>>>>  at
>>>> org
>>>> .apache
>>>> .lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:743)
>>>>  at
>>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
>>>> 1902)
>>>>  at
>>>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:
>>>> 1880)
>>>>  at
>>>> com
>>>> .scalix
>>>> .index.manager.BatchUpdater.performContentAdds(BatchUpdater.java: 
>>>> 386)
>>>>  at
>>>> com.scalix.index.manager.BatchUpdater.processMods 
>>>> (BatchUpdater.java:
>>>> 176)
>>>>  at
>>>> com.scalix.index.manager.QueueManager.processQueue 
>>>> (QueueManager.java:
>>>> 134)
>>>>  at com.scalix.index.manager.QueueManager.run(QueueManager.java:81)
>>>>  at java.lang.Thread.run(Unknown Source)
>>>>
>>>> We are using Lucene 2.4.  Any ideas what is happening?
>>>>
>>>> Thanks!
>>>>
>>>> - Chris
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>> --------------------------
>>> Grant Ingersoll
>>> http://www.lucidimagination.com/
>>>
>>> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
>>> using Solr/Lucene:
>>> http://www.lucidimagination.com/search
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>> <lucene_msg.txt>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>> --------------------------
>> Grant Ingersoll
>> http://www.lucidimagination.com/
>>
>> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)
>> using Solr/Lucene:
>> http://www.lucidimagination.com/search
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message