lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 16719] - java.io.IOException: Pipe closed
Date Fri, 14 Feb 2003 17:16:11 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16719>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16719

java.io.IOException: Pipe closed





------- Additional Comments From Eric.Isakson@sas.com  2003-02-14 17:16 -------
I started seeing this also, it is not being raised as an exception just dumped 
to System.err.

In DocumentWriter.invertDocument(Document doc) when you exceed maxFieldLength 
it does stream.close which closes the reader used for input to HTMLParser. The 
stack trace is generated by ParserThread.run() when the IOException occurs as 
the HTMLParser attempts to write to pipeOut which was closed when the reader 
was closed from the other thread.

I suppose you could have other IO problems that might raise IOException and you 
would want to see those end up in your log so you wouldn't want to not report 
these.

I don't know enough about threading to know if there is some solution to this 
that would allow IndexWriter to wait for the parser to finish or interrupt it 
before closing the reader or for the HTMLParser to notice that its input was 
closed, so I just wanted to share what I found.

maxFieldLength is set in IndexWriter and passed in when you construct the 
DocumentWriter:
  /** The maximum number of terms that will be indexed for a single field in a
    document.  This limits the amount of memory required for indexing, so that
    collections with very large files will not crash the indexing process by
    running out of memory.

    <p>By default, no more than 10,000 terms will be indexed for a field. */
  public int maxFieldLength = 10000;

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message