lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: semi-infinite loop during merging
Date Thu, 16 Apr 2009 12:10:56 GMT
One question: are you using IndexWriter.close(false)?  I wonder if
there's some path whereby the merges fail to abort (and simply keep
retrying) if you do that...

More inlined below...

On Thu, Apr 16, 2009 at 5:42 AM, Christiaan Fluit
<> wrote:
> I spent a lot of time on getting the stacktraces, but JET seems to make this
> impossible. Ctrl-Break, connecting with JConsole, even a "Dump Threads"
> button in my UI that uses Threads.getAllStacktraces were not able to produce
> a dump of all threads.


> I just got an additional confirmation that the problem also occurs with the
> Java build, but unfortunately the client's data is too sensitive to share it
> with me.

Could they "kill -QUIT" or  Ctrl-Break when it's happening?

> One option is to hack the Lucene 2.4.1 code to print out some additional
> debug info. Do you known some println's that would help?

How about adding this at the top of IndexWriter.handleMergeException:

  if (infoStream != null) {
    message("handleMergeException: merge=" +
merge.segString(directory) + " exc=" + t);


You could also sprinkle prints in between where message("merge:
total"...) occurs and the call to commitMerge, in
IndexWriter.mergeMiddle.  The mystery here is why you see these two
lines in a row:

  IW 0 [CrawlThread]: merge: total 18586 docs
  IW 0 [CrawlThread]: LMP: findMerges: 10 segments

That first line is supposed to be followed by a "commitMerge: ..."
line.  I suspect some exception (maybe a MergeAbortedException) is
being hit, leading to the commit not happening.

> Also I suspect that JET-compiled code is able to do Thread.dumpStack (but
> not Thread.getAllStackTraces), so what are good locations for doing that?
> E.g. IndexWriter.merge, etc.

This would be great -- I would add that in handleMergeException as well.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message