lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3515) Possible slowdown of indexing/merging on 3.x vs trunk
Date Thu, 13 Oct 2011 19:11:12 GMT


Robert Muir commented on LUCENE-3515:

it looks like the bug will especially affect any directory that uses bufferedindexinput (NIOFS/SimpleFS).

The problem is multitermsenum doesnt reuse the sub-docs&positionsenums, so for each term*segment
we clone the input, and
bufferedindexinput.clone() sets the clone's buffer to null.

so across lots of low freq-terms we re-read 4096 bytes (MERGE_BUFFER_SIZE) to refill the buffer
on each one...

mmapdirectory is less affected because it has no buffer to re-read but seems like fixing the
reusing would even help it...
> Possible slowdown of indexing/merging on 3.x vs trunk
> -----------------------------------------------------
>                 Key: LUCENE-3515
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Michael McCandless
>             Fix For: 3.5, 4.0
>         Attachments: LUCENE-index-34.patch, LUCENE-index-40.patch,,, stdout-snow-leopard.tar.gz
> Opening an issue to pursue the possible slowdown Marc Sturlese uncovered.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message