lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <>
Subject [jira] Created: (LUCENE-2706) ByteBlockPool should allow different block sizes
Date Fri, 15 Oct 2010 14:00:35 GMT
ByteBlockPool should allow different block sizes 

                 Key: LUCENE-2706
             Project: Lucene - Java
          Issue Type: Improvement
    Affects Versions: 4.0
            Reporter: Simon Willnauer
            Assignee: Simon Willnauer

Currently ByteBlockPool has a fixed byteblock size of 1 << 15 bytes. This limit also
makes BytesRefHash bound to 32kb buffers which is not ideal in many cases. For instance in
MTQ (LUCENE-2960) we might want to have a smaller block size of 1024 or 2048 for more efficiency.
While this example  suffers from another limitation since the blockSize used in TermsHashPerField
is 32kb which in turn defines the size limit for a single term in the index. To make effective
use of different byte block sizes we need to find a solution for this problem too but I think
index statistics are on the way and max term length for a field could also make merges more
efficient if we can alter the byte block size especially for fields with very short terms.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message