lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <reng...@ix.netcom.com>
Subject Re: [jira] Resolved: (LUCENE-693) ConjunctionScorer - more tuneup
Date Fri, 23 Nov 2007 17:09:37 GMT
I don't understand this.

If the problem is the "clone", just remove the clone and call  
Arrays.sort() using the bounded version (from,to indices)

The insertion sort code is modifying the array in place, the sort()  
will do the same.

Maybe I am missing something?

On Nov 23, 2007, at 11:02 AM, Yonik Seeley (JIRA) wrote:

>
>      [ https://issues.apache.org/jira/browse/LUCENE-693? 
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Yonik Seeley resolved LUCENE-693.
> ---------------------------------
>
>     Resolution: Fixed
>
> committed.
>
>> ConjunctionScorer - more tuneup
>> -------------------------------
>>
>>                 Key: LUCENE-693
>>                 URL: https://issues.apache.org/jira/browse/LUCENE-693
>>             Project: Lucene - Java
>>          Issue Type: Bug
>>          Components: Search
>>    Affects Versions: 2.1
>>         Environment: Windows Server 2003 x64, Java 1.6, pretty  
>> large index
>>            Reporter: Peter Keegan
>>         Attachments: conjunction.patch, conjunction.patch,  
>> conjunction.patch, conjunction.patch, conjunction.patch.nosort1
>>
>>
>> (See also: #LUCENE-443)
>> I did some profile testing with the new ConjuctionScorer in 2.1  
>> and discovered a new bottleneck in ConjunctionScorer.sortScorers.  
>> The java.utils.Arrays.sort method is cloning the Scorers array on  
>> every sort, which is quite expensive on large indexes because of  
>> the size of the 'norms' array within, and isn't necessary.
>> Here is one possible solution:
>>   private void sortScorers() {
>> // squeeze the array down for the sort
>> //    if (length != scorers.length) {
>> //      Scorer[] temps = new Scorer[length];
>> //      System.arraycopy(scorers, 0, temps, 0, length);
>> //      scorers = temps;
>> //    }
>>     insertionSort( scorers,length );
>>     // note that this comparator is not consistent with equals!
>> //    Arrays.sort(scorers, new Comparator() {         // sort the  
>> array
>> //        public int compare(Object o1, Object o2) {
>> //          return ((Scorer)o1).doc() - ((Scorer)o2).doc();
>> //        }
>> //      });
>>
>>     first = 0;
>>     last = length - 1;
>>   }
>>   private void insertionSort( Scorer[] scores, int len)
>>   {
>>       for (int i=0; i<len; i++) {
>>           for (int j=i; j>0 && scores[j-1].doc() > scores[j].doc 
>> ();j-- ) {
>>               swap (scores, j, j-1);
>>           }
>>       }
>>       return;
>>   }
>>   private void swap(Object[] x, int a, int b) {
>>     Object t = x[a];
>>     x[a] = x[b];
>>     x[b] = t;
>>   }
>>
>> The squeezing of the array is no longer needed.
>> We also initialized the Scorers array to 8 (instead of 2) to avoid  
>> having to grow the array for common queries, although this  
>> probably has less performance impact.
>> This change added about 3% to query throughput in my testing.
>> Peter
>
> -- 
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message