lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <>
Subject [jira] [Commented] (LUCENE-8757) Better Segment To Thread Mapping Algorithm
Date Thu, 02 May 2019 12:41:00 GMT


Simon Willnauer commented on LUCENE-8757:

Hey Atri,

thanks for putting up this patch, here is some additional feedback:
 - can we stick with an protected non-static method on IndexSearcher subclasses should be
able to override your impl. I think it's ok to have a static method like this:
 public static LeafSlice[] slices (List<LeafReaderContext> leaves, int maxDocsPerSlice,
int maxSegPerSlice){code}
that you can call from the protected method with your defaults?
 - you might want to change your sort to something like this: 
Collections.sort(leaves, Collections.reverseOrder(Comparator.comparingInt(l -> l.reader().maxDoc())));{code}

 - I think the _Leaves_ class is unnecessary we can just use _List<LeafReaderContext>_

> Better Segment To Thread Mapping Algorithm
> ------------------------------------------
>                 Key: LUCENE-8757
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Atri Sharma
>            Priority: Major
>         Attachments: LUCENE-8757.patch
> The current segments to threads allocation algorithm always allocates one thread per
segment. This is detrimental to performance in case of skew in segment sizes since small segments
also get their dedicated thread. This can lead to performance degradation due to context switching
> A better algorithm which is cognizant of size skew would have better performance for
realistic scenarios

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message