lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Atri Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-8757) Better Segment To Thread Mapping Algorithm
Date Wed, 15 May 2019 19:15:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840683#comment-16840683
] 

Atri Sharma commented on LUCENE-8757:
-------------------------------------

Hi [~jpountz],

I was going through IndexSearcher code and see that there are no checks that we do in IndexSearcher
to ensure that LeafReaderContexts are ordered by docID. Should we add those checks while constructing
a new instance? I think that can be orthogonal to this patch, since this patch anyways orders
leaf slices by docIDs. WDYT?

> Better Segment To Thread Mapping Algorithm
> ------------------------------------------
>
>                 Key: LUCENE-8757
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8757
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Atri Sharma
>            Assignee: Simon Willnauer
>            Priority: Major
>         Attachments: LUCENE-8757.patch, LUCENE-8757.patch, LUCENE-8757.patch, LUCENE-8757.patch,
LUCENE-8757.patch
>
>
> The current segments to threads allocation algorithm always allocates one thread per
segment. This is detrimental to performance in case of skew in segment sizes since small segments
also get their dedicated thread. This can lead to performance degradation due to context switching
overheads.
>  
> A better algorithm which is cognizant of size skew would have better performance for
realistic scenarios



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message