lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Miller (JIRA)" <>
Subject [jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel
Date Fri, 11 May 2018 05:14:00 GMT


Mark Miller commented on SOLR-12338:

bq. Yeah, but we do not need that flag for the case of LogReplayer, right? Because we are
calling execute method in single-thread.

Technically that sounds right, but I'm not sure I read the contract explicitly promises that.
If we have good testing, it's not much of a concern.

bq. OrderedExecutor ensuring that tasks are kicked off in order for a same id. Yeah, task1
get taken off the queue only after it finishes.

Yeah, so I don't think I spot an open issue for a race.

bq. I think we will throttle the incoming updates properly by doing SOLR-12305.

Ah right, had been looking at that issue recently too and had it on my mind. That is more
where that comment belongs. I was thinking these queues would work with documents coming in
and getting buffered, but they won't get held up from dropping off the document to the tlog.
But anyway, I think that natural throttling is a good first step. I think at the end of the
day, we will want to end up with a Filter though that can do QOS and intelligent throttling
based on data, but I'm pro whatever gets us out of infinite tlog replay soonest short term.

> Replay buffering tlog in parallel
> ---------------------------------
>                 Key: SOLR-12338
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Cao Manh Dat
>            Assignee: Cao Manh Dat
>            Priority: Major
>         Attachments: SOLR-12338.patch, SOLR-12338.patch
> Since updates with different id are independent, therefore it is safe to replay them
in parallel. This will significantly reduce recovering time of replicas in high load indexing

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message