lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Smiley (JIRA)" <>
Subject [jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel
Date Tue, 29 May 2018 20:00:00 GMT


David Smiley commented on SOLR-12338:

Please consider this alternative utility class:

/** A set of locks by a key {@code T}, kind of like Google Striped but the keys are sparse/lazy.
private static class SparseStripedLock<T> {
  private final Semaphore sizeSemaphore;
  private ConcurrentHashMap<T, CountDownLatch> map = new ConcurrentHashMap<>();

  SparseStripedLock(int maxSize) {
    this.sizeSemaphore = new Semaphore(maxSize);

  void add(T t) throws InterruptedException {
    if (t != null) {
      CountDownLatch myLock = new CountDownLatch(1);
      while (true) {
        CountDownLatch existingLock = map.putIfAbsent(t, myLock); // returns null if no existing
        if (existingLock == null) {
          break;// myLock was successfully inserted (and is pre-locked) already locked, was
successfully inserted
        existingLock.await();// wait for existing lock/permit to become available (see remove()
        // we will most likely exit in next loop, though if contended then possibly not

    // won the lock
    sizeSemaphore.acquire();  //nocommit do at start of add()?

  void remove(T t) {
    if (t != null) {
      map.remove(t).countDown(); // remove and signal to any "await"-ers

Notice the comments, the loop, the new name, use of CountDownLatch, and the one nocommit/question.

> Replay buffering tlog in parallel
> ---------------------------------
>                 Key: SOLR-12338
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Cao Manh Dat
>            Assignee: Cao Manh Dat
>            Priority: Major
>         Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch,
> Since updates with different id are independent, therefore it is safe to replay them
in parallel. This will significantly reduce recovering time of replicas in high load indexing

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message