pinot-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #5039: Close Kafka consumer before building or downloading immutable segment
Date Mon, 10 Feb 2020 17:39:42 GMT
mcvsubbu commented on a change in pull request #5039: Close Kafka consumer before building
or downloading immutable segment
URL: https://github.com/apache/incubator-pinot/pull/5039#discussion_r377204241
 
 

 ##########
 File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/LLRealtimeSegmentDataManager.java
 ##########
 @@ -203,10 +203,14 @@ public void deleteSegmentFile() {
   private final String _resourceDataDir;
   private final IndexLoadingConfig _indexLoadingConfig;
   private final Schema _schema;
-  // Semaphore for each partitionId only. See the comments in {@link RealtimeTableDataManager}.
+  // Semaphore for each partitionId only, which is to prevent two different Kafka consumers
+  // from consuming with the same partitionId in parallel in the same host.
+  // See the comments in {@link RealtimeTableDataManager}.
   private final Semaphore _partitionConsumerSemaphore;
-  // A boolean flag to check whether the current thread acquires the semaphore, so that the
semaphore be released only once within the same thread.
-  private final AtomicBoolean _acquireConsumerSemaphore;
+  // A boolean flag to check whether the current thread has acquired the semaphore.
+  // This boolean is needed because the semaphore is shared by threads; every thread holding
this semaphore can
+  // modify the permit. This boolean make sure the semaphore gets released only once within
the same thread.
 
 Review comment:
   ```suggestion
     // modify the permit. This boolean make sure the semaphore gets released only once when
the partition stops consuming.
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Mime
View raw message