accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] ctubbsii commented on a change in pull request #546: Fix multiple concurrency bugs in Master.gatherTableInformation()
Date Mon, 02 Jul 2018 18:40:58 GMT
ctubbsii commented on a change in pull request #546: Fix multiple concurrency bugs in Master.gatherTableInformation()
URL: https://github.com/apache/accumulo/pull/546#discussion_r199583551
 
 

 ##########
 File path: server/master/src/main/java/org/apache/accumulo/master/Master.java
 ##########
 @@ -1147,11 +1149,16 @@ private long balanceTablets() {
   private SortedMap<TServerInstance,TabletServerStatus> gatherTableInformation(
       Set<TServerInstance> currentServers) {
     long start = System.currentTimeMillis();
-    int threads = Math.max(getConfiguration().getCount(Property.MASTER_STATUS_THREAD_POOL_SIZE),
1);
-    ExecutorService tp = Executors.newFixedThreadPool(threads);
-    final SortedMap<TServerInstance,TabletServerStatus> result = new TreeMap<>();
+    ExecutorService tp = Executors.newCachedThreadPool();
+    // use ConcurrentSkipListMap for two reasons. First, multiple threads may concurrently
put.
+    // Second, its ok for one thread to iterate over map entries while another thread puts.
+    final SortedMap<TServerInstance,TabletServerStatus> result = new ConcurrentSkipListMap<>();
     for (TServerInstance serverInstance : currentServers) {
       final TServerInstance server = serverInstance;
+      // Since an unbounded thread pool is being used, rate limit how fast task are added
to the
+      // executor. This prevents the threads from growing large unless there are lots of
+      // unresponsive tservers.
+      sleepUninterruptibly(5, TimeUnit.MILLISECONDS);
 
 Review comment:
   I think maybe this should be some fraction of the client timeout, because the client timeout
indicates some user awareness of how long the RPCs should take.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message