hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Sun (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-16385) StatsNoJobTask could exit early before all partitions have been processed
Date Wed, 05 Apr 2017 18:07:41 GMT
Chao Sun created HIVE-16385:
-------------------------------

             Summary: StatsNoJobTask could exit early before all partitions have been processed
                 Key: HIVE-16385
                 URL: https://issues.apache.org/jira/browse/HIVE-16385
             Project: Hive
          Issue Type: Bug
          Components: Statistics
            Reporter: Chao Sun
            Assignee: Chao Sun


For a partitioned table, the class {{StatsNoJobTask}} is supposed to launch threads for all
partitions and compute their stats. However, it could exit early after at most 100 seconds:

{code}
  private void shutdownAndAwaitTermination(ExecutorService threadPool) {

    // Disable new tasks from being submitted
    threadPool.shutdown();
    try {

      // Wait a while for existing tasks to terminate
      if (!threadPool.awaitTermination(100, TimeUnit.SECONDS)) {
        // Cancel currently executing tasks
        threadPool.shutdownNow();

        // Wait a while for tasks to respond to being cancelled
        if (!threadPool.awaitTermination(100, TimeUnit.SECONDS)) {
          LOG.debug("Stats collection thread pool did not terminate");
        }
      }
    } catch (InterruptedException ie) {

      // Cancel again if current thread also interrupted
      threadPool.shutdownNow();

      // Preserve interrupt status
      Thread.currentThread().interrupt();
    }
  }
{code}

The {{shutdown}} call does not wait for all submitted tasks to complete, and the {{awaitTermination}}
call waits at most 100 seconds. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message