hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Zhang" <>
Subject Review Request: HIVE-2127. Improve stats gathering reliability by retries on failures
Date Mon, 25 Apr 2011 18:25:40 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hive.


The major changes are:

 0) 2 parameters are introduced: hive.stats.retries.max (default 0) to be the maximum # of
retries on SQLException failures, and hive.stats.retries.wait (default 3 sec) to be the base
time window (explained below) to wait before the next retry. 

 1) introduced a couple of Utilities function to execute SQL queries with retries on failures.
One Utilities function is to determine the wait time based on the number of failures and a
base wait window (same as the one introduced in HDFS-767 for DFSClient to retry on BlockMissingExceptions).
The actual wait time is determined by baseWindow * failues + baseWindow * (failure + 1) *
(random number between [0.0,1.0]).

 2) changed the to use PreparedStatement to be able to use executeWithRetries().

 3) change the and to use retries on SQL connections
and SQL executions. 


  trunk/common/src/java/org/apache/hadoop/hive/conf/ 1095959 
  trunk/conf/hive-default.xml 1095959 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ 1095959 
  trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/ 1095959

  trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/ 1095959 



Running unit tests. 



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message