hadoop-hive-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From na...@apache.org
Subject svn commit: r905489 - in /hadoop/hive/trunk: CHANGES.txt conf/hive-default.xml
Date Tue, 02 Feb 2010 01:13:58 GMT
Author: namit
Date: Tue Feb  2 01:13:58 2010
New Revision: 905489

URL: http://svn.apache.org/viewvc?rev=905489&view=rev
Log:
HIVE-1119. Change default size of average size to 16M
(Zheng Shao via namit)


Modified:
    hadoop/hive/trunk/CHANGES.txt
    hadoop/hive/trunk/conf/hive-default.xml

Modified: hadoop/hive/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/hive/trunk/CHANGES.txt?rev=905489&r1=905488&r2=905489&view=diff
==============================================================================
--- hadoop/hive/trunk/CHANGES.txt (original)
+++ hadoop/hive/trunk/CHANGES.txt Tue Feb  2 01:13:58 2010
@@ -109,6 +109,9 @@
 
     HIVE-1119. Serialize all tasks (Zheng Shao via namit)
 
+    HIVE-1119. Change default size of average size to 16M
+    (Zheng Shao via namit)
+
 Release 0.5.0 -  Unreleased
 
   INCOMPATIBLE CHANGES

Modified: hadoop/hive/trunk/conf/hive-default.xml
URL: http://svn.apache.org/viewvc/hadoop/hive/trunk/conf/hive-default.xml?rev=905489&r1=905488&r2=905489&view=diff
==============================================================================
--- hadoop/hive/trunk/conf/hive-default.xml (original)
+++ hadoop/hive/trunk/conf/hive-default.xml Tue Feb  2 01:13:58 2010
@@ -386,7 +386,7 @@
 <property>
   <name>hive.merge.mapredfiles</name>
   <value>false</value>
-  <description>Merge small files at the end of any job(map only or map-reduce)</description>
+  <description>Merge small files at the end of a map-reduce job</description>
 </property>
 
 <property>
@@ -402,6 +402,12 @@
 </property>
 
 <property>
+  <name>hive.merge.size.smallfiles.avgsize</name>
+  <value>16000000</value>
+  <description>When the average output file size of a job is less than this number,
Hive will start an additional map-reduce job to merge the output files into bigger files.
 This is only done for map-only jobs if hive.merge.mapfiles is true, and for map-reduce jobs
if hive.merge.mapredfiles is true.</description>
+</property>
+
+<property>
   <name>hive.script.auto.progress</name>
   <value>false</value>
   <description>Whether Hive Tranform/Map/Reduce Clause should automatically send progress
information to TaskTracker to avoid the task getting killed because of inactivity.  Hive sends
progress information when the script is outputting to stderr.  This option removes the need
of periodically producing stderr messages, but users should be cautious because this may prevent
infinite loops in the scripts to be killed by TaskTracker.  </description>



Mime
View raw message