hive-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From x..@apache.org
Subject hive git commit: HIVE-16004: OutOfMemory in SparkReduceRecordHandler with vectorization mode (Colin Ma, reviewed by Ferdinand Xu, Xuefu Zhang)
Date Fri, 24 Feb 2017 02:21:21 GMT
Repository: hive
Updated Branches:
  refs/heads/master 79580a64b -> 27683610f


HIVE-16004: OutOfMemory in SparkReduceRecordHandler with vectorization mode (Colin Ma, reviewed
by Ferdinand Xu, Xuefu Zhang)


Project: http://git-wip-us.apache.org/repos/asf/hive/repo
Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/27683610
Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/27683610
Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/27683610

Branch: refs/heads/master
Commit: 27683610f0665fe05e59d3cf5130fe33ecd81e77
Parents: 79580a6
Author: Ferdinand Xu <cheng.a.xu@intel.com>
Authored: Fri Feb 24 02:41:19 2017 +0800
Committer: Ferdinand Xu <cheng.a.xu@intel.com>
Committed: Fri Feb 24 02:41:19 2017 +0800

----------------------------------------------------------------------
 .../apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java  | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hive/blob/27683610/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java
----------------------------------------------------------------------
diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java
b/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java
index 8251900..27bed9c 100644
--- a/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkReduceRecordHandler.java
@@ -364,6 +364,7 @@ public class SparkReduceRecordHandler extends SparkRecordHandler {
   private <E> boolean processVectors(Iterator<E> values, byte tag) throws HiveException
{
     VectorizedRowBatch batch = batches[tag];
     batch.reset();
+    buffer.reset();
 
     /* deserialize key into columns */
     VectorizedBatchUtil.addRowToBatchFrom(keyObject, keyStructInspector, 0, 0, batch, buffer);


Mime
View raw message