Return-Path: X-Original-To: apmail-hive-commits-archive@www.apache.org Delivered-To: apmail-hive-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CA220179F0 for ; Thu, 23 Apr 2015 02:34:13 +0000 (UTC) Received: (qmail 42584 invoked by uid 500); 23 Apr 2015 02:34:12 -0000 Delivered-To: apmail-hive-commits-archive@hive.apache.org Received: (qmail 42437 invoked by uid 500); 23 Apr 2015 02:34:12 -0000 Mailing-List: contact commits-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hive-dev@hive.apache.org Delivered-To: mailing list commits@hive.apache.org Received: (qmail 41409 invoked by uid 99); 23 Apr 2015 02:34:11 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Apr 2015 02:34:11 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 1F514E10A4; Thu, 23 Apr 2015 02:34:11 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: szehon@apache.org To: commits@hive.apache.org Date: Thu, 23 Apr 2015 02:34:21 -0000 Message-Id: In-Reply-To: <2eb239a642f448cbb6eb651785954663@git.apache.org> References: <2eb239a642f448cbb6eb651785954663@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [12/44] hive git commit: HIVE-9902 - Map join small table files need more replications [Spark Branch] (Jimmy via Chao) HIVE-9902 - Map join small table files need more replications [Spark Branch] (Jimmy via Chao) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1665384 13f79535-47bb-0310-9956-ffa450edef68 Project: http://git-wip-us.apache.org/repos/asf/hive/repo Commit: http://git-wip-us.apache.org/repos/asf/hive/commit/5e9f74ca Tree: http://git-wip-us.apache.org/repos/asf/hive/tree/5e9f74ca Diff: http://git-wip-us.apache.org/repos/asf/hive/diff/5e9f74ca Branch: refs/heads/master Commit: 5e9f74ca377b576d8f08912073577f6ae6569eb9 Parents: 4abe146 Author: Sun Chao Authored: Mon Mar 9 23:37:19 2015 +0000 Committer: Szehon Ho Committed: Wed Apr 22 19:33:50 2015 -0700 ---------------------------------------------------------------------- .../org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hive/blob/5e9f74ca/ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java ---------------------------------------------------------------------- diff --git a/ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java b/ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java index c3ba21a..94144a2 100644 --- a/ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java +++ b/ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java @@ -143,7 +143,7 @@ public class SparkHashTableSinkOperator } // TODO find out numOfPartitions for the big table int numOfPartitions = replication; - replication = (short) Math.min(MIN_REPLICATION, numOfPartitions); + replication = (short) Math.max(MIN_REPLICATION, numOfPartitions); } htsOperator.console.printInfo(Utilities.now() + "\tDump the side-table for tag: " + tag + " with group count: " + tableContainer.size() + " into file: " + path);