Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 46143B313 for ; Thu, 12 Jan 2012 23:03:34 +0000 (UTC) Received: (qmail 89769 invoked by uid 500); 12 Jan 2012 23:03:33 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 89692 invoked by uid 500); 12 Jan 2012 23:03:32 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 89684 invoked by uid 99); 12 Jan 2012 23:03:32 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2012 23:03:32 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.213.176] (HELO mail-yx0-f176.google.com) (209.85.213.176) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Jan 2012 23:03:24 +0000 Received: by yenq7 with SMTP id q7so1458702yen.35 for ; Thu, 12 Jan 2012 15:03:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.136.99 with SMTP id v63mr723452yhi.46.1326409383894; Thu, 12 Jan 2012 15:03:03 -0800 (PST) Received: by 10.236.184.3 with HTTP; Thu, 12 Jan 2012 15:03:03 -0800 (PST) X-Originating-IP: [96.10.173.94] In-Reply-To: <4F0F660D.6020401@gmail.com> References: <4F0F660D.6020401@gmail.com> Date: Thu, 12 Jan 2012 18:03:03 -0500 Message-ID: Subject: Re: why 1 reducer on simple join? From: Koert Kuipers To: user@hive.apache.org Content-Type: multipart/alternative; boundary=485b397dd125275d3d04b65cc0d7 --485b397dd125275d3d04b65cc0d7 Content-Type: text/plain; charset=ISO-8859-1 I tried set mapred.reduce.tasks = xyz; hive ignored it. Selects run fine. The query uses 44 mappers. On Thu, Jan 12, 2012 at 6:00 PM, Wojciech Langiewicz wrote: > Hello, > Have you tried running only select, without creating table? What are > results? > How did you tried to set number of reducers? Have you used this: > set mapred.reduce.tasks = xyz; > How many mappers does this query use? > > > On 12.01.2012 23:53, Koert Kuipers wrote: > >> I am running a basic join of 2 tables and it will only run with 1 reducer. >> why is that? i tried to set the number of reducers and it didn't work. >> hive >> just ignored it. >> >> create table z as select x.* from table1 x join table2 y where ( >> x.col1 = y.col1 and >> x.col2 = y.col2 and >> x.col3 = y.col3 and >> x.col4 = y.col4 and >> x.col5 = y.col5 >> ); >> >> both tables are backed by multiple files / blocks / chunks >> >> > -- > Wojciech Langiewicz > --485b397dd125275d3d04b65cc0d7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I tried set mapred.reduce.tasks =3D xyz; hive ignored it.
Selects ru= n fine. The query uses 44 mappers.

On Thu= , Jan 12, 2012 at 6:00 PM, Wojciech Langiewicz <wlangiewicz@gmail.com> wro= te:

Hello,
Have you tried running only select, without creating table? What are result= s?
How did you tried to set number of reducers? Have you used this:
set mapred.reduce.tasks =3D xyz;
How many mappers does this query use?

On 12.01.2012 23:53, Koert Kuipers wrote:

I am running a basic join of 2 tables and it will only run with 1 reducer.<= br> why is that? i tried to set the number of reducers and it didn't work. = hive
just ignored it.

create table z as select x.* from table1 x join table2 y where (
x.col1 =3D y.col1 and
x.col2 =3D y.col2 and
x.col3 =3D y.col3 and
x.col4 =3D y.col4 and
x.col5 =3D y.col5
);

both tables are backed by multiple files / blocks / chunks

--
Wojciech Langiewicz

--485b397dd125275d3d04b65cc0d7--