Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14B5817A17 for ; Mon, 6 Apr 2015 17:17:02 +0000 (UTC) Received: (qmail 65099 invoked by uid 500); 6 Apr 2015 17:17:00 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 65037 invoked by uid 500); 6 Apr 2015 17:17:00 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 65027 invoked by uid 99); 6 Apr 2015 17:17:00 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Apr 2015 17:17:00 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of oracle.blog3@gmail.com designates 209.85.213.180 as permitted sender) Received: from [209.85.213.180] (HELO mail-ig0-f180.google.com) (209.85.213.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Apr 2015 17:16:55 +0000 Received: by igbqf9 with SMTP id qf9so24550804igb.1 for ; Mon, 06 Apr 2015 10:15:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bcSsNOeDHGzARuyLqWLL1HgtVCHrfFQwLnrgie3NrVg=; b=aVqA6lc8r7QIjvXPoCiS5syl16v+I2gVJkDjYEweP9NKznSiM8b70EMok6FgHCAAoR tpBCkTRUVNMlylxu5x30RJ3ISrMpMjeYanm2M8ZRJVg/6bnEDvwcOzHSwyCHL6RUEPbf E9u39Z5gkSjspPleIfw2iX6o5BlyoJKW8H0F24QpZE9pVuhd3Euvu6hL+/at+foBaiV8 T2kE8V5T1oe4ZT7WchD9NpMOOeA8HR2g1Y/arBmawHSabCuuBM9huwFuU35MxG/B9bcd w20mXNMMSbpBF1qDTN/a8zMGMm/94AgIzw2e0lhsk0oJZ5ENoWB08dc7JQppDMVavXtE fL2w== MIME-Version: 1.0 X-Received: by 10.107.164.209 with SMTP id d78mr23958592ioj.73.1428340505077; Mon, 06 Apr 2015 10:15:05 -0700 (PDT) Received: by 10.107.3.32 with HTTP; Mon, 6 Apr 2015 10:15:05 -0700 (PDT) In-Reply-To: <20150402160153.GE26725@voyageur.mas.nom.fr> References: <20150402160153.GE26725@voyageur.mas.nom.fr> Date: Mon, 6 Apr 2015 12:15:05 -0500 Message-ID: Subject: Re: Hive and engine performance tez vs mr From: max scalf To: Hive Mailing List Content-Type: multipart/alternative; boundary=001a114220d06c6e8f05131170b4 X-Virus-Checked: Checked by ClamAV on apache.org --001a114220d06c6e8f05131170b4 Content-Type: text/plain; charset=UTF-8 Try setting the below in Hive and see what happens..btw what are you configs in hive if any? set mapred.map.tasks = 20; On Thu, Apr 2, 2015 at 11:01 AM, Erwan MAS wrote: > Hello , > > I have a issue on hive , with tez engine . When try to execute a query , > with > tez engine , the query is 9 times slower than map/reduce . > > The query is a left outer join on two table using orc storage . > > With map/reduce i have : > Job 0 : Map 27 Reduce 256 > Job 1 : Map 27 Reduce 256 > Time taken 110 sec > > With tez i have : > Map 1 : 1/1 Map 4 : 3/3 Reducer 2: 256/256 Reducer 3: 256/256 > Time taken 930 sec > > With my configuration tez want to use only one mapper for some part . > > How to increase this number of mapper ? > Which variable on hive , i must set to change this behavior ? > > My context : > Hive 0.13 on Hortonworks 2.1 > > -- > ____________________________________________________________ > / Erwan MAS /\ > | mailto:erwan@mas.nom.fr |_/ > ___|________________________________________________________ | > \___________________________________________________________\__/ > --001a114220d06c6e8f05131170b4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Try setting the below in Hive an= d see what happens..btw what are you configs in hive if any?
set mapred.map.tasks =3D 20;
=

On Thu, Apr= 2, 2015 at 11:01 AM, Erwan MAS <erwan@mas.nom.fr> wrote:
=
Hello ,

I have a issue on hive , with tez engine . When=C2=A0 try to execute a quer= y , with
tez engine , the query is 9 times slower than map/reduce .

The query is a left outer join on two table using orc storage .

With map/reduce i have=C2=A0 :
Job 0 : Map 27 Reduce 256
Job 1 : Map 27 Reduce 256
Time taken 110 sec

With tez i have :
Map 1 :=C2=A0 1/1 Map 4 : 3/3 Reducer 2: 256/256 Reducer 3: 256/256
Time taken 930 sec

With my configuration tez want to use only one mapper for some part .

How to increase this number of mapper ?
Which variable on hive , i must set to change this behavior=C2=A0 ?

My context :
=C2=A0 =C2=A0Hive 0.13 on Hortonworks 2.1

--
=C2=A0 =C2=A0 =C2=A0_______________________________________________________= _____
=C2=A0 =C2=A0 / Erwan MAS=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/\
=C2=A0 =C2=A0| mailto:erwan@mas.nom.fr<= /a>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|_/
___|________________________________________________________=C2=A0 =C2=A0|<= br> \___________________________________________________________\__/

--001a114220d06c6e8f05131170b4--