Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
MIME-Version: 1.0
In-Reply-To: <F5AC072B-3D8E-4C93-8228-1079556C5A07@gmail.com>
References: <F5AC072B-3D8E-4C93-8228-1079556C5A07@gmail.com>
From: goun na <gounna@gmail.com>
Date: Mon, 30 Jan 2017 20:24:04 +0900
Message-ID: <CADyiJTO7Scm4LUZnc7JGQz_Gch_FkVKrspTOwR1RR0PWZLmY5A@mail.gmail.com>
Subject: Re: Experimental results using TPC-DS (versus Spark and Presto)
To: user@hive.apache.org
Content-Type: multipart/alternative; boundary=94eb2c0b8454a54fff05474e0d2f
archived-at: Mon, 30 Jan 2017 11:24:12 -0000

--94eb2c0b8454a54fff05474e0d2f
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks for sharing benchmark results. May I ask why you choose ORC?

2017-01-30 19:57 GMT+09:00 =EA=B9=80=EB=8F=99=EC=9B=90 <eastcirclek@gmail.c=
om>:

> Hi,
>
> Recently I did some experiments using Hive, Spark, and Presto using TPC-D=
S
> benchmark
> and I'd like to share the result with the community: http://www.
> slideshare.net/ssuser6bb12d/hive-presto-and-spark-on-tpcds-benchmark
> I entirely depend on the benchmark kit from Hortonwork:
> https://github.com/hortonworks/hive-testbench
>
> Here I have a question about query 72.
> Hive LLAP shows better performance than Presto and Spark for most queries=
,
> but it shows very poor performance on the execution of query 72.
> While Presto also struggles with query 72, Spark finishes the execution o=
f
> query 72 a lot faster than Hive (page 9 and 10).
> I've observed a weird pattern in CPU utilization from Presto and Hive
> executing query 72 (page 11).
> When I turned off Spark's WholeStageCodeGen, Spark also takes a very long
> time to finish the execution of query 72 (page 12).
> Did I miss some feature of Hive to improve the performance of that kind o=
f
> query?
> I use the following setting for Hive experiments: https://github.
> com/hortonworks/hive-testbench/blob/hive14/sample-queries-tpcds/testbench=
.
> settings
>
> Except query 72, Hive with LLAP shows very good performance for both smal=
l
> and large workload anyway.
>
> - Dongwon Kim
>

--94eb2c0b8454a54fff05474e0d2f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks for sharing benchmark results. May I ask why you ch=
oose ORC?</div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">20=
17-01-30 19:57 GMT+09:00 =EA=B9=80=EB=8F=99=EC=9B=90 <span dir=3D"ltr">&lt;=
<a href=3D"mailto:eastcirclek@gmail.com" target=3D"_blank">eastcirclek@gmai=
l.com</a>&gt;</span>:<br><blockquote class=3D"gmail_quote" style=3D"margin:=
0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"word-=
wrap:break-word">Hi,<div><br></div><div>Recently I did some experiments usi=
ng Hive, Spark, and Presto using TPC-DS benchmark</div><div>and I&#39;d lik=
e to share the result with the community:=C2=A0<a href=3D"http://www.slides=
hare.net/ssuser6bb12d/hive-presto-and-spark-on-tpcds-benchmark" target=3D"_=
blank">http://www.<wbr>slideshare.net/ssuser6bb12d/<wbr>hive-presto-and-spa=
rk-on-<wbr>tpcds-benchmark</a></div><div>I entirely depend on the benchmark=
 kit from Hortonwork: <a href=3D"https://github.com/hortonworks/hive-testbe=
nch" target=3D"_blank">https://github.com/<wbr>hortonworks/hive-testbench</=
a></div><div><br></div><div>Here I have a question about query 72.</div><di=
v>Hive LLAP shows better performance than Presto and Spark for most queries=
, but it shows very poor performance on the execution of query 72.</div><di=
v>While Presto also struggles with query 72, Spark finishes the execution o=
f query 72 a lot faster than Hive (page 9 and 10).</div><div>I&#39;ve obser=
ved a weird pattern in CPU utilization from Presto and Hive executing query=
 72 (page 11).</div><div>When I turned off Spark&#39;s WholeStageCodeGen, S=
park also takes a very long time to finish the execution of query 72 (page =
12).</div><div>Did I miss some feature of Hive to improve the performance o=
f that kind of query?</div><div>I use the following setting for Hive experi=
ments:=C2=A0<a href=3D"https://github.com/hortonworks/hive-testbench/blob/h=
ive14/sample-queries-tpcds/testbench.settings" target=3D"_blank">https://gi=
thub.<wbr>com/hortonworks/hive-<wbr>testbench/blob/hive14/sample-<wbr>queri=
es-tpcds/testbench.<wbr>settings</a></div><div><br></div><div>Except query =
72, Hive with LLAP shows very good performance for both small and large wor=
kload anyway.</div><div><br></div><div>- Dongwon Kim</div></div></blockquot=
e></div><br></div>

--94eb2c0b8454a54fff05474e0d2f--