hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristen Hardwick <khardw...@spryinc.com>
Subject Re: issue with Tez execution engine in HDP 2.2
Date Mon, 15 Dec 2014 19:01:35 GMT
Vikram,

Thanks for the quick response. I am using the following versions:

Hive  0.14.0.2.2.0.0
Tez  0.5.2.2.2.0.0

Do you know where I can find a Hortonworks RPM for Tez 0.5.3 so that I can
upgrade manually? I used the Ambari installation from the Hortonworks site,
so I think the version conflict is built into what is available for
download.

Thanks!
Kristen

On Mon, Dec 15, 2014 at 1:29 PM, Vikram Dixit <vikram@hortonworks.com>
wrote:
>
> Hi Kristen,
>
> Can you share the version of tez you are using? Hive 0.14 (in HDP 2.2)
> needs tez 0.5.3 or beyond. Not sure if there was an earlier
> installation of tez that is being used here.
>
> Thanks
> Vikram.
>
> On Mon, Dec 15, 2014 at 10:13 AM, Kristen Hardwick
> <khardwick@spryinc.com> wrote:
> > Hi all,
> >
> > We recently created a new Amazon cluster with an installation of GA HDP
> 2.2
> > from the Hortonworks website. One of the main reasons we wanted to try
> this
> > version was to use Hive with Tez with the new cost based optimizer.
> However,
> > running simple queries with Tez as the execution engine does not work.
> >
> > I created a single line file, created the Hive table for it, and ran this
> > query to confirm proper data load:
> >
> > select * from test;
> >
> > No issues, so I ran this query with the "mr" execution engine, and had
> > success:
> >
> > select count(*) from test; -- 1
> >
> > Then I ran the same query with the "tez" execution engine, and had no
> > success. I got the following error:
> >
> > Status: Running (Executing on YARN cluster with App id
> > application_1418140477139_0067)
> >
> >
> --------------------------------------------------------------------------------
> > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
> >
> --------------------------------------------------------------------------------
> > Map 1 FAILED 1 0 0 1 4 0
> > Reducer 2 KILLED 1 0 0 1 0 1
> >
> --------------------------------------------------------------------------------
> > VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 9.61 s
> >
> --------------------------------------------------------------------------------
> > Status: Failed
> > Vertex failed, vertexName=Map 1, vertexId=vertex_1418140477139_0067_1_00,
> > diagnostics=[Task failed, taskId=task_1418140477139_0067_1_00_000000,
> > diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running
> > task:java.lang.IllegalArgumentException
> > at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:72)
> > at
> >
> org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:158)
> > at
> >
> org.apache.tez.runtime.common.resources.MemoryDistributor.requestMemory(MemoryDistributor.java:97)
> > at
> >
> org.apache.tez.runtime.api.impl.TezTaskContextImpl.requestInitialMemory(TezTaskContextImpl.java:192)
> > at
> >
> org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.initialize(OrderedPartitionedKVOutput.java:95)
> >
> > Have any of you seen this issue before? That "requestInitialMemory"
> function
> > failure combined with the "IllegalArgumentException" message made me
> think
> > that maybe Tez was asking for a container that was too big...but I didn't
> > see anything odd in the properties. Then again, I don't have a lot of
> > insight into what this new version of Tez requires.
> >
> > Any help is appreciated! Please let me know if you need additional
> > information.
> > Kristen Hardwick
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
View raw message