hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikram Dixit <vik...@hortonworks.com>
Subject Re: issue with Tez execution engine in HDP 2.2
Date Tue, 16 Dec 2014 00:37:24 GMT
Hi Kristen,

I discussed this with tez folks and they tell me the version of tez
you have should work with that version of hive. It seems like the
request for container could have been very large (or -1 perhaps). Can
you share the hive logs as well as the application logs for this
issue. Probably worth a jira ticket for tracking purposes.

Thanks
Vikram.

On Mon, Dec 15, 2014 at 11:01 AM, Kristen Hardwick
<khardwick@spryinc.com> wrote:
> Vikram,
>
> Thanks for the quick response. I am using the following versions:
>
> Hive  0.14.0.2.2.0.0
> Tez  0.5.2.2.2.0.0
>
> Do you know where I can find a Hortonworks RPM for Tez 0.5.3 so that I can
> upgrade manually? I used the Ambari installation from the Hortonworks site,
> so I think the version conflict is built into what is available for
> download.
>
> Thanks!
> Kristen
>
> On Mon, Dec 15, 2014 at 1:29 PM, Vikram Dixit <vikram@hortonworks.com>
> wrote:
>>
>> Hi Kristen,
>>
>> Can you share the version of tez you are using? Hive 0.14 (in HDP 2.2)
>> needs tez 0.5.3 or beyond. Not sure if there was an earlier
>> installation of tez that is being used here.
>>
>> Thanks
>> Vikram.
>>
>> On Mon, Dec 15, 2014 at 10:13 AM, Kristen Hardwick
>> <khardwick@spryinc.com> wrote:
>> > Hi all,
>> >
>> > We recently created a new Amazon cluster with an installation of GA HDP
>> > 2.2
>> > from the Hortonworks website. One of the main reasons we wanted to try
>> > this
>> > version was to use Hive with Tez with the new cost based optimizer.
>> > However,
>> > running simple queries with Tez as the execution engine does not work.
>> >
>> > I created a single line file, created the Hive table for it, and ran
>> > this
>> > query to confirm proper data load:
>> >
>> > select * from test;
>> >
>> > No issues, so I ran this query with the "mr" execution engine, and had
>> > success:
>> >
>> > select count(*) from test; -- 1
>> >
>> > Then I ran the same query with the "tez" execution engine, and had no
>> > success. I got the following error:
>> >
>> > Status: Running (Executing on YARN cluster with App id
>> > application_1418140477139_0067)
>> >
>> >
>> > --------------------------------------------------------------------------------
>> > VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
>> >
>> > --------------------------------------------------------------------------------
>> > Map 1 FAILED 1 0 0 1 4 0
>> > Reducer 2 KILLED 1 0 0 1 0 1
>> >
>> > --------------------------------------------------------------------------------
>> > VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 9.61 s
>> >
>> > --------------------------------------------------------------------------------
>> > Status: Failed
>> > Vertex failed, vertexName=Map 1,
>> > vertexId=vertex_1418140477139_0067_1_00,
>> > diagnostics=[Task failed, taskId=task_1418140477139_0067_1_00_000000,
>> > diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running
>> > task:java.lang.IllegalArgumentException
>> > at
>> > com.google.common.base.Preconditions.checkArgument(Preconditions.java:72)
>> > at
>> >
>> > org.apache.tez.runtime.common.resources.MemoryDistributor.registerRequest(MemoryDistributor.java:158)
>> > at
>> >
>> > org.apache.tez.runtime.common.resources.MemoryDistributor.requestMemory(MemoryDistributor.java:97)
>> > at
>> >
>> > org.apache.tez.runtime.api.impl.TezTaskContextImpl.requestInitialMemory(TezTaskContextImpl.java:192)
>> > at
>> >
>> > org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput.initialize(OrderedPartitionedKVOutput.java:95)
>> >
>> > Have any of you seen this issue before? That "requestInitialMemory"
>> > function
>> > failure combined with the "IllegalArgumentException" message made me
>> > think
>> > that maybe Tez was asking for a container that was too big...but I
>> > didn't
>> > see anything odd in the properties. Then again, I don't have a lot of
>> > insight into what this new version of Tez requires.
>> >
>> > Any help is appreciated! Please let me know if you need additional
>> > information.
>> > Kristen Hardwick
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message