Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Message-ID: <54B0366A.6030803@apache.org>
Date: Fri, 09 Jan 2015 12:13:30 -0800
From: Gopal V <gopalv@apache.org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: dev@hive.apache.org
CC: Alexander Pivovarov <apivovarov@gmail.com>
Subject: Re: hive 0.14.1 + tez 0.4 sometimes throws NPE on small data
References: 
 <CAKKt98Sn=+Og5CRnk-C0m+YEGnRxOChXkRxxHQCq5Rpbo7n3Cw@mail.gmail.com>
In-Reply-To: 
 <CAKKt98Sn=+Og5CRnk-C0m+YEGnRxOChXkRxxHQCq5Rpbo7n3Cw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 1/9/15, 11:53 AM, Alexander Pivovarov wrote:
> I noticed that if amount of data which should be sent to reducers is small
> then tez might throw NPE.

I noticed that you sent it to the tez lists as well. This is a hive 
specific issue.

https://issues.apache.org/jira/browse/HIVE-8361

to be precise (from the following backtrace)

> Caused by: java.lang.NullPointerException
>          at
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:341)
>          at
> org.apache.hadoop.hive.ql.exec.PTFOperator.processOp(PTFOperator.java:115)
>          at
> org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793)
>          at
> org.apache.hadoop.hive.ql.exec.ExtractOperator.processOp(ExtractOperator.java:45)
>          at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.processKeyValues(ReduceRecordProcessor.java:325)
>          ... 8 more

HTH.

Cheers,
Gopal