hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From krish <kri22go...@gmail.com>
Subject Re: Row exception in Hive while using join
Date Mon, 09 Mar 2015 04:44:40 GMT
Hi Divakar,

I tried ur script, still getting same error. query hang after 68%.

script complete map,copy and shore phase as indicate in completion graph
but it hang on reduce phase. any one have any idea why  is it happening.

below is system log.
------------------------------
------------------------------
----------------------------------------------------------------------------------

syslog logs

INFO org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 135000000
rows
2015-03-09 10:06:14,477 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 135000000
rows
2015-03-09 10:06:15,311 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 136000000 rows
2015-03-09 10:06:15,311 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 136000000
rows
2015-03-09 10:06:15,918 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 137000000 rows
2015-03-09 10:06:15,918 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 137000000
rows
2015-03-09 10:06:16,524 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 138000000 rows
2015-03-09 10:06:16,525 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 138000000
rows
2015-03-09 10:06:17,146 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 139000000 rows
2015-03-09 10:06:17,380 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 139000000
rows
2015-03-09 10:06:18,019 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 140000000 rows
2015-03-09 10:06:18,361 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 140000000
rows
2015-03-09 10:06:18,989 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 141000000 rows
2015-03-09 10:06:19,249 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 141000000
rows
2015-03-09 10:06:20,749 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 142000000 rows
2015-03-09 10:06:20,876 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 142000000
rows
2015-03-09 10:06:22,703 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 143000000 rows
2015-03-09 10:06:22,703 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 143000000
rows
2015-03-09 10:06:23,354 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 144000000 rows
2015-03-09 10:06:23,354 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 144000000
rows
2015-03-09 10:06:24,190 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 145000000 rows
2015-03-09 10:06:24,685 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 145000000
rows
2015-03-09 10:06:25,320 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 146000000 rows
2015-03-09 10:06:25,670 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 146000000
rows
2015-03-09 10:06:26,680 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 147000000 rows
2015-03-09 10:06:26,680 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 147000000
rows
2015-03-09 10:06:27,357 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 148000000 rows
2015-03-09 10:06:27,357 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 148000000
rows
2015-03-09 10:06:28,000 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 149000000 rows
2015-03-09 10:06:28,024 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 149000000
rows
2015-03-09 10:06:28,884 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 150000000 rows
2015-03-09 10:06:28,884 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 150000000
rows
2015-03-09 10:06:30,687 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 151000000 rows
2015-03-09 10:06:31,585 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 151000000
rows
2015-03-09 10:06:32,573 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 152000000 rows
2015-03-09 10:06:32,573 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 152000000
rows
2015-03-09 10:06:33,643 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 153000000 rows
2015-03-09 10:06:33,643 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 153000000
rows
2015-03-09 10:06:34,473 INFO
org.apache.hadoop.hive.ql.exec.JoinOperator: 6 forwarding 154000000 rows
2015-03-09 10:06:34,473 INFO
org.apache.hadoop.hive.ql.exec.SelectOperator: 7 forwarding 154000000
rows
------------------------------------------------------------------------------------------


On Fri, Mar 6, 2015 at 10:59 AM, Divakar Reddy <divakarreddy.a@gmail.com>
wrote:

> Can you try this script?
>
> create table table_llv_N_C as select
> a.chromosome_number,
> a.position,
> b.id
> from (select * from table_line_n_passed where chromosome_number is not
> null) a
> join
> table_line_c_passed  b
> on
> a.chromosome_number=b.chromosome_number;
>
> On Thu, Mar 5, 2015 at 1:41 PM, Vikram Dixit <vikram@hortonworks.com>
> wrote:
>
>>  Hi Krish,
>>
>>  Which version of hive are you using? Can you also provide the logs for
>> this run?
>>
>>  Regards
>> Vikram.
>>
>>   From: krish <kri22gopal@gmail.com>
>> Reply-To: "user@hive.apache.org" <user@hive.apache.org>
>> Date: Thursday, March 5, 2015 at 3:14 AM
>> To: "user-info@hive.apache.org" <user-info@hive.apache.org>, "
>> user@hive.apache.org" <user@hive.apache.org>
>> Subject: Fwd: Row exception in Hive while using join
>>
>>
>> I got the following exception while executing join on Hive Query and
>> reducer hang after 68% completion.
>>
>>
>>  java.lang.RuntimeException:
>> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
>> processing row (tag=1)
>> {"key":{"joinkey0":"12"},"value":{"_col2":"rs317647905"},"alias":1}
>>         at
>> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:270)
>>         at
>> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:506)
>>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
>>         at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
>>         at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
>> Error while processing row (tag=1)
>> {"key":{"joinkey0":"12"},"value":{"_col2":"rs317647905"},"alias":1}
>>         at
>> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:258)
>>         ... 7 more
>> Caused by: org.apache.hadoop.
>>
>> ---------------------------------------------------------------------------------------
>>
>>  my query and table structure:
>>
>>  create table table_llv_N_C as select
>> table_line_n_passed.chromosome_number,table_line_n_passed.position,
>> table_line_c_passed.id from table_line_n_passed join table_line_c_passed
>> on
>> (table_line_n_passed.chromosome_number=table_line_c_passed.chromosome_number)
>>
>>  hive> desc table_line_n_passed;
>> OK
>> chromosome_number       string
>>
>>  position        int
>> id      string
>> ref     string
>> alt     string
>> quality double
>> filter  string
>> info    string
>> format  string
>> line6   string
>> Time taken: 0.854 seconds
>> Why am I getting this error, and how can I solve it?
>>
>>
>>
>>
>>
>>  --
>> with regards
>> krish!!!!!!
>>
>
>


-- 
with regards
krish!!!!!!

Mime
View raw message