hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith <hive.b...@gmail.com>
Subject Bug in Hive Partition windowing functions?
Date Tue, 29 Apr 2014 15:18:01 GMT
Hi, 

we have an issue with windowing function query never completed when
running against the large dataset > 25,000 rows. That is the reducer
(only one) never exit and it appears stuck in an infinite loop. 
I looked at the Reducer counter and it never changes over the 6 hours when it 
gets stuck in a loop.

When the data set is small < 25K rows, it runs fine.

Is there any work around this issue? We tested
against Hive 0.11/0.12/0.13 and the same result is the same.

create table window_function_fail
as
select a.*,
sum(case when bprice is not null then 1 else 0 end) over (partition by
date,name order by otime,bprice,aprice desc ROWS BETWEEN UNBOUNDED
PRECEDING AND CURRENT ROW) bidpid
from
large_table a;

create table large_table(
date               string    ,             
name             string   ,        
stime              string  ,          
bprice            decimal  ,              
aprice            decimal   ,             
otime             double                             
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' stored as textfile;

Thanks in advance.




Mime
View raw message