Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: pass (nike.apache.org: domain of hive.bugs@gmail.com designates
 209.85.192.68 as permitted sender)
Message-ID: <535FC2A9.6030502@gmail.com>
Date: Tue, 29 Apr 2014 11:18:01 -0400
From: Keith <hive.bugs@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.4.0
MIME-Version: 1.0
To: user@hive.apache.org
Subject: Bug in Hive Partition windowing functions?
References: <535FB77A.5060208@gmail.com>
In-Reply-To: <535FB77A.5060208@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hi, 

we have an issue with windowing function query never completed when
running against the large dataset > 25,000 rows. That is the reducer
(only one) never exit and it appears stuck in an infinite loop. 
I looked at the Reducer counter and it never changes over the 6 hours when it 
gets stuck in a loop.

When the data set is small < 25K rows, it runs fine.

Is there any work around this issue? We tested
against Hive 0.11/0.12/0.13 and the same result is the same.

create table window_function_fail
as
select a.*,
sum(case when bprice is not null then 1 else 0 end) over (partition by
date,name order by otime,bprice,aprice desc ROWS BETWEEN UNBOUNDED
PRECEDING AND CURRENT ROW) bidpid
from
large_table a;

create table large_table(
date               string    ,             
name             string   ,        
stime              string  ,          
bprice            decimal  ,              
aprice            decimal   ,             
otime             double                             
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' stored as textfile;

Thanks in advance.