hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Kim <benkimkim...@gmail.com>
Subject Re: HIve temp file
Date Sun, 19 Jan 2014 23:05:53 GMT
The problem was that my custom built UDAF with subqueries caused all UDAF
logics to run only on reducers.
I created a jira relating to this
(HIVE-6230<https://issues.apache.org/jira/browse/HIVE-6230>
)

*Benjamin Kim*
*benkimkimben at gmail*


On Tue, Jan 14, 2014 at 3:25 PM, Ben Kim <benkimkimben@gmail.com> wrote:

> Hello!
>
> When i run hive job, it creates large temporary files on
>
> /tmp/hive-ben/hive_2014-01-14_15-01-39_521_3015861149916225685-1/
>
> somewhere around 300GB. this number tends to get larger if i use lower
> number of reducers.
>
> with 5 reducers the size goes up to 1TB
>
> my input files are total 1GB, but it's distributed with large number of
> keys on MR.
>
>
> what are these temp files for?
>
> Thanks!
>
> *Benjamin Kim*
> *benkimkimben at gmail*
>

Mime
View raw message