hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajo Fod <ajo....@gmail.com>
Subject Re: Is there a reason why this simple query would take a very long time?
Date Mon, 24 Jan 2011 21:54:02 GMT
you could try to set the number of reducers e.g:
set mapred.reduce.tasks=4;

set this before doing the select.

-Ajo

On Mon, Jan 24, 2011 at 1:13 PM, Jonathan Coveney <jcoveney@gmail.com> wrote:
> I have a 10 node server or so, and have been mainly using pig on it, but
> would like to try out Hive.
> I am running this query, which doesn't take too long in Pig, but is taking
> quite a long time in Hive.
>
> hive -e "select count(1) as ct from my_table where v1='02' and v2 =
> 11112222;" > thecount
> One thing is that this job only uses 1 reducer, but it is taking most of its
> time in its reduce step. I tried manually setting more reducers, but I think
> that for a job without groups, it forces 1 reducer?
> Either way, would love to know why this is dragging? It's worth noting that
> my_table is not saved in the Hive format, but rather as a flat file. I
> realize that this can influence performance, but shouldn't it at least
> perform on par with pig?
> Thanks for your help
> Jon

Mime
View raw message