hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijay <tec...@gmail.com>
Subject Re: Hive too slow?
Date Tue, 08 Mar 2011 06:10:45 GMT
If you go to the jobtracker's web UI, it provides plenty of details
about each job. Even with all the default settings of a typical
hadoop/hive installation, 4 minutes for 2200 rows is extremely slow.
It feels like there is some kind of problem but it is hard to guess
what that could be. Digging through the web UI can tell you how much
time is spent on map vs reduce. It can also provide some insight into
the I/O operations.

Good luck!
-Vijay

On Mon, Mar 7, 2011 at 9:24 PM, abhishek pathak
<forever_yours_abhi@yahoo.co.in> wrote:
> I suspected as such.My system is a Core2Duo,1.86 Ghz.I understand that
> map-reduce is not instantaneous, just wanted to confirm that 2200 rows in 4
> minutes is indeeed not normal behaviour.Could you point me at some places
> where i can get some info on how to tune this up?
> Regards,
> Abhishek
> ________________________________
> From: Ajo Fod <ajo.fod@gmail.com>
> To: user@hive.apache.org
> Sent: Mon, 7 March, 2011 9:21:51 PM
> Subject: Re: Hive too slow?
>
> In my experience, hive is not instantaneous like other DBs, but 4 minutes to
> count 2200 rows seems unreasonable.
>
> For comparison my query of 169k rows one one computer with 4 cores running
> 1Ghz (approx) took 20 seconds.
>
> Cheers,
> Ajo.
>
> On Mon, Mar 7, 2011 at 1:19 AM, abhishek pathak
> <forever_yours_abhi@yahoo.co.in> wrote:
>>
>> Hi,
>> I am a hive newbie.I just finished setting up hive on a cluster of two
>> servers for my organisation.As a test drill, we operated some simple
>> queries.It took the standard map-reduce algorithm around 4 minutes just to
>> execute this query:
>> count(1) from tablename;
>> The answer returned was around 2200.Clearly, this is not a big number by
>> hadoop standards.My question is whether this is a standard performance or is
>> there some configuration that is not optimised?Will scaling up of data to
>> say,50 times, produce any drastic slowness?I tried reading the documentation
>> but was not clear on these issues, and i would like to have an idea before
>> this setup starts working in a production environment.
>> Thanks in advance,
>> Regards,
>> Abhishek Pathak
>>
>>
>
>
>

Mime
View raw message