hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <praveen.pe...@nokia.com>
Subject RE: How to speed up of Map/Reduce job?
Date Tue, 01 Feb 2011 15:08:55 GMT
Hi Igor,
I am not sure if Hadoop is designed for realtime requests. I have a feeling that you are trying
to use Hadoop in a way that it isnot designed for. From my experience, Hadoop cluster will
be much slower than "local" hadoop mode when processing smaller dataset, because there is
always extra overhead of task and job management in cluster mode.

From: ext Igor Bubkin [igba14@gmail.com]
Sent: Tuesday, February 01, 2011 3:19 AM
To: common-issues@hadoop.apache.org
Cc: common-user@hadoop.apache.org
Subject: How to speed up of Map/Reduce job?

Hello everybody

I have a problem. I installed Hadoop on 2-nodes cluster and run Wordcount
example. It takes about 20 sec for processing of 1,5MB text file. We want to
use Map/Reduce in real time (interactive: by user's requests). User can't
wait for his request 20 sec. This is too long. Is it possible to reduce time
of Map/Reduce job? Or may be I misunderstand something?

Igor Babkin, Mifors.com

View raw message