hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From onur ascigil <onurasci...@hotmail.com>
Subject Hadoop Performance
Date Tue, 24 Nov 2009 05:52:11 GMT

I am running Hadoop on a single machine and have some questions about its performance.
I have a simple java program that runs breadth first search on a graph
with 5 nodes. It involves several map-reduce iterations. 

 I observed that, Hadoop takes too long to produce
results on such a simple job. So I attached a java profiler to my mapreduce job 
(runJar) to see what is going on. The java profiler reported several IPC 
connections to ports 54310 and 54311. Each of these IPCs to Jobtracker and 
HDFS takes around 10 seconds!

First of all why are these IPCs take this long? 
And I am wondering if there is anyway to improve
the performance of these IPC calls. Does Hadoop
have such a large fixed-cost ? 

I would really appreciate any comments or suggestions.
Thanks in advance,
Windows Live Hotmail: Your friends can get your Facebook updates, right from HotmailĀ®.
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message