hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongsheng Wang <phid...@yahoo.com>
Subject help needed for hadoop
Date Fri, 27 Apr 2007 19:47:46 GMT
One of my task is to calculate some statistics from a very large amount of log files for our
customers. We are trying out hadoop to solve this problem. 
>From what I can see, it is a perfect problem for hadoop designed to solve.

The mapper and reducer code are very straight ward. But when we try to run it on a two node
cluster, it is surprisingly slow. It has been running for three hours and did not finish half
of 250M log files. And, I am not seeing much disk or network usage.

I want to know if there is something I should check. (Maybe some configurations?)

Any help will be appreciated. If there is more information needed, let me know. 

Thanks in advance

       
---------------------------------
Ahhh...imagining that irresistible "new car" smell?
 Check outnew cars at Yahoo! Autos.
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message