hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongsheng Wang <phid...@yahoo.com>
Subject help needed for hadoop
Date Fri, 27 Apr 2007 19:47:46 GMT
One of my task is to calculate some statistics from a very large amount of log files for our
customers. We are trying out hadoop to solve this problem. 
>From what I can see, it is a perfect problem for hadoop designed to solve.

The mapper and reducer code are very straight ward. But when we try to run it on a two node
cluster, it is surprisingly slow. It has been running for three hours and did not finish half
of 250M log files. And, I am not seeing much disk or network usage.

I want to know if there is something I should check. (Maybe some configurations?)

Any help will be appreciated. If there is more information needed, let me know. 

Thanks in advance

Ahhh...imagining that irresistible "new car" smell?
 Check outnew cars at Yahoo! Autos.
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message