hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <jason.had...@gmail.com>
Subject Re: "Timed out waiting for rpc response" after running a large number of jobs
Date Sun, 20 Sep 2009 01:55:08 GMT
It is not uncommon for the task tracker http servers to get overwhelmed with
requests for map outputs, when there where many map tasks.
Increasing the number of threads can help.

On Sat, Sep 19, 2009 at 6:32 PM, Kunsheng Chen <keyek@yahoo.com> wrote:

> Hi everyone,
>
>
> I am running two map-reduce program, they were working good but when the
> data turns into around 900MB (50000+ files). things weird happen to remind
> me as below:
>
>
>  'Communication problem with server: java.net.SocketTimeoutException: timed
> out waiting for rpc response'
>
> Also there is some other reminder like "fail to allocate memory".
>
>
> Strange is that the program keeps running and shows mapping and reduce
> percentage after those errors....seems it is still progressing in a slow
> pace.
>
>
> Does anyone have some idea ?
>
>
>
> Thanks,
>
> -Kun
>
>
>
>
>
>
>


-- 
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message