hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Per Jacobsson" <...@pjacobsson.com>
Subject Re: Merging of the local FS files threw an exception
Date Wed, 01 Oct 2008 19:04:39 GMT
I've collected the syslogs from the failed reduce jobs.  What's the best way
to get them to you? Let me know if you need anything else, I'll have to shut
down these instances some time later today.

Overall I've run this same job before with no problems. The only change is
the added gzip of the output. Don't know if it's worth anything, but the
four failures all happened on different machines. I'll be running this job
plenty of times so if the problem keeps happening it will be obvious.
/ Per

On Wed, Oct 1, 2008 at 11:23 AM, Arun C Murthy <acm@yahoo-inc.com> wrote:

>
> Do you still have the task logs for the reduce?
>
> I suspect are running into
> http://issues.apache.org/jira/browse/HADOOP-3647 which we never could
> reproduce reliably to pin it down or fix.
>
> However, in light of http://issues.apache.org/jira/browse/HADOOP-4277 we
> suspect this could be caused by a bug in the LocalFileSystem which could
> hide data-corruption on your local disk leading to errors on these nature.
> Could you try running your job with that patch once the release 0.18.2 is
> available?
>
> Any information you provide could greatly aid to confirm our above
> hypothesis, so it's much appreciated!
>
> Arun
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message