hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: "lost task" suspect Distributed Cache to blame
Date Tue, 02 Mar 2010 23:55:15 GMT

On 3/2/10 12:49 PM, "Edward Capriolo" <edlinuxguru@gmail.com> wrote:

> This job is somewhat special (for us) in that in involves shipping
> large files over distributed cache. My working theory is that
> something goes wrong with the distributed cache/job tracker and the
> Job/Task/Tips never have a chance.
> Has anyone ever experienced something like this?

How big is big and what archiving format are you using?  Jar has issues with
files >2gb.

I've also noticed that dcache seems to get written to -every- tmp dir on a
node, which seems a bit ridiculous.

View raw message