hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Baldeschwieler <eri...@yahoo-inc.com>
Subject Re: question on Hadoop configuration for non cpu intensive jobs - 0.15.1
Date Fri, 28 Dec 2007 18:13:22 GMT
I created HADOOP-2497 to describe this bug.

Was your sequence file stored on HDFS?  Because HDFS does provide  
checksums.

On Dec 28, 2007, at 7:20 AM, Jason Venner wrote:

> Our OOM was being caused by a damaged sequence data file. We had  
> assumed that the sequence files had checksums, which appears to be  
> in correct.
> The deserializer was reading a bad length out of the file and  
> trying to allocate 4gig of ram.


Mime
View raw message