hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Venugopal" <...@andrew.cmu.edu>
Subject Re: Could not obtain block: blk_-2634319951074439134_1129 file=/user/root/crawl_debug/segments/20080825053518/content/part-00002/data
Date Thu, 28 Aug 2008 15:20:22 GMT
Its slightly counterintuitive, but I used to get errors like this when my
reducers would run out of memory. Turns out that if a reducer uses up too
much memory and brings down a node, that it could also kill the services
that are making map data available to other reducers. I cant explain exactly
why this exact error happens, but I have found that the culprit is often
memory usage (normally in the reducer).
Ashish

On Thu, Aug 28, 2008 at 7:59 AM, Jason Venner <jason@attributor.com> wrote:

> We have started to see this class of error under hadoop 0.16.1 on a
> medium sized hdfs cluster under moderate load
>
> wangxu wrote:
> > Hi,all
> > I am using hadoop-0.18.0-core.jar and nutch-2008-08-18_04-01-55.jar,
> > and running hadoop on one namenode and 4 slaves.
> > attached is my hadoop-site.xml, and I didn't change the file
> > hadoop-default.xml
> >
> > when data in segments are large,this kind of errors occure:
> >
> > java.io.IOException: Could not obtain block:
> blk_-2634319951074439134_1129
> file=/user/root/crawl_debug/segments/20080825053518/content/part-00002/data
> >       at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1462)
> >       at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1312)
> >       at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1417)
> >       at java.io.DataInputStream.readFully(DataInputStream.java:178)
> >       at
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:64)
> >       at
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:102)
> >       at
> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1646)
> >       at
> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1712)
> >       at
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1787)
> >       at
> org.apache.hadoop.mapred.SequenceFileRecordReader.getCurrentValue(SequenceFileRecordReader.java:104)
> >       at
> org.apache.hadoop.mapred.SequenceFileRecordReader.next(SequenceFileRecordReader.java:79)
> >       at
> org.apache.hadoop.mapred.join.WrappedRecordReader.next(WrappedRecordReader.java:112)
> >       at
> org.apache.hadoop.mapred.join.WrappedRecordReader.accept(WrappedRecordReader.java:130)
> >       at
> org.apache.hadoop.mapred.join.CompositeRecordReader.fillJoinCollector(CompositeRecordReader.java:398)
> >       at
> org.apache.hadoop.mapred.join.JoinRecordReader.next(JoinRecordReader.java:56)
> >       at
> org.apache.hadoop.mapred.join.JoinRecordReader.next(JoinRecordReader.java:33)
> >       at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
> >       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
> >       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> >       at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
> >
> >
> > how can I correct this?
> > thanks.
> > Xu
> >
> >
> --
> Jason Venner
> Attributor - Program the Web <http://www.attributor.com/>
> Attributor is hiring Hadoop Wranglers and coding wizards, contact if
> interested
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message