hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: HDFS Error - BlockReader: error in packet header
Date Sat, 15 Sep 2012 02:05:29 GMT
Hi Jason,

Does the DN log have something in it that corresponds to these errors?
Is there also some stacktrace/further text after the line you've
pasted until? Can we have it?

Also, did you tweak any checksum size related configs in your config files?

On Sat, Sep 15, 2012 at 3:20 AM, Jason Huang <jason.huang@icare.com> wrote:
> Hello,
>
> Looking for some help in setting up hadoop 1.0.3 in Pseudo distributed mode...
>
> I was able to install hadoop, config the .xml files and start all nodes:
> $ JPS
> 6645 Jps
> 6030 SecondaryNameNode
> 6185 TaskTracker
> 5851 NameNode
> 6095 JobTracker
> 5939 DataNode
>
> However, when I tried to play around with a couple of Map-reduce jobs
> with provided example jar files I got the following errors:
>
> (1) $ bin/hadoop jar hadoop-examples-1.0.3.jar pi 10 100
> Number of Maps  = 10
> Samples per Map = 100
> Wrote input for Map #0
> Wrote input for Map #1
> Wrote input for Map #2
> Wrote input for Map #3
> Wrote input for Map #4
> Wrote input for Map #5
> Wrote input for Map #6
> Wrote input for Map #7
> Wrote input for Map #8
> Wrote input for Map #9
> Starting Job
> 12/09/14 17:39:06 INFO mapred.FileInputFormat: Total input paths to process : 10
> 12/09/14 17:39:06 INFO mapred.JobClient: Running job: job_201209141701_0003
> 12/09/14 17:39:07 INFO mapred.JobClient:  map 0% reduce 0%
> 12/09/14 17:39:16 INFO mapred.JobClient: Task Id :
> attempt_201209141701_0003_m_000011_0, Status : FAILED
> Error initializing attempt_201209141701_0003_m_000011_0:
> java.io.IOException: BlockReader: error in packet header(chunkOffset :
> 142336, dataLen : 3538944, seqno : 3350829872548206857 (last: 0))
>         at org.apache.hadoop.hdfs.DFSClient$BlockReader.readChunk(DFSClient.java:1580)
>
> (2) $ ./bin/hadoop jar hadoop-examples-1.0.3.jar wordcount
> /user/jasonhuang/input /user/jasonhuang/output
> 12/09/14 17:37:51 INFO input.FileInputFormat: Total input paths to process : 1
> 12/09/14 17:37:51 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> 12/09/14 17:37:51 WARN snappy.LoadSnappy: Snappy native library not loaded
> 12/09/14 17:37:57 INFO mapred.JobClient: Cleaning up the staging area
> hdfs://localhost:9000/tmp/hadoop-jasonhuang/mapred/staging/jasonhuang/.staging/job_201209141701_0002
> 12/09/14 17:37:57 ERROR security.UserGroupInformation:
> PriviledgedActionException as:jasonhuang
> cause:org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> java.io.IOException: BlockReader: error in packet header(chunkOffset :
> 19968, dataLen : 1835351087, seqno : 7023413562532324724 (last: 0))
>
> Does anyone have idea on why the error occurs and how I can fix them?
>
> thanks!
>
> Jason



-- 
Harsh J

Mime
View raw message