hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1619) FSInputChecker attempts to seek past EOF
Date Mon, 16 Jul 2007 21:19:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12513039
] 

Raghu Angadi commented on HADOOP-1619:
--------------------------------------

Can someone familiar with PigInputFormat comment if it is actually not trying to skip beyond
the file length. It might be depending on skip() return to identify EOF (nothing wrong with
that).. But with HADOOP-1470, skip() is implemented as a wrapper over seek(), which throws
an EOFException  when we try to seek past. 

If skip in Pig package is supposed to be valid then we can check if there is some other bug.



> FSInputChecker attempts to seek past EOF
> ----------------------------------------
>
>                 Key: HADOOP-1619
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1619
>             Project: Hadoop
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.14.0
>            Reporter: Nigel Daley
>            Priority: Blocker
>             Fix For: 0.14.0
>
>
> I'm not sure which class in the stack trace below is responsible for attempting to seek
past the end of file. 
> 2007-07-16 20:31:40,598 INFO org.apache.hadoop.mapred.TaskInProgress: Error from task_200707162028_0014_m_000000_0:
java.io.IOException: Cannot seek after EOF
> 	at org.apache.hadoop.dfs.DFSClient$DFSInputStream.seek(DFSClient.java:1040)
> 	at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:37)
> 	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:188)
> 	at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:234)
> 	at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:176)
> 	at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:193)
> 	at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:157)
> 	at org.apache.hadoop.fs.FSInputChecker.readFully(FSInputChecker.java:353)
> 	at org.apache.hadoop.fs.FSInputChecker.seek(FSInputChecker.java:331)
> 	at org.apache.hadoop.fs.FSInputChecker.skip(FSInputChecker.java:306)
> 	at java.io.FilterInputStream.skip(FilterInputStream.java:125)
> 	at java.io.FilterInputStream.skip(FilterInputStream.java:125)
> 	at com.yahoo.pig.impl.io.InputStreamPosition.skip(InputStreamPosition.java:55)
> 	at java.io.BufferedInputStream.skip(BufferedInputStream.java:349)
> 	at java.io.FilterInputStream.skip(FilterInputStream.java:125)
> 	at com.yahoo.pig.impl.builtin.RandomSampleLoader.getNext(RandomSampleLoader.java:34)
> 	at com.yahoo.pig.impl.mapreduceExec.PigInputFormat$PigRecordReader.next(PigInputFormat.java:169)
> 	at org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:171)
> 	at com.yahoo.pig.impl.mapreduceExec.PigMapReduce.run(PigMapReduce.java:98)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:189)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1771)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message