hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raihan Jamal <jamalrai...@gmail.com>
Subject Re: Unexpected end of input stream
Date Tue, 28 Aug 2012 16:27:39 GMT
That basically means your data was not in the correct format when you move
or copied the data to HDFS. So there is one file which is corrupted, you
can find the file name in your error logs.



*Raihan Jamal*



On Tue, Aug 28, 2012 at 7:23 AM, Kiwon Lee <kiwoni.lee@gmail.com> wrote:

> Hi
>
> I have a lot of compressed gzip files on hdfs.
> An exception has occurred at TaskTracker, during processing of MR.
> If any file is invalid, may I know that?
>
>
> 2012-08-28 09:17:56,320 INFO ExecMapper: ExecMapper: processed 0 rows:
> used memory = 125190136
> 2012-08-28 09:17:56,324 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
> Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
>  2012-08-28 09:17:56,326 ERROR
> org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
> as:ubuntu (auth:SIMPLE) cause:java.io.IOException: java.io.EOFException:
> Unexpected end of input stream
> 2012-08-28 09:17:56,326 WARN org.apache.hadoop.mapred.Child: Error running
> child
> java.io.IOException: java.io.EOFException: Unexpected end of input stream
>         at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
>         at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
>         at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:275)
>         at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
>         at
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
>         at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:108)
>         at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:210)
>         at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:195)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:393)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
>         at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: java.io.EOFException: Unexpected end of input stream
>         at
> org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:143)
>         at
> org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:83)
>         at java.io.InputStream.read(InputStream.java:82)
>         at
> org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:209)
>         at org.apache.hadoop.util.LineReader.readLine(LineReader.java:173)
>         at
> org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:160)
>         at
> org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:38)
>         at
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:273)
>         ... 13 more
>
>
> --
>
> *Best Regards.** Ethan (Kiwon Lee)*
>    kiwoni.lee@gmail.com
>
>
>

Mime
View raw message