hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject NullPointerExceptipn in getPos of buffered FsDataInputStream
Date Tue, 10 Sep 2013 00:53:26 GMT
I'm finding funny behaviour in the multifilewordcount example .  The getPos implementation
in the RecirdReader is failing for me when the input steam is closed, and throwing a cryptic
null pointer exception.

What is the correct behaviour for getPos  in a record reader, and how 
should it behave when the underlying stream is null?  It appears this can 
happen in the rawlocalfilesystem.  Not sure if its implemented more safely 
in DistributedfileSYstem just yet.


   I've found that the getPos in the RawLocalFileSystem's input stream can 
throw a null pointer exception if its underlying stream is closed.

I discovered this when playing with a custom record reader.

to patch it, I simply check if a call to "stream.available()" throws an 
exception, and if so, I return 0 in the getPos() function.

The existing getPos() implementation is found here:

https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20/src/examples/org/apache/hadoop/examples/MultiFileWordCount.java

What should be the correct behaviour of getPos() in the RecordReader?


http://stackoverflow.com/questions/18708832/hadoop-rawlocalfilesystem-and-getpos

-- 
Jay Vyas 
http://jayunit100.blogspot.com
Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message