hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: InputFiles, Splits, Maps, Tasks Questions 1.3 Base
Date Wed, 24 Oct 2007 19:42:17 GMT
Lance Amundsen wrote:
> OK, that is encouraging.  I'll take another pass at it.  I succeeded
> yesterday with an in-memory only InputFormat, but only after I commented
> out some of the split referencing code, like the following in MapTask.java
>     if (instantiatedSplit instanceof FileSplit) {
>       FileSplit fileSplit = (FileSplit) instantiatedSplit;
>       job.set("map.input.file", fileSplit.getPath().toString());
>       job.setLong("map.input.start", fileSplit.getStart());
>       job.setLong("map.input.length", fileSplit.getLength());
>     }

Yes, that code should not exist, but it shouldn't affect you either. 
You should be subclassing InputSplit, not FileSplit, so this code 
shouldn't operate on your splits.


View raw message