hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: InputFiles, Splits, Maps, Tasks Questions 1.3 Base
Date Wed, 24 Oct 2007 19:42:17 GMT
Lance Amundsen wrote:
> OK, that is encouraging.  I'll take another pass at it.  I succeeded
> yesterday with an in-memory only InputFormat, but only after I commented
> out some of the split referencing code, like the following in MapTask.java
> 
>     if (instantiatedSplit instanceof FileSplit) {
>       FileSplit fileSplit = (FileSplit) instantiatedSplit;
>       job.set("map.input.file", fileSplit.getPath().toString());
>       job.setLong("map.input.start", fileSplit.getStart());
>       job.setLong("map.input.length", fileSplit.getLength());
>     }

Yes, that code should not exist, but it shouldn't affect you either. 
You should be subclassing InputSplit, not FileSplit, so this code 
shouldn't operate on your splits.

Doug

Mime
View raw message