hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin Reed (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-933) Application defined InputSplits do not work
Date Fri, 26 Jan 2007 23:11:49 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12467956
] 

Benjamin Reed commented on HADOOP-933:
--------------------------------------

I found another place that assumed FileSplit. See attached patch. Our test cases pass now,
so that should be the end of it. (At least for our use of InputSplit.)

> Application defined InputSplits do not work
> -------------------------------------------
>
>                 Key: HADOOP-933
>                 URL: https://issues.apache.org/jira/browse/HADOOP-933
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.10.1
>            Reporter: Benjamin Reed
>             Fix For: 0.10.1
>
>         Attachments: JobInProgress.patch, MapTask.patch
>
>
> If an application defines its own InputSplit, the task tracker chokes when it cannot
deserialize the InputSplit when it deserializes MapTasks it receives from the JobTracker.
This is because the TaskTracker does not resolve classes from the job jar file. The attached
patch delays resolution of the InputSplit until it is running in the context of the child
process where it can resolve the InputSplit class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message