hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2089) JobToken should be passed over RPC
Date Fri, 01 Oct 2010 23:42:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917106#action_12917106
] 

Devaraj Das commented on MAPREDUCE-2089:
----------------------------------------

The jobtoken is used for shuffle and it is also used for Pipes, for both the secure/unsecure
cases. Can't skip it..

> JobToken should be passed over RPC
> ----------------------------------
>
>                 Key: MAPREDUCE-2089
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2089
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client, job submission, jobtracker, security, task, tasktracker
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>             Fix For: 0.22.0
>
>         Attachments: mr-2089.patch, mr-2089.patch
>
>
> JobToken file is currently created by the JobTracker, and stored in the HDFS. Later on,
the TaskTrackers pull that file from the HDFS as part of job localization. The HDFS operations
can be avoided if the the jobToken file is passed over RPC (Jobclient -> JobTracker ->
TaskTrackers).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message