hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-181) Secure job submission
Date Fri, 11 Sep 2009 05:42:57 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754012#action_12754012
] 

Amar Kamat commented on MAPREDUCE-181:
--------------------------------------

Had a chat with Owen and here is the job submission process with few extra addons :
# jobclient requests the jobtracker for a jobid [say $jobid]
# jobclient upload job.xml, job.jar, job.split, job.splitmetainfo, version, libs, archives
etc to the staging area i.e ~/.staging/$jobid
# jobclient now contructs a job-submission-token which contains 
  ## job staging area location (for job start and restart)
  ## job-submission version (for client-master compatibility)
  ## some checksum info (will expand on this later)
  ## user-credentials (for now username)
# jobclient passes job-submission-token over the rpc to jobtracker
# jobtracker persists this info in mapred.system.dir
# jobtracker uses the user-credentials in the job-meta-info to read the job.xml and job.splitmetainfo.

# jobtracker checks for job staging checksum
# when the tasktracker asks for a task, a Task is passed which contains the location of job.split
along with start-offset and length. 
# upon restart the jobtracker reads the job-meta info and re-submits the job (where the checksum
check is done again)
# once the job is done, the staging area is deleted 

Checksum:
# job.xml md5 : this prevents jobtracker/tasktrackers from using a changed jobconf across
job-submission and restarts.
# job-staging-area modification time : this prevents jobtracker and tasktracker for running
jobs for which the staging area has changed.


> Secure job submission 
> ----------------------
>
>                 Key: MAPREDUCE-181
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-181
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: hadoop-3578-branch-20-example-2.patch, hadoop-3578-branch-20-example.patch,
HADOOP-3578-v2.6.patch, HADOOP-3578-v2.7.patch, MAPRED-181-v3.8.patch
>
>
> Currently the jobclient accesses the {{mapred.system.dir}} to add job details. Hence
the {{mapred.system.dir}} has the permissions of {{rwx-wx-wx}}. This could be a security loophole
where the job files might get overwritten/tampered after the job submission. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message