hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edgar Pek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6527) Data race on field org.apache.hadoop.mapred.JobConf.credentials
Date Thu, 03 Mar 2016 18:49:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178358#comment-15178358
] 

Edgar Pek commented on MAPREDUCE-6527:
--------------------------------------

Thank you, for the reply. 
This also meant that the thread could have accessed the files it does not have permission
for because setCredentials was not set yet? Is that correct?

> Data race on field org.apache.hadoop.mapred.JobConf.credentials
> ---------------------------------------------------------------
>
>                 Key: MAPREDUCE-6527
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6527
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Ali Kheradmand
>            Assignee: Haibo Chen
>         Attachments: mapreduce6527.001.patch
>
>
> I am running the test suite against a dynamic race detector called RV-Predict. Here is
a race report that I got: 
> {noformat}
> Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{
>     Concurrent read in thread T327 (locks held: {})
>  ---->  at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505)
>         at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:70)
>         at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524)
>     T327 is created by T22
>         at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:218)
>     Concurrent write in thread T22 (locks held: {Monitor@496c673a, Monitor@496319b0})
>  ---->  at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510)
>         at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787)
>         at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241)
>         at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
>         at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
>          locked Monitor@496319b0 at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a)

>         at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245)
>          locked Monitor@496c673a at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229)

>     T22 is created by T1
>         at org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111)
> }}}
> {noformat}
> In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob function, we have
the following lines:
> {code}
>     Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
>     job.job.setCredentials(credentials);
> {code}
> It looks a bit suspicious: Job extends thread and at the end of its constructor it starts
a new thread which creates a new instance of JobContextImpl which reads credentials. However,
the first thread concurrently sets credentials after a creating the Job instance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message