hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2060) IOException: Filesystem closed on submitJob
Date Fri, 10 Sep 2010 00:49:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907847#action_12907847
] 

Devaraj Das commented on MAPREDUCE-2060:
----------------------------------------

Hey Dan, this is really odd. For all RPC methods, the RPC server creates a UGI for the remote
user and the method call is invoked within a UGI.doAs. closeAllForUGI closes the filesystems
that were opened in the context of this doAs. The UGI for the remote user should be different
from the UGI of the JobTracker user.. So I don't understand how can closeAllForUGI close the
filesystems opened by the JobTracker user (UGI). Is this consistently happening for all job
submissions.

> IOException: Filesystem closed on submitJob
> -------------------------------------------
>
>                 Key: MAPREDUCE-2060
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2060
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.22.0
>         Environment: https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941
> https://svn.apache.org/repos/asf/hadoop/hdfs/trunk@993542
>            Reporter: Dan Adkins
>
> I get the following strange error on the jobtracker when attempting to submit a job:
> 10/09/09 20:31:35 INFO ipc.Server: IPC Server handler 7 on 31000, call submitJob(job_201009092028_0001,
hdfs://hns4.sea1.qc:21000/tmp/hadoop-mr20/mapred/staging/dadkins/.staging/job_201009092028_0001,
org.apache.hadoop.security.Credentials@20c87621) from 10.128.130.145:49253: error: java.io.IOException:
Filesystem closed
> java.io.IOException: Filesystem closed
> 	at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:307)
> 	at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1212)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:494)
> 	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1491)
> 	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:395)
> 	at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3078)
> 	at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:3014)
> 	at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:2996)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:349)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1380)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1376)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1105)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1374)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message