hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3760) DFS operations fail because of Stream closed error
Date Tue, 15 Jul 2008 10:43:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613576#action_12613576
] 

Amar Kamat commented on HADOOP-3760:
------------------------------------

Here is the complete error
{code}
08/07/15 10:21:04 INFO mapred.FileInputFormat: Total input paths to process : 100
java.io.IOException: Stream closed.
       at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.isClosed(DFSClient.java:2240)
       at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2744)
       at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:2652)
       at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:59)
       at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:79)
       at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:726)
       at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:966)
       at org.apache.hadoop.examples.Sort.run(Sort.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
       at org.apache.hadoop.examples.Sort.main(Sort.java:158)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
       at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
       at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:59)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
Exception closing file system.dir/job/job.xml
java.io.IOException: Stream closed.
       at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.isClosed(DFSClient.java:2240)
       at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2687)
       at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:2652)
       at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:220)
       at org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:236)
       at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1379)
       at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:230)
       at org.apache.hadoop.fs.FileSystem$ClientFinalizer.run(FileSystem.java:215)

{code}
The way this can be reproduced is 
1) Run random-writer 
2) Run sort ... here the job submission fails with the above mentioned error.


Also reduce tasks fail with the error 
{code}
2008-07-15 09:31:28,631 INFO org.apache.hadoop.hdfs.DFSClient: org.apache.hadoop.ipc.RemoteException:

org.apache.hadoop.hdfs.server.namenode.NotReplicatedYetException: Not replicated yet:tmp-dir/part-00000
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1115)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:340)
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)

	at org.apache.hadoop.ipc.Client.call(Client.java:707)
	at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
	at $Proxy1.addBlock(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
	at $Proxy1.addBlock(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2445)
	at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2328)
	at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1800(DFSClient.java:1740)
	at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1917)

{code}

> DFS operations fail because of Stream closed error
> --------------------------------------------------
>
>                 Key: HADOOP-3760
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3760
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs
>            Reporter: Amar Kamat
>            Priority: Blocker
>
> DFS operations fail because of  {{java.io.IOException: Stream closed.}}. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message