hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3681) Infinite loop in dfs close
Date Fri, 04 Jul 2008 02:19:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610420#action_12610420
] 

Koji Noguchi commented on HADOOP-3681:
--------------------------------------

+1 Tested manually (throwing IOException in the middle).

With second patch
{noformat}
bash-3.00$ ls -l testfile
-rw-r--r--  1 knoguchi users 75396 Jul  4 01:46 testfile
bash-3.00$ $HADOOP_HOME/bin/hadoop dfs -put testfile  /user/knoguchi
08/07/04 02:13:43 WARN dfs.DFSClient: DataStreamer Exception: java.io.IOException: testing
08/07/04 02:13:43 WARN dfs.DFSClient: Error Recovery for block null bad datanode[0]
bash-3.00$ echo $?
0
bash-3.00$ $HADOOP_HOME/bin/hadoop dfs -ls /user/knoguchi
Found 1 items
/user/knoguchi/testfile <r 1>   0       2008-07-04 02:13        rw-r--r--       knoguchi
       supergroup
{noformat}

With third patch, 
{noformat}
bash-3.00$ $HADOOP_HOME/bin/hadoop dfs -put testfile  /user/knoguchi
08/07/04 02:15:03 WARN dfs.DFSClient: DataStreamer Exception: java.io.IOException: testing
08/07/04 02:15:03 WARN dfs.DFSClient: Error Recovery for block null bad datanode[0]
put: Could not get block locations. Aborting...
Exception closing file /user/knoguchi/testfile
java.io.IOException: Could not get block locations. Aborting...
        at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2084)
        at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1300(DFSClient.java:1702)
        at org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1822)
bash-3.00$ echo $?
255
{noformat}


> Infinite loop in dfs close
> --------------------------
>
>                 Key: HADOOP-3681
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3681
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.17.0
>            Reporter: Koji Noguchi
>            Assignee: Lohit Vijayarenu
>             Fix For: 0.17.1, 0.18.0
>
>         Attachments: H-3681-jstack.txt, HADOOP-3681-1.patch, HADOOP-3681-2.patch, HADOOP-3681-3-17.patch,
HADOOP-3681-3-18.patch, HADOOP-3681-3-19.patch
>
>
> We had dfsClient -put  hang outputting 
> {noformat}
> 2008-06-28 10:05:12,595 WARN org.apache.hadoop.dfs.DFSClient: DataStreamer Exception:
java.net.SocketTimeoutException:
> timed out waiting for rpc response
> 2008-06-28 10:05:12,595 WARN org.apache.hadoop.dfs.DFSClient: Error Recovery for block
null bad datanode[0]
> 2008-06-28 10:05:51,067 INFO org.apache.hadoop.dfs.DFSClient: Could not complete file
> /_temporary/_task_200806262325_4136_r_000408_0/part-00408
> retrying...
> 2008-06-28 10:05:52,898 INFO org.apache.hadoop.dfs.DFSClient: Could not complete file
> /_temporary/_task_200806262325_4136_r_000408_0/part-00408
> retrying...
> 2008-06-28 10:05:54,893 INFO org.apache.hadoop.dfs.DFSClient: Could not complete file
> /_temporary/_task_200806262325_4136_r_000408_0/part-00408
> retrying...
> 2008-06-28 10:05:56,920 INFO org.apache.hadoop.dfs.DFSClient: Could not complete file
> /_temporary/_task_200806262325_4136_r_000408_0/part-00408
> retrying...
> 2008-06-28 10:05:57,765 INFO org.apache.hadoop.dfs.DFSClient: Could not complete file
> /_temporary/_task_200806262325_4136_r_000408_0/part-00408
> retrying...
> 2008-06-28 10:05:58,199 INFO org.apache.hadoop.dfs.DFSClient: Could not complete file
> /_temporary/_task_200806262325_4136_r_000408_0/part-00408
> retrying...
> [repeats forever]
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message