hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-604) DataNodes get NullPointerException and become unresponsive on 50010
Date Thu, 26 Oct 2006 21:43:21 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-604?page=comments#action_12445010 ] 
            
dhruba borthakur commented on HADOOP-604:
-----------------------------------------

I was investigating HADOOP-643. The problem is that the FSDataSet is not protected by locks.
In this case, data-receive thread is receiving some data from another datanaode and is traversing
the dataset to validate that the directory is valid. At the same time,  the main datanode
thread (offerService()) could be modifying the FSDataset. Protecting the DataXceiveServer
thread from the offerservice thread is all this is needed.

It appears that the DataTransfer does not need any protection.

> DataNodes get NullPointerException and become unresponsive on 50010
> -------------------------------------------------------------------
>
>                 Key: HADOOP-604
>                 URL: http://issues.apache.org/jira/browse/HADOOP-604
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.6.0
>            Reporter: Christian Kunz
>         Assigned To: Raghu Angadi
>
> I wonder whether handling of 'children' in FSDataset is thread-safe.
> Exception in thread "org.apache.hadoop.dfs.DataNode$DataXceiveServer@19b04e2" java.lang.NullPointerException
>         at org.apache.hadoop.dfs.FSDataset$FSDir.checkDirTree(FSDataset.java:162)
>         at org.apache.hadoop.dfs.FSDataset$FSDir.checkDirTree(FSDataset.java:162)
>         at org.apache.hadoop.dfs.FSDataset$FSDir.checkDirTree(FSDataset.java:162)
>         at org.apache.hadoop.dfs.FSDataset$FSDir.checkDirTree(FSDataset.java:162)
>         at org.apache.hadoop.dfs.FSDataset$FSVolume.checkDirs(FSDataset.java:238)
>         at org.apache.hadoop.dfs.FSDataset$FSVolumeSet.checkDirs(FSDataset.java:326)
>         at org.apache.hadoop.dfs.FSDataset.checkDataDir(FSDataset.java:522)
>         at org.apache.hadoop.dfs.DataNode$DataXceiveServer.run(DataNode.java:472)
>         at java.lang.Thread.run(Thread.java:595)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message