hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3201) namenode should be able to retrieve block metadata from a datanode
Date Sat, 12 Apr 2008 01:09:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588180#action_12588180
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-3201:
------------------------------------------------

FSNamesystem.gotHeartbeat(..., Object[] xferResults, Object[] deleteList) uses Object[] as
pass-by-reference parameters.  It is better to define a class and use it as the return type.
 For example,
{code}
  class HreatbeatResponse {
    boolean registration;

    Block[] xferBlocks;
    DatanodeInfo[][] xferDatanodeInfo;

    Block[] deleteList;
  }

//the new header would be
HreatbeatResponse FSNamesystem.gotHeartbeat(DatanodeID nodeID,
                              long capacity,
                              long dfsUsed,
                              long remaining,
                              int xceiverCount,
                              int xmitsInProgress)
{code}


> namenode should be able to retrieve block metadata from a datanode
> ------------------------------------------------------------------
>
>                 Key: HADOOP-3201
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3201
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Tsz Wo (Nicholas), SZE
>
> The Append design (HADOOP-1700) requires that the namenode be able to contact the datanode
when a lease expires. 
> When a lease expires, the namenode has to fix up the size of the last block of the file
that was being written to. The namenode contacts the datanodes, retrieves the block generation
stamp and the length of the lastblock of all known replicas, determines which replicas are
good and which one are to be deleted, stamps the winning replicas with a new generation stamp
and deletes losing replicas. Once this process is complete, the namenode can allow a new writer
to append to this file. Details of this design are in HADOOP-1700.
> A few options available to us:
> 1. The namenode sends the request as a response to the next heartbeat RPC from the datanode.
This methodology is currently used for requesting block reports from datanodes.
> 2. The datanode has an RPC server. A pool of threads in the namenode can be used to make
RPCs to the datanodes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message