hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Doubt regarding finding Databaode --> Block map.
Date Sat, 19 Nov 2011 06:32:57 GMT
Pavan,

Moving to hdfs-dev@hadoop.apache.org (bcc'd original destination).
Please use appropriate lists for queries to get the best responses.

Also helps stating what version/branch of Hadoop you're attempting to work on.

On Sat, Nov 19, 2011 at 10:09 AM, Pavan Kulkarni
<pavan.baburao@gmail.com> wrote:
> Hi all,
>
>  I am working on a class project which involves Development in HADOOP.
> To implement a certain part I need the Datanode -> Block's map.(i.e. a
> mapping which shows the list of blocks on a given datanode)
> But I am unable to find this mapping in any of the classes. Any help would
> be  highly appreciated.

What you're looking for should ideally be found at a NameNode.
FSNamesystem there would use a BlockManager, which uses a
DatanodeManager maintains a list of DataNodeDescriptors, and each of
those contain a BlockInfo 'list' structure, which carry finalized
blocks.

While I'm not sure what you're attempting to do or how you're
attempting to do it, if you have access to the live FSNamesystem
object, you may get a complete iterator for a node via:

namesystem.getBlockManager().getDatanodeManager().getDatanode(dataNodeID).getBlockIterator()

And then iterate upon that for each BlockInfo object, which has all
the per-block metadata you need.

Where dataNodeID is a DatanodeID identifier (You can alternatively use
the DatanodeManager#getDatanodeByHost(String host) call).

Happy hacking :)

(Please feel free to correct me if there's a more optimal, or direct
way to get what he wants.)

-- 
Harsh J

Mime
View raw message