hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrian (Xinyu) Liu" <adrian....@sas.com>
Subject Transfer blocks from one datanode to another
Date Thu, 06 Sep 2012 09:42:47 GMT
Hi All,

Nowadays, I am working with HDFS and implementing some functionalities base on HDFS API.
As I knew, one specific file is divided into several blocks and distributed into different
datanode with certain replication numbers.
And I want to find out a series of HDFS API which can meet the following requirement:

1.       Given a specific filename and related information that already uploaded into the
HDFS, retrieve: how many blocks are there,

each datanode contain which blocks, etc.

2.       Given a specific filename, source datanode, specific block id, destination datanode
and related information to transfer the block

from source node to destination node.

I've read several materials and API reference about HDFS and can't find appropriate ones,
the objective of this mail is to make sure
if such  API existed, and if so, what are they (especially the second one, transfer a specific
block of a specific file from a certain datanode to another)

There is a tool called Balancer already existed in HDFS package, I am reading the source code,
but it's too intricate to track the line, can anyone help me?

Thanks & Best Regards,
Adrian(Xinyu) Liu
Customer Competence Division, SAS R&D Beijing
TEL: +86 10 83193849   MOBILE: +86 186 1064 1590

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message