hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harold Valdivia Garcia <harold.valdi...@upr.edu>
Subject how to read replicated blocks with hdfs api?
Date Thu, 06 Aug 2009 20:20:07 GMT
Hi... I was reading the HDFS code, and I can't find a way to read the
replicated blocks of a block-file.

DFS.getFileBlockLocations returns all blocks of a file
File = block-a, block-b, ..... block-n.

each of these blocks has its replicated blocks. if for instance the
replication factor is 3, how can I retrieve block-a1, block-a2, block-a3 in
parallel from my user code?

I did read DFSClient, DFSClient.DFInputStream to understand how hadoop
retrieves data from blocks, but it is hard.
There is no an easy way to do this?
Harold Dwight Valdivia Garcia
Graduate Student
M.S Computer Engineering
University of Puerto Rico, Mayaguez Campus

View raw message