hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Engin Sözer <enginso...@hotmail.com>
Subject BlockPlacementPolicy and File Deletion
Date Thu, 30 Jan 2014 00:43:54 GMT
Hello all,

I'm currently trying to develop a Block Placement Policy in my research project. However,
I'm having some problems. If anybody could hep me, I would really appreciate that. Below are
my questions:

1) In the block placement policy, how can I get the name of the original file that the specific
block belongs to? There's the srcPath parameter, and I can use this when I'm injecting the
file to hdfs for the first time. But what about replicating the blocks that are under-replicated?
In that case, how can I find the original filename?

2) When I want to delete a file, is blockplacementpolicy included in the process? Or chooseReplicaToDelete
function only works for over-replicated blocks? I found out that blockManager class is handling
this operation. But I couldn't really figure out where to look. The reason I want to understand
is that I'm keeping specific information for every file/block that I'm writing to hdfs. Thus
when I add or delete a file, I need to write the information to the datanodes that these files
are affecting.

I appreciate your help in advance.

Best regards,
Engin Sozer

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message