hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Mitic <iva...@microsoft.com>
Subject RE: Block placement Policy
Date Sat, 04 May 2013 16:10:26 GMT
Hi Mohammad,

You will have to implement BlockPlacementPolicy interface (abstract class). There are two
existing implementations in Hadoop trunk, BlockPlacementPolicyDefault and BlockPlacementPolicyWithNodeGroup.

To configure Hadoop to use your policy, define dfs.block.replicator.classname in hdfs-site.xml.
For example:

  <property>
    <name>dfs.block.replicator.classname</name>
    <value>org.apache.hadoop.hdfs.server.namenode.AzureBlockPlacementPolicy</value>
  </property>

Usually, you will also want to have your own script that computes the node topology/location
(look for net.topology.script.file.name in trunk, or topology.script.file.name in branch-1).

Hope this helps,
Ivan

From: Mohammad Tariq [mailto:dontariq@gmail.com]
Sent: Saturday, May 04, 2013 8:35 AM
To: user@hadoop.apache.org
Subject: Re: Block placement Policy

You might find this useful : https://issues.apache.org/jira/browse/HDFS-385

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com<http://cloudfront.blogspot.com>

On Sat, May 4, 2013 at 8:57 PM, Mohammad Mustaqeem <3m.mustaqeem@gmail.com<mailto:3m.mustaqeem@gmail.com>>
wrote:
I have read somewhere that a user can specified his own ReplicaPlacementPolicy.
How can I specify my own ReplicaPlacementPolicy?
I you have any sample ReplicaPlacementPolicy, then please share it..

--
*With regards ---*
*Mohammad Mustaqeem*,
M.Tech (CSE)
MNNIT Allahabad
9026604270


Mime
View raw message