hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cho ju il <tjst...@kgrid.co.kr>
Subject How to use a number of file system?
Date Mon, 24 Nov 2014 06:14:42 GMT
I operates a three Hadoop cluster.
Hadoop cluster is not configured Federation.
Each is different Hadoop clusters.
Service applications are accessible to each of the Hadoop cluster.
 
I use something like this?
 
 
Configuration conf1 = new Configuration();
conf1.addResource( new Path("core-site-zone1.xml") );
conf1.addResource( new Path("hdfs-site-zone1.xml") );
FileSystem dfsZone1 = FileSystem.get(conf1);
 
Configuration conf2 = new Configuration();
conf2.addResource( new Path("core-site-zone2.xml") );
conf2.addResource( new Path("hdfs-site-zone2.xml") );
FileSystem dfsZone2 = FileSystem.get(conf2);
 
Configuration conf3 = new Configuration();
conf3.addResource( new Path("core-site-zone3.xml") );
conf3.addResource( new Path("hdfs-site-zone3.xml") );
FileSystem dfsZone3 = FileSystem.get(conf3);
 
 
## sample
## core-site.xml
&lt;configuration&gt;
	&lt;property&gt;
		&lt;name&gt;fs.defaultFS&lt;/name&gt;
		&lt;value&gt;viewfs:///&lt;/value&gt;
	&lt;/property&gt;
	&lt;property&gt;
		&lt;name&gt;fs.viewfs.impl&lt;/name&gt;
		&lt;value&gt;org.apache.hadoop.fs.viewfs.ViewFileSystem&lt;/value&gt;
	&lt;/property&gt;	
	&lt;property&gt;
		&lt;name&gt;fs.viewfs.mounttable.default.link./hdfs1&lt;/name&gt;
		&lt;value&gt;hdfs://ns01/hdfs1&lt;/value&gt;
	&lt;/property&gt;
&lt;/configuration&gt;

## hdfs-site.xml
&lt;configuration&gt;	&lt;!-- namespace --&gt;	&lt;property&gt;		&lt;name&gt;dfs.nameservices&lt;/name&gt;
	&lt;value&gt;ns01&lt;/value&gt;	&lt;/property&gt;	&lt;property&gt;
	&lt;name&gt;dfs.ha.namenodes.ns01&lt;/name&gt;		&lt;value&gt;nn01,nn02&lt;/value&gt;
&lt;/property&gt;
	&lt;!-- zone1 --&gt;	&lt;property&gt;		&lt;name&gt;dfs.namenode.rpc-address.ns01.nn01&lt;/name&gt;
	&lt;value&gt;namenode01:40000&lt;/value&gt;	&lt;/property&gt;	&lt;property&gt;
	&lt;name&gt;dfs.namenode.http-address.ns01.nn01&lt;/name&gt;		&lt;value&gt;namenode01:40070&lt;/value&gt;
&lt;/property&gt;	&lt;property&gt;		&lt;name&gt;dfs.namenode.rpc-address.ns01.nn02&lt;/name&gt;
	&lt;value&gt;namenode02:40000&lt;/value&gt;	&lt;/property&gt;	&lt;property&gt;
	&lt;name&gt;dfs.namenode.http-address.ns01.nn02&lt;/name&gt;		&lt;value&gt;namenode02:40070&lt;/value&gt;
&lt;/property&gt;	&lt;property&gt;		&lt;name&gt;dfs.client.failover.proxy.provider.ns01&lt;/name&gt;
	&lt;value&gt;org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider&lt;/value&gt;
&lt;/property&gt;
	&lt;property&gt;			&lt;name&gt;dfs.blocksize&lt;/name&gt;			&lt;value&gt;134217728&lt;/value&gt;
&lt;/property&gt;	&lt;property&gt;			&lt;name&gt;dfs.replication&lt;/name&gt;
		&lt;value&gt;2&lt;/value&gt;	&lt;/property&gt;
&lt;/configuration&gt;
Mime
View raw message