hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: HDFS Federation
Date Wed, 25 May 2016 20:26:50 GMT
You might be thinking of renames.  For example, when using ViewFs with Federation, the client-side
mount configuration might have path /data1 backed by NameNode1 and path /data2 backed by NameNode2.
 Renaming /data1/file1 to /data2/file1 would not be supported, because it would need to be
copied across NameNodes, and then HDFS would not be able to satisfy its promise that rename
is atomic.  There are more details about this in the ViewFs guide.

http://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/ViewFs.html

--Chris Nauroth

From: Kun Ren <ren.hdfs@gmail.com<mailto:ren.hdfs@gmail.com>>
Date: Wednesday, May 25, 2016 at 11:03 AM
To: Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
Cc: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: HDFS Federation

Thanks a lot, Chris.

I remembered that Federation doesn't support some cross-namenodes operations, if so,  which
operations that Federation doesn't support and why? Thanks again.

On Wed, May 25, 2016 at 12:17 PM, Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
wrote:
Hello Kun,

Yes, this command works with federation.  The command would copy the file from NameNode 1/block
pool 1 to NameNode 2/block pool 2.

--Chris Nauroth

From: Kun Ren <ren.hdfs@gmail.com<mailto:ren.hdfs@gmail.com>>
Date: Wednesday, May 25, 2016 at 8:57 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: HDFS Federation

Hi Genius,

Does HDFS Federation support the cross namenodes operations?

For example:

./bin/hdfs dfs -cp input1/a.xml input2/b.xml

Supposed that input1 belongs namenode 1, and input 2 belongs namenode 2, does Federation support
this operation? And if not, why?

Thanks.


Mime
View raw message