hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Uma Maheswara Rao G 72686 <mahesw...@huawei.com>
Subject Re: HDFS small files
Date Thu, 13 Oct 2011 17:25:18 GMT
----- Original Message -----
From: Sesha Kumar <sesha911@gmail.com>
Date: Thursday, October 13, 2011 7:34 pm
Subject: HDFS small files
To: hdfs-user@hadoop.apache.org

> Hello guys,
> I've just started hdfs implementation.  I've got the source of hdfs.
> I want to know from where i should start to read the source?  Am 
> trying to
> get a hold on implementation things of hdfs.
> Also i want to know in what way the federation model deals with 
> minimizingsmall files problem? How is the namespace separated?

Federation is basically to scale the NameNodes. There will be maltiple NameSpaces again here.
Clients will have mount tables and connect to corresponding NN. If you create many small files,
obiously more memory is required at NN side to store. So, In Federation case, we have multiple
NNs, they can hold more files metadata.

Below doc can help you to understand more about usage.


View raw message