hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravi...@ymail.com>
Subject Re: can block size for namenode be different from datanode block size?
Date Wed, 25 Mar 2015 18:29:24 GMT
Hi Mich!

The block size you are referring to is used only on the datanodes. The file that the namenode
writes (fsimage OR editlog) is not chunked using this block size.

     On Wednesday, March 25, 2015 8:12 AM, Dr Mich Talebzadeh <mich@peridale.co.uk>


The block size for HDFS is currently set to 128MB by defauilt. This is

My point is that I assume thisĀ  parameter in hadoop-core.xml sets the
block size for both namenode and datanode. However, the storage and
random access for metadata in nsamenode is different and suits smaller
block sizes.

For example in Linux the OS block size is 4k which means one HTFS blopck
sizeĀ  of 128MB can hold 32K OS blocks. For metadata this may not be
useful and smaller block size will be suitable and hence my question.



View raw message