hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sridhar basam <...@basam.org>
Subject Re: Block size in HDFS
Date Fri, 10 Jun 2011 15:43:21 GMT
On Fri, Jun 10, 2011 at 11:05 AM, Pedro Costa <psdc1978@gmail.com> wrote:

> Hi,
>
> If I define HDFS to use blocks of 64 MB, and I store in HDFS a 1KB
> file, this file will ocupy 64MB in the HDFS?
>

No, it will occupy something much closer to 1KB than 64MB. There is some
small overhead related to metadata about the block plus the space on the
underlying filesystem (ext2, ext3, etc)  required to store the 1KB file.
 You can just cd into the datanode data directories and look for the block
in question.

HDFS isn't really optimized for small files though.

 Sridhar

Mime
View raw message