hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kiyoshi Mizumaru <kiyoshi.mizum...@gmail.com>
Subject TestDFSIO writes files on HDFS with wrong block size?
Date Fri, 14 May 2010 06:38:10 GMT
Hi all, this is my first post to this list, and if i'm not in
appropriate place, please let me know.


I have just created a Hadoop instance and its HDFS is configured as:
  dfs.replication = 1
  dfs.block.size = 536870912 (512MB)

Then I typed the following command to run TestDFSIO against this instance:
  % hadoop jar hadoop-*-test.jar TestDFSIO -write -nrFiles 1 -fileSize 1024

One file with 1024MB size should consist of 2 blocks of size 512MB,
but filesystem browser shows that /benchmarks/TestDFSIO/io_data/test_io_0
consists of 16 blocks of size 64MB, and its replication is 3, so 48 blocks
are displayed in total..

This is not what I expected, does anyone know what's wrong?

I'm using Cloudera's Distribution for Hadoop (hadoop-0.20-0.20.2+228-1)
with Sun Java6 (jdk-6u19-linux-amd64).  Thanks in advance and sorry for
my poor English, I'm still leaning it.
--
Kiyoshi

Mime
View raw message