Return-Path: 
 <common-user-return-26775-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: (qmail 48538 invoked from network); 9 Mar 2011 12:01:17 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 9 Mar 2011 12:01:17 -0000
Received: (qmail 12110 invoked by uid 500); 9 Mar 2011 12:01:14 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 12060 invoked by uid 500); 9 Mar 2011 12:01:14 -0000
Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-user@hadoop.apache.org>
List-Id: <common-user.hadoop.apache.org>
Reply-To: common-user@hadoop.apache.org
Delivered-To: mailing list common-user@hadoop.apache.org
Received: (qmail 12052 invoked by uid 99); 9 Mar 2011 12:01:14 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Mar 2011 12:01:14 +0000
X-ASF-Spam-Status: No, hits=3.7 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL,WEIRD_PORT
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of juneng603@gmail.com designates
 209.85.160.48 as permitted sender)
Received: from [209.85.160.48] (HELO mail-pw0-f48.google.com) (209.85.160.48)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Mar 2011 12:01:06 +0000
Received: by pwi14 with SMTP id 14so166913pwi.35
        for <common-user@hadoop.apache.org>;
 Wed, 09 Mar 2011 04:00:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:message-id:date:from:user-agent:mime-version:to
         :subject:content-type;
        bh=WnhboWPoUCTSwAFOPC9Qwitc7ZymSl2AYAQnFAkSO0w=;
        b=XtTuX8hbKjWVFFzFdB9vOTZFfBtR2GUsZg/pCit0onxo72uuo6F1LiR4SOo74FrMhj
         FqBg/ffn+PKbQtXUI1JJrLYAAh5HD1pE2L92ihonfcuEvBvOHB/4r2dq2jjqW70OX6mZ
         DwIn4L4OJtDT3UyzENNx/cLjhF8xnkO8xh3CQ=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:subject
         :content-type;
        b=jhqNN+nL+BqkiS9z31uJlmfuQgxw6We9uqfhT+Y+jJUSpwo5lK+Xylpv2gNLUe5/3+
         gMNC08gHZAuiionB7JoPafiGz1xXxvx/zI15x7FaH1gnKKDE+K7auT+EMIG5bApfL0Wx
         y9hH/SUsmH3z8y7NWQFpvWlh9vg8HvzznlGDU=
Received: by 10.142.152.34 with SMTP id z34mr2401479wfd.197.1299672044802;
        Wed, 09 Mar 2011 04:00:44 -0800 (PST)
Received: from [10.66.17.246] ([111.91.139.53])
        by mx.google.com with ESMTPS id w32sm2336437wfh.7.2011.03.09.04.00.42
        (version=SSLv3 cipher=OTHER);
        Wed, 09 Mar 2011 04:00:43 -0800 (PST)
Message-ID: <4D776B44.5090800@gmail.com>
Date: Wed, 09 Mar 2011 20:57:56 +0900
From: Jun Young Kim <juneng603@gmail.com>
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
 rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7
MIME-Version: 1.0
To: common-user@hadoop.apache.org
Subject: what's the differences between file.blocksize and dfs.blocksize in
 a job.xml?
Content-Type: multipart/alternative;
 boundary="------------010406010501060304060004"
X-Virus-Checked: Checked by ClamAV on apache.org

--------------010406010501060304060004
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

hi,

I am wondering the concepts of file.blocksize and dfs.blocksize.

in hdfs-site.xml, I set
<property>
<name>dfs.block.size</name>
<value>536870912</value>
<final>true</final>
</property>

in job.xml, I found
*file.blocksize* 	67108864


*dfs.blocksize* 	536870912


dfs browser's page>

*Name*
	*Type*
	*Size*
	*Replication*
	*Block Size*
	*Modification Time*
	*Permission*
	*Owner*
	*Group*
*20110309160005 
<http://thadps06.scast.nhnsystem.com:50075/browseDirectory.jsp?dir=%2Fuser%2Firteam%2F20110309160005&namenodeInfoPort=50070&delegation=null>*
	*dir*
	
	
	
	*2011-03-09 16:51*
	*rwxr-xr-x*
	*test*
	*supergroup*
*all0307.ep 
<http://thadps06.scast.nhnsystem.com:50075/browseDirectory.jsp?dir=%2Fuser%2Firteam%2Fall0307.ep&namenodeInfoPort=50070&delegation=null>*
	*file*
	*21.53 GB*
	*2*
	*64 MB*
	*2011-03-09 15:58*
	*rw-r--r--*
	*test*
	*supergroup*
*all0307.svc 
<http://thadps06.scast.nhnsystem.com:50075/browseDirectory.jsp?dir=%2Fuser%2Firteam%2Fall0307.svc&namenodeInfoPort=50070&delegation=null>*
	*file*
	*21.53 GB*
	*2*
	*64 MB*
	*2011-03-09 15:13*
	*rw-r--r--*
	*test*
	*supergroup*



total size of inputs of a job is about 44GB(all0307.ep + all0307.svc).
in the step of maping, the split's numbers are 690. (that means a map 
task took a single block size as 64MB).

I thought the splits counts should be about 88 because a single block 
size is 512MB and input file's size are 44GB).

How could I get the result I want?

thanks.

-- 
Junyoung Kim (juneng603@gmail.com)


--------------010406010501060304060004--