Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <5023B6AB.807@cse.psu.edu>
Date: Thu, 09 Aug 2012 09:10:03 -0400
From: "Ellis H. Wilson III" <ellis@cse.psu.edu>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:10.0.6esrpre) Gecko/20120807 Thunderbird/10.0.6
MIME-Version: 1.0
To: user@hadoop.apache.org
Subject: fs.local.block.size vs file.blocksize
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: ehw111@cse.psu.edu

Hi all!

Can someone please briefly explain the difference?  I do not see 
deprecated warnings for fs.local.block.size when I run with them set and 
I see two copies of RawLocalFileSystem.java (the other is 
local/RawLocalFs.java).

The things I really need to get answers to are:
1. Is the default boosted to 64MB from Hadoop 1.0 to Hadoop 2.0?  I 
believe it is, but want validation on that.
2. Which one controls shuffle block-size?
3. If I have a single machine non-distributed instance, and point it at 
file://, do both of these control the persistent data's block size or 
just one of them or what?
4. Is there any way to run with say a 512MB blocksize for the persistent 
data and the default 64MB blocksize for the shuffled data?

Thanks!

ellis