hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: why my test result on dfs short circuit read is slower?
Date Mon, 18 Feb 2013 10:16:26 GMT

On Feb 17, 2013, at 7:09 PM, "Liu, Raymond" <raymond.liu@intel.com> wrote:

> io.file.buffer.size

Drop this down to 64KB not 128KB. 

You have 16 cpu which really means 8 cores and 4 disks. 

Do you have Ganglia up and running? 

I'll wager that you'll see a lot of wait cpu cycles in both cases. 

Also how much memory do you have on each machine? 

Tuning is going to be hardware specific and without really understanding what each parameter
does, you can hurt performance. 

Michael Segel  | (m) 312.755.9623

Segel and Associates

The opinions expressed here are mine, while they may reflect a cognitive thought, that is
purely accidental. 
Use at your own risk. 
Michael Segel
michael_segel (AT) hotmail.com

View raw message