Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 raymond.liu@intel.com does not designate 143.182.124.21 as permitted sender)
From: "Liu, Raymond" <raymond.liu@intel.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Subject: why my test result on dfs short circuit read is slower?
Thread-Topic: why my test result on dfs short circuit read is slower?
Thread-Index: Ac4L8yO0iWsCqVYcQtqlClElHzeq+Q==
Date: Sat, 16 Feb 2013 03:10:14 +0000
Message-ID: 
 <391D65D0EBFC9B4B95E117F72A360F1A1CB449@SHSMSX101.ccr.corp.intel.com>
Accept-Language: zh-CN, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi

	I tried to use short circuit read to improve my hbase cluster MR scan perf=
ormance.

	I have the following setting in hdfs-site.xml

	dfs.client.read.shortcircuit set to true
	dfs.block.local-path-access.user set to MR job runner.

	The cluster is 1+4 node and each data node have 16cpu/4HDD, with all hbase=
 table major compact thus all data is local.
	I have hoped that the short circuit read will improve the performance.

	While the test result is that with short circuit read enabled, the perform=
ance actually dropped 10-15%. Say scan a 50G table cost around 100s instead=
 of 90s.

	My hadoop version is 1.1.1, any idea on this? Thx!

Best Regards,
Raymond Liu