Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5BE72E9E3 for ; Mon, 18 Feb 2013 10:17:06 +0000 (UTC) Received: (qmail 24100 invoked by uid 500); 18 Feb 2013 10:17:01 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 23719 invoked by uid 500); 18 Feb 2013 10:17:00 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 23455 invoked by uid 99); 18 Feb 2013 10:16:59 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Feb 2013 10:16:59 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of michael_segel@hotmail.com designates 65.55.111.97 as permitted sender) Received: from [65.55.111.97] (HELO blu0-omc2-s22.blu0.hotmail.com) (65.55.111.97) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 18 Feb 2013 10:16:50 +0000 Received: from BLU0-SMTP180 ([65.55.111.71]) by blu0-omc2-s22.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 18 Feb 2013 02:16:29 -0800 X-EIP: [lwzBfO8z0lR49nCy2MzAuhYzCfMDOVjs] X-Originating-Email: [michael_segel@hotmail.com] Message-ID: Received: from [10.1.10.10] ([173.15.87.38]) by BLU0-SMTP180.phx.gbl over TLS secured channel with Microsoft SMTPSVC(6.0.3790.4675); Mon, 18 Feb 2013 02:16:28 -0800 From: Michael Segel Content-Type: multipart/alternative; boundary="Apple-Mail=_D12F4249-DE74-4644-8383-369F6C5E5919" MIME-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: why my test result on dfs short circuit read is slower? Date: Mon, 18 Feb 2013 04:16:26 -0600 References: <391D65D0EBFC9B4B95E117F72A360F1A1CB449@SHSMSX101.ccr.corp.intel.com> <391D65D0EBFC9B4B95E117F72A360F1A1CB559@SHSMSX101.ccr.corp.intel.com> <5C70D5E2-ACB6-4EF8-93A3-67E1DDA016B8@hortonworks.com> <391D65D0EBFC9B4B95E117F72A360F1A1CB58C@SHSMSX101.ccr.corp.intel.com> <391D65D0EBFC9B4B95E117F72A360F1A1CB5CB@SHSMSX101.ccr.corp.intel.com> <391D65D0EBFC9B4B95E117F72A360F1A1CCB6D@SHSMSX101.ccr.corp.intel.com> To: user@hadoop.apache.org In-Reply-To: <391D65D0EBFC9B4B95E117F72A360F1A1CCB6D@SHSMSX101.ccr.corp.intel.com> X-Mailer: Apple Mail (2.1499) X-OriginalArrivalTime: 18 Feb 2013 10:16:28.0420 (UTC) FILETIME=[03D0FC40:01CE0DC1] X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_D12F4249-DE74-4644-8383-369F6C5E5919 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" On Feb 17, 2013, at 7:09 PM, "Liu, Raymond" = wrote: > io.file.buffer.size Drop this down to 64KB not 128KB.=20 You have 16 cpu which really means 8 cores and 4 disks.=20 Do you have Ganglia up and running?=20 I'll wager that you'll see a lot of wait cpu cycles in both cases.=20 Also how much memory do you have on each machine?=20 Tuning is going to be hardware specific and without really understanding = what each parameter does, you can hurt performance.=20 Michael Segel | (m) 312.755.9623 Segel and Associates The opinions expressed here are mine, while they may reflect a cognitive = thought, that is purely accidental.=20 Use at your own risk.=20 Michael Segel michael_segel (AT) hotmail.com --Apple-Mail=_D12F4249-DE74-4644-8383-369F6C5E5919 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="iso-8859-1"
On Feb 17, 2013, = at 7:09 PM, "Liu, Raymond" <raymond.liu@intel.com> = wrote:

Michael = Segel  | (m) 312.755.9623

The opinions expressed here are mine, while they may reflect a = cognitive thought, that is purely accidental. 
Use at = your own risk. 
Michael Segel
michael_segel = (AT) hotmail.com


<= /div>



= --Apple-Mail=_D12F4249-DE74-4644-8383-369F6C5E5919--