Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 97258 invoked from network); 27 Nov 2007 22:58:09 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 27 Nov 2007 22:58:09 -0000 Received: (qmail 17687 invoked by uid 500); 27 Nov 2007 22:57:51 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 17649 invoked by uid 500); 27 Nov 2007 22:57:51 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 17640 invoked by uid 99); 27 Nov 2007 22:57:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2007 14:57:51 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO brutus.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2007 22:57:40 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 192D4714208 for ; Tue, 27 Nov 2007 14:57:43 -0800 (PST) Message-ID: <5308371.1196204263083.JavaMail.jira@brutus> Date: Tue, 27 Nov 2007 14:57:43 -0800 (PST) From: "Amar Kamat (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Updated: (HADOOP-1965) Handle map output buffers better In-Reply-To: <16506106.1190985650941.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat updated HADOOP-1965: ------------------------------- Status: Patch Available (was: Open) > Handle map output buffers better > -------------------------------- > > Key: HADOOP-1965 > URL: https://issues.apache.org/jira/browse/HADOOP-1965 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Affects Versions: 0.16.0 > Reporter: Devaraj Das > Assignee: Amar Kamat > Fix For: 0.16.0 > > Attachments: 1965_single_proc_150mb_gziped.jpeg, 1965_single_proc_150mb_gziped.pdf, 1965_single_proc_150mb_gziped_breakup.png, HADOOP-1965-1.patch, HADOOP-1965-Benchmark.patch, HADOOP-1965-Benchmark.patch, HADOOP-1965-Benchmark.patch > > > Today, the map task stops calling the map method while sort/spill is using the (single instance of) map output buffer. One improvement that can be done to improve performance of the map task is to have another buffer for writing the map outputs to, while sort/spill is using the first buffer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.