Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <22105838.1153220714845.JavaMail.jira@brutus>
Date: Tue, 18 Jul 2006 04:05:14 -0700 (PDT)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Resolved: (HADOOP-307) Many small jobs benchmark for
 MapReduce
In-Reply-To: <13009062.1150636529837.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

     [ http://issues.apache.org/jira/browse/HADOOP-307?page=all ]

Doug Cutting resolved HADOOP-307.
---------------------------------

    Fix Version/s: 0.5.0
       Resolution: Fixed

I just committed this.  Thanks!

> Many small jobs benchmark for MapReduce
> ---------------------------------------
>
>                 Key: HADOOP-307
>                 URL: http://issues.apache.org/jira/browse/HADOOP-307
>             Project: Hadoop
>          Issue Type: Task
>          Components: mapred
>            Reporter: Sanjay Dahiya
>            Priority: Minor
>             Fix For: 0.5.0
>
>         Attachments: patch.txt, patch.txt
>
>
> A benchmark that runs many small MapReduce tasks in sequence. A single map reduce implementation is used, it is invoked multiple times with input as the output from previous run. The input to first Map is a TextInputFormat ( a text file with few hundred KBs). Input records are passed to output without much processing. The idea is to benchmark the time taken by initialization of Mapper and Reducer. An initial prototyping on a single machine with 20 MR tasks in sequence took ~47 seconds per task. Looking for suggestions on what else can be included in the benchmark. 

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira