hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-776) Gridmix: Trace-based benchmark for Map/Reduce
Date Wed, 22 Jul 2009 10:27:14 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Douglas updated MAPREDUCE-776:

    Attachment: MR776-0.patch

_Extremely_ preliminary shell of a nascent patch-in-progress using mocked Rumen traces for
testing. Mostly experiments and scaffolding tagged with TODO items at this point, but the
driver works and the I/O is correct through the shuffle within expected tolerances.

> Gridmix: Trace-based benchmark for Map/Reduce
> ---------------------------------------------
>                 Key: MAPREDUCE-776
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-776
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: benchmarks
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>         Attachments: MR776-0.patch
> Previous benchmarks ( HADOOP-2369 , HADOOP-3770 ), while informed by production jobs,
were principally load generating tools used to validate stability and performance under saturation.
The important dimensions of that load- submission order/rate, I/O profile, CPU usage, etc-
only accidentally match that of the real load on the cluster. Given related work that characterizes
production load ( MAPREDUCE-751 ), it would be worthwhile to use mined data to impose a corresponding
load for tuning and guiding development of the framework.
> The first version will focus on modeling task I/O, submission, and memory usage.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message