hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3439) [Gridmix] Support addons in Gridmix
Date Mon, 09 Mar 2015 22:06:40 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Allen Wittenauer updated MAPREDUCE-3439:
    Affects Version/s:     (was: 0.24.0)

> [Gridmix] Support addons in Gridmix
> -----------------------------------
>                 Key: MAPREDUCE-3439
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3439
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/gridmix
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>              Labels: addons, gridmix, gridmix3
> At times there is a need to benchmark certain Hadoop client APIs. Often, this is done
by running simple & standard sort-like programs on Hadoop and then using an external utility
to benchmark the APIs. But then the benchmarking results tend to be off from reality as the
load on the cluster doesn't match the actual load. We believe that Gridmix3 - which is a Hadoop
workload simulator - can prove useful here. Gridmix3 already provides a mechanism to load
the cluster - often called as a 'test cluster' - using a real trace thus mimicking the real-life
> Currently, Gridmix3 consumes a representative workload trace and loads the Hadoop cluster
to match what is seen in the trace. Gridmix3 can be enhanced to also support user scripts
(hereby referred as 'addons') which will be loaded within Gridmix3 and will get updates like
> 1. Job submission
> 2. Job completion
> 3. Cluster status
> These addons can also ping/access a live, close-to-real-life Hadoop cluster. This will
allow users to benchmark the Hadoop cluster while it is running. 

This message was sent by Atlassian JIRA

View raw message