hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3770) improve composition, submission and result collection of gridmix
Date Thu, 13 Nov 2008 22:21:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Douglas updated HADOOP-3770:

    Status: Open  (was: Patch Available)

I haven't been over the details, but had a few general suggestions after a first pass:
* Most of this doesn't conform to the coding guidelines. Converting tabs to spaces, removing
commented-out code, putting constants and defaults in a reasonable place, etc. should be done
before this can be committed.
* Temporary directories should be configurable and default off of a single, configurable temp
dir rather than being hard-coded off /tmp
* If independent from the original, the configuration and drivers of gridmix2 should not be
in the same place. If this is intended as a replacement for gridmix, it should modify the
existing benchmark rather than creating _file_, _file2_ pairs. If it's a new benchmark, it
should be in src/benchmarks/gridmix2.
* Would it be possible to split the pig benchmarks into a separate JIRA? This is simply too
large to review well.
* GridMixRunner is unnecessarily enormous. Most of the methods are setting defaults and performing
work best encapsulated in the \*Creator classes that currently do trivial work. This class
would also benefit from utility methods converting results from Configuration::getStrings
to int[] (instead of subclassing Configuration), abstracting out the creation of unique Strings
for runs (the use of Calendar/Date may not be the correct choice), javadoc, and general cleanup
* Exceptions are almost always ignored; most probably should not be.

> improve composition, submission and result collection of gridmix
> ----------------------------------------------------------------
>                 Key: HADOOP-3770
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3770
>             Project: Hadoop Core
>          Issue Type: Improvement
>            Reporter: Lingyun Yang
>            Assignee: Runping Qi
>         Attachments: patch-3770.txt, patch-3770.txt, patch-3770.v2.txt
> Current gridmix submits jobs using a set of scripts, which is inconvenient and the results
are difficult to collect.  To improve the gridmix submission and results collection, we implemented
a new program  using JobControl to submit and collect the results of jobs 
> Also the new gridmix allows to have more different types of jobs such as, pig jobs, jobs
with combiners etc. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message