hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2151) [rumen] Add a map of jobconf key-value pairs in LoggedJob
Date Fri, 29 Oct 2010 06:27:37 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926186#action_12926186

Ravi Gummadi commented on MAPREDUCE-2151:

(1) The paths to the new tests seem to be different from existing tests. src/test/org/apache/hadoop/tools/rumen/Test*
instead of src/test/mapred/org/apache/hadoop/tools/rumen/. Is this intentional ?

(2) TestRumenJobTraces has testJobConfigurationParser() and this patch added TestJobConfigurationParser.java.
Can the old testcase be removed/moved from TestRumenJobTraces ?

(3) Are we not targeting to get these interested configuration properties into trace file
? I don't see a way to specify the interested properties regular expression or the prefix
of the interested configuration properties in TraceBuilder. With this patch also, the trace
file doesn't contain any new configuration properties as TraceBuilder.run() gets the interested
properties from JobConfPropertyNames. Should we add an option to TraceBuilder that takes regular
expression and dumps all the config properties that match the regular expression ?

(4) This patch matches only the first part of configuration property(till first "."). Is there
an easy way to make it match more than that ?
Basically, I would like to specify "mapreduce.job." as prefix for my interested configuration
Also it would be cool to be able to specify uninterested properties' prefixes like "mapreduce.jobtracker."
and "mapreduce.tasktracker." to get all other config properties other than those matching
these 2 patterns.

> [rumen] Add a map of jobconf key-value pairs in LoggedJob
> ---------------------------------------------------------
>                 Key: MAPREDUCE-2151
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2151
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: tools/rumen
>            Reporter: Hong Tang
>            Assignee: Hong Tang
>         Attachments: mr-2151-yhadoop-20.201.patch
> It'd be useful to retain application level configuration settings in LoggedJob.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message