hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ray Chiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6057) Remove obsolete entries from mapred-default.xml
Date Mon, 01 Sep 2014 03:36:21 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117003#comment-14117003
] 

Ray Chiang commented on MAPREDUCE-6057:
---------------------------------------

I did a more thorough searching through the files and I've found the following references
(outside of mapred-default.xml):

1) mapreduce.map.skip.proc.count.autoincr
    mapreduce.reduce.skip.proc.count.autoincr

    found in 

    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
    hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json

2) map.sort.class

    found in

    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
    hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json

3) mapreduce.local.clientfactory.class.name

    found in

    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/resources/job_1329348432655_0001_conf.xml
    hadoop-tools/hadoop-sls/src/main/data/2jobs2min-rumen-jh.json

If anyone has any comments about the uses for these files or suggestion for modifications,
it would be much appreciated.

> Remove obsolete entries from mapred-default.xml
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6057
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6057
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Ray Chiang
>            Assignee: Ray Chiang
>            Priority: Minor
>              Labels: newbie
>
> The following properties are defined in mapred-default.xml but no longer exist in MRJobConfig.
>   map.sort.class
>   mapred.child.env
>   mapred.child.java.opts
>   mapreduce.app-submission.cross-platform
>   mapreduce.client.completion.pollinterval
>   mapreduce.client.output.filter
>   mapreduce.client.progressmonitor.pollinterval
>   mapreduce.client.submit.file.replication
>   mapreduce.cluster.acls.enabled
>   mapreduce.cluster.local.dir
>   mapreduce.framework.name
>   mapreduce.ifile.readahead
>   mapreduce.ifile.readahead.bytes
>   mapreduce.input.fileinputformat.list-status.num-threads
>   mapreduce.input.fileinputformat.split.minsize
>   mapreduce.input.lineinputformat.linespermap
>   mapreduce.job.counters.limit
>   mapreduce.job.max.split.locations
>   mapreduce.job.reduce.shuffle.consumer.plugin.class
>   mapreduce.jobhistory.address
>   mapreduce.jobhistory.admin.acl
>   mapreduce.jobhistory.admin.address
>   mapreduce.jobhistory.cleaner.enable
>   mapreduce.jobhistory.cleaner.interval-ms
>   mapreduce.jobhistory.client.thread-count
>   mapreduce.jobhistory.datestring.cache.size
>   mapreduce.jobhistory.done-dir
>   mapreduce.jobhistory.http.policy
>   mapreduce.jobhistory.intermediate-done-dir
>   mapreduce.jobhistory.joblist.cache.size
>   mapreduce.jobhistory.keytab
>   mapreduce.jobhistory.loadedjobs.cache.size
>   mapreduce.jobhistory.max-age-ms
>   mapreduce.jobhistory.minicluster.fixed.ports
>   mapreduce.jobhistory.move.interval-ms
>   mapreduce.jobhistory.move.thread-count
>   mapreduce.jobhistory.principal
>   mapreduce.jobhistory.recovery.enable
>   mapreduce.jobhistory.recovery.store.class
>   mapreduce.jobhistory.recovery.store.fs.uri
>   mapreduce.jobhistory.store.class
>   mapreduce.jobhistory.webapp.address
>   mapreduce.local.clientfactory.class.name
>   mapreduce.map.skip.proc.count.autoincr
>   mapreduce.output.fileoutputformat.compress
>   mapreduce.output.fileoutputformat.compress.codec
>   mapreduce.output.fileoutputformat.compress.type
>   mapreduce.reduce.skip.proc.count.autoincr
>   mapreduce.shuffle.connection-keep-alive.enable
>   mapreduce.shuffle.connection-keep-alive.timeout
>   mapreduce.shuffle.max.connections
>   mapreduce.shuffle.max.threads
>   mapreduce.shuffle.port
>   mapreduce.shuffle.ssl.enabled
>   mapreduce.shuffle.ssl.file.buffer.size
>   mapreduce.shuffle.transfer.buffer.size
>   mapreduce.shuffle.transferTo.allowed
>   yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts
> Submitting bug for comment/feedback about which properties should be kept in mapred-default.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message