hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6057) Remove obsolete entries from mapred-default.xml
Date Thu, 28 Aug 2014 13:14:09 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113733#comment-14113733
] 

Jason Lowe commented on MAPREDUCE-6057:
---------------------------------------

Please be careful with this, as the entries in mapred-default.xml serve two valuable purposes:

* Provide default values for cases where the code isn't providing a hardcoded default value
* Documentation of property values and defaults

So we should not remove anything that is currently being used by the code, and not everything
being used is in MRJobConfig.  For example, the mapreduce.jobhistory.* properties are covered
by JHAdminConfig, the shufflehandler properties are covered by ShuffleHandler, etc.

Before removing any of these properties the code base should be searched to ensure they are
truly not being referenced.

> Remove obsolete entries from mapred-default.xml
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6057
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6057
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Ray Chiang
>            Priority: Minor
>              Labels: newbie
>
> The following properties are defined in mapred-default.xml but no longer exist in MRJobConfig.
>   map.sort.class
>   mapred.child.env
>   mapred.child.java.opts
>   mapreduce.app-submission.cross-platform
>   mapreduce.client.completion.pollinterval
>   mapreduce.client.output.filter
>   mapreduce.client.progressmonitor.pollinterval
>   mapreduce.client.submit.file.replication
>   mapreduce.cluster.acls.enabled
>   mapreduce.cluster.local.dir
>   mapreduce.framework.name
>   mapreduce.ifile.readahead
>   mapreduce.ifile.readahead.bytes
>   mapreduce.input.fileinputformat.list-status.num-threads
>   mapreduce.input.fileinputformat.split.minsize
>   mapreduce.input.lineinputformat.linespermap
>   mapreduce.job.counters.limit
>   mapreduce.job.max.split.locations
>   mapreduce.job.reduce.shuffle.consumer.plugin.class
>   mapreduce.jobhistory.address
>   mapreduce.jobhistory.admin.acl
>   mapreduce.jobhistory.admin.address
>   mapreduce.jobhistory.cleaner.enable
>   mapreduce.jobhistory.cleaner.interval-ms
>   mapreduce.jobhistory.client.thread-count
>   mapreduce.jobhistory.datestring.cache.size
>   mapreduce.jobhistory.done-dir
>   mapreduce.jobhistory.http.policy
>   mapreduce.jobhistory.intermediate-done-dir
>   mapreduce.jobhistory.joblist.cache.size
>   mapreduce.jobhistory.keytab
>   mapreduce.jobhistory.loadedjobs.cache.size
>   mapreduce.jobhistory.max-age-ms
>   mapreduce.jobhistory.minicluster.fixed.ports
>   mapreduce.jobhistory.move.interval-ms
>   mapreduce.jobhistory.move.thread-count
>   mapreduce.jobhistory.principal
>   mapreduce.jobhistory.recovery.enable
>   mapreduce.jobhistory.recovery.store.class
>   mapreduce.jobhistory.recovery.store.fs.uri
>   mapreduce.jobhistory.store.class
>   mapreduce.jobhistory.webapp.address
>   mapreduce.local.clientfactory.class.name
>   mapreduce.map.skip.proc.count.autoincr
>   mapreduce.output.fileoutputformat.compress
>   mapreduce.output.fileoutputformat.compress.codec
>   mapreduce.output.fileoutputformat.compress.type
>   mapreduce.reduce.skip.proc.count.autoincr
>   mapreduce.shuffle.connection-keep-alive.enable
>   mapreduce.shuffle.connection-keep-alive.timeout
>   mapreduce.shuffle.max.connections
>   mapreduce.shuffle.max.threads
>   mapreduce.shuffle.port
>   mapreduce.shuffle.ssl.enabled
>   mapreduce.shuffle.ssl.file.buffer.size
>   mapreduce.shuffle.transfer.buffer.size
>   mapreduce.shuffle.transferTo.allowed
>   yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts
> Submitting bug for comment/feedback about which properties should be kept in mapred-default.xml.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message