hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sam liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6204) TestJobCounters should use new properties instead JobConf.MAPRED_TASK_JAVA_OPTS
Date Wed, 24 Dec 2014 07:53:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258086#comment-14258086
] 

sam liu commented on MAPREDUCE-6204:
------------------------------------

Hi Tsuyoshi,

I did more tests on PPC64, it has some difference from x86:
1. Without any change, the map/reduce tasks will print:
1.1 on x86:
MAPRED_TASK_JAVA_OPTS=-Xms32m -Xmx1G
MAPRED_MAP_TASK_JAVA_OPTS==null
MAPRED_REDUCE_TASK_JAVA_OPTS=null

1.2 on PPC64:
MAPRED_TASK_JAVA_OPTS=-Xms32m -Xmx1G
MAPRED_MAP_TASK_JAVA_OPTS=-Xmx1000m -Xms1000m -Xmn100m -Xtune:virtualized -Xshareclasses:name=mrscc_%g,groupAccess,cacheDir=/var/hadoop/tmp,nonFatal
-Xscmx20m -Xdump:java:file=/var/hadoop/tmp/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt -Xdump:heap:file=/var/hadoop/tmp/heapdump.%Y%m%d.%H%M%S.%pid.%seq.phd
MAPRED_REDUCE_TASK_JAVA_OPTS=-Xmx1000m -Xms1000m -Xmn100m -Xtune:virtualized -Xshareclasses:name=mrscc_%g,groupAccess,cacheDir=/var/hadoop/tmp,nonFatal
-Xscmx20m -Xdump:java:file=/var/hadoop/tmp/javacore.%Y%m%d.%H%M%S.%pid.%seq.txt -Xdump:heap:file=/var/hadoop/tmp/heapdump.%Y%m%d.%H%M%S.%pid.%seq.phd

2. With this patch, both x86 and PPC64 will print same message:
MAPRED_TASK_JAVA_OPTS=-Xmx200m
MAPRED_MAP_TASK_JAVA_OPTS=-Xms32m -Xmx1G
MAPRED_REDUCE_TASK_JAVA_OPTS=-Xms32m -Xmx1G

I will look into PPC64 part to see if there is any other issue there.

I also opened another jira MAPREDUCE-6205 to update MAPRED_MAP_TASK_JAVA_OPTS and MAPRED_REDUCE_TASK_JAVA_OPTS
with MAPRED_TASK_JAVA_OPTS together, but the patch brought another new ut failure. I also
will look into that jira later.

So this ut really need this patch on PPC64 now and as you said we should use new properties
instead of deprecated one.

Thanks!

> TestJobCounters should use new properties instead JobConf.MAPRED_TASK_JAVA_OPTS
> -------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6204
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6204
>             Project: Hadoop Map/Reduce
>          Issue Type: Test
>          Components: test
>    Affects Versions: 2.6.0
>            Reporter: sam liu
>            Assignee: sam liu
>            Priority: Minor
>         Attachments: MAPREDUCE-6204-1.patch, MAPREDUCE-6204.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message