hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6168) Old MR client is still broken when receiving new counters from MR job
Date Tue, 25 Nov 2014 17:17:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224857#comment-14224857
] 

Karthik Kambatla commented on MAPREDUCE-6168:
---------------------------------------------

I would call the issue fixed in 2.x only if 2.2 client is able to talk to 2.x server and viceversa.
This doesn't seem to be the case for 2.6. Let me know if I am missing something here. 

> Old MR client is still broken when receiving new counters from MR job
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6168
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6168
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Zhijie Shen
>            Assignee: Junping Du
>            Priority: Blocker
>
> In the following scenarios:
> 1. Either insecure or secure;
> 2. MR 2.2 with new shuffle on NM;
> 3. Submitting via old client.
> We will see the following console exception:
> {code}
> 14/11/17 14:56:19 INFO mapreduce.Job: Job job_1416264695865_0003 completed successfully
> java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_REDUCES
>         at java.lang.Enum.valueOf(Enum.java:236)
>         at org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
>         at org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
>         at org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
>         at org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
>         at org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
>         at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
>         at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
>         at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>         at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
>         at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>         at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
>         at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
>         at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {code}
> The problem is supposed to be fixed by MAPREDUCE-5831, however, it seems that we haven't
cover all the problematic code path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message