ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Veselovsky (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-4813) Ignite map-reduce engine should set MRJobConfig.TASK_ATTEMPT_ID
Date Fri, 10 Mar 2017 11:32:04 GMT
Ivan Veselovsky created IGNITE-4813:
---------------------------------------

             Summary: Ignite map-reduce engine should set MRJobConfig.TASK_ATTEMPT_ID
                 Key: IGNITE-4813
                 URL: https://issues.apache.org/jira/browse/IGNITE-4813
             Project: Ignite
          Issue Type: Bug
          Components: hadoop
    Affects Versions: 1.8
            Reporter: Ivan Veselovsky
            Assignee: Ivan Veselovsky
             Fix For: 2.0


Hadoop "join" example fails on Ignite with the error like this:

{code}
 Out: class org.apache.ignite.IgniteCheckedException: class org.apache.ignite.IgniteCheckedException:
null
[14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2MapTask.run0(HadoopV2MapTask.java:102)
[14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Task.run(HadoopV2Task.java:55)
[14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2TaskContext.run(HadoopV2TaskContext.java:266)
[14:27:29,636][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.runTask(HadoopRunnableTask.java:209)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call0(HadoopRunnableTask.java:144)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask$1.call(HadoopRunnableTask.java:116)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask$1.call(HadoopRunnableTask.java:114)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2TaskContext.runAsJobOwner(HadoopV2TaskContext.java:573)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call(HadoopRunnableTask.java:114)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopRunnableTask.call(HadoopRunnableTask.java:46)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService$2.body(HadoopExecutorService.java:186)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
[14:27:29,637][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at java.lang.Thread.run(Thread.java:745)
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: Caused by: java.lang.NullPointerException
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.<init>(TaskAttemptContextImpl.java:49)
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.hadoop.mapreduce.lib.join.Parser$WNode.createRecordReader(Parser.java:348)
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.hadoop.mapreduce.lib.join.Parser$CNode.createRecordReader(Parser.java:486)
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.hadoop.mapreduce.lib.join.CompositeInputFormat.createRecordReader(CompositeInputFormat.java:143)
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	at org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2MapTask.run0(HadoopV2MapTask.java:69)
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 	... 12 more
[14:27:29,638][INFO ][Thread-3][jvm-a6fc1c46] PID-31907  Out: 
{code}

This is because org.apache.ignite.internal.processors.hadoop.impl.v2.HadoopV2Context sets
the job id and task id, but does not set task attempt id. In Hadoop this is done in method
org.apache.hadoop.mapred.Task#localizeConfiguration .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message