hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vaibhav negi (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2097) java.lang.NullPointerException in reduce task
Date Wed, 29 Sep 2010 14:55:33 GMT
java.lang.NullPointerException in reduce task
---------------------------------------------

                 Key: MAPREDUCE-2097
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2097
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: task
    Affects Versions: 0.20.2
         Environment: RHEL 5.5 
            Reporter: vaibhav negi


While executing hive query "select count(*) from table " , i got this error message 
Ended Job = job_201009291356_0003 with errors
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

On investigating task tracker logs , i found out

2010-09-29 14:31:46,839 WARN org.apache.hadoop.conf.Configuration: /home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.local.dir;  Ignoring.
2010-09-29 14:31:46,862 WARN org.apache.hadoop.conf.Configuration: /home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.job.tracker;  Ignoring.
2010-09-29 14:31:46,943 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
with processName=SHUFFLE, sessionId=
2010-09-29 14:31:47,133 WARN org.apache.hadoop.conf.Configuration: /home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.local.dir;  Ignoring.
2010-09-29 14:31:47,133 WARN org.apache.hadoop.conf.Configuration: /home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: dfs.data.dir;  Ignoring.
2010-09-29 14:31:47,143 WARN org.apache.hadoop.conf.Configuration: /home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: mapred.job.tracker;  Ignoring.
2010-09-29 14:31:47,143 WARN org.apache.hadoop.conf.Configuration: /home/hadoop/mapred/local/taskTracker/jobcache/job_201009291356_0003/attempt_201009291356_0003_r_000000_3/job.xml:a
attempt to override final parameter: dfs.name.dir;  Ignoring.
2010-09-29 14:31:47,209 INFO org.apache.hadoop.mapred.ReduceTask: ShuffleRamManager: MemoryLimit=141387360,
MaxSingleShuffleLimit=35346840
2010-09-29 14:31:47,219 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201009291356_0003_r_000000_3
Thread started: Thread for merging on-disk files
2010-09-29 14:31:47,220 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201009291356_0003_r_000000_3
Thread waiting: Thread for merging on-disk files
2010-09-29 14:31:47,222 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201009291356_0003_r_000000_3
Thread started: Thread for merging in memory files
2010-09-29 14:31:47,223 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201009291356_0003_r_000000_3
Need another 17 map output(s) where 0 is already in progress
2010-09-29 14:31:47,224 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201009291356_0003_r_000000_3
Scheduled 0 outputs (0 slow hosts and0 dup hosts)
2010-09-29 14:31:47,224 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201009291356_0003_r_000000_3
Thread started: Thread for polling Map Completion Events
2010-09-29 14:31:47,233 FATAL org.apache.hadoop.mapred.TaskRunner: attempt_201009291356_0003_r_000000_3
GetMapEventsThread Ignoring exception : java.lang.NullPointerException
	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)



ReduceTask.java:2683 is :  List<MapOutputLocation> loc = mapLocations.get(host)

where host is:-URI u = URI.create(event.getTaskTrackerHttp());  host = u.getHost() ;

What's the fix??

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message