hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-471) Verbose (unrelated) exception stack is printed when input file in the Load is not present
Date Mon, 06 Oct 2008 23:24:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pradeep Kamath updated PIG-471:
-------------------------------

    Status: Patch Available  (was: Open)

Attached patch for the issue. The problem was that when the file is missing, the job submission
fails in getSplit() even BEFORE map or reduce tasks are ever run. In MapReduceLauncher after
we come to know the job has failed, we try to get mapTaskReport and reduceTaskReport (in Launcher.getStats())
from Hadoop which results in an IOexception. The fix catches this and gives a warning to the
user IF the job had succeeded

> Verbose (unrelated) exception stack is printed when input file in the Load is not present
> -----------------------------------------------------------------------------------------
>
>                 Key: PIG-471
>                 URL: https://issues.apache.org/jira/browse/PIG-471
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>             Fix For: types_branch
>
>
> When input file in load is absent the following is printed on the screen:
> {noformat}
> 2008-10-06 10:24:28,048 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
- java.io.IOException: bla does not exist
>         at org.apache.pig.backend.executionengine.PigSlicer.validate(PigSlicer.java:105)
>         at org.apache.pig.impl.io.ValidatingInputFileSpec.validate(ValidatingInputFileSpec.java:59)
>         at org.apache.pig.impl.io.ValidatingInputFileSpec.<init>(ValidatingInputFileSpec.java:44)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:200)
>         at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
>         at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:370)
>         at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
>         at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
>         at java.lang.Thread.run(Thread.java:595)
> 2008-10-06 10:24:28,061 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException:
Unable to store for alias: 1 [java.io.IOException: java.lang.NullPointerException
>         at java.util.TreeMap.getEntry(TreeMap.java:324)
>         at java.util.TreeMap.get(TreeMap.java:255)
>         at org.apache.hadoop.mapred.JobTracker.getMapTaskReports(JobTracker.java:1894)
>         at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> ]
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:255)
>         at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:647)
>         at org.apache.pig.PigServer.execute(PigServer.java:638)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:278)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:439)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:249)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)
> Caused by: org.apache.pig.backend.executionengine.ExecException: java.io.IOException:
java.lang.NullPointerException
>         at java.util.TreeMap.getEntry(TreeMap.java:324)
>         at java.util.TreeMap.get(TreeMap.java:255)
>         at org.apache.hadoop.mapred.JobTracker.getMapTaskReports(JobTracker.java:1894)
>         at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
>         ... 9 more
> Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException
>         at java.util.TreeMap.getEntry(TreeMap.java:324)
>         at java.util.TreeMap.get(TreeMap.java:255)
>         at org.apache.hadoop.mapred.JobTracker.getMapTaskReports(JobTracker.java:1894)
>         at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
>         at org.apache.hadoop.ipc.Client.call(Client.java:715)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
>         at org.apache.hadoop.mapred.$Proxy1.getMapTaskReports(Unknown Source)
>         at org.apache.hadoop.mapred.JobClient.getMapTaskReports(JobClient.java:935)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:90)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:92)
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:245)
>         ... 8 more
> 2008-10-06 10:24:28,061 [main] ERROR org.apache.pig.tools.grunt.GruntParser - Unable
to store for alias: 1 [java.io.IOException: java.lang.NullPointerException
>         at java.util.TreeMap.getEntry(TreeMap.java:324)
>         at java.util.TreeMap.get(TreeMap.java:255)
> {noformat}
> Of this only the first part which actually tells that the input file is absent is useful,
the rest is misleading and confusing

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message