pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sekhon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4418) NullPointerException in JVMReuseImpl
Date Fri, 22 May 2015 10:56:17 GMT

    [ https://issues.apache.org/jira/browse/PIG-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555970#comment-14555970
] 

Hari Sekhon commented on PIG-4418:
----------------------------------

Jobs failures were intermittent and seemed non-deterministic so I tried running re-running
these jobs on mapreduce backend instead of tez and so far it's been much more reliable successfully
indexing billions of docs for many jobs without any more failures with everything else remaining
the same.

It may just be coincidence but I started having these problems around the time of upgrade
from HDP 2.2.0.0 to HDP 2.2.4.2.

I'm actually migrating this pipeline from Pig to Spark so I'm not intending to do any more
testing on this right now given it's working reliably when back on mapreduce.

> NullPointerException in JVMReuseImpl
> ------------------------------------
>
>                 Key: PIG-4418
>                 URL: https://issues.apache.org/jira/browse/PIG-4418
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.14.0
>            Reporter: Jeff Zhang
>            Assignee: Rohini Palaniswamy
>            Priority: Critical
>             Fix For: 0.15.0
>
>         Attachments: PIG-4418-1.patch, PIG-4418-2.patch
>
>
> {code}
> 2015-02-13 15:17:11,067 INFO [TezChild] task.TezTaskRunner: Encounted an error while
executing task: attempt_1423730493153_0019_1_04_000002_0
> java.lang.NullPointerException
> 	at org.apache.pig.JVMReuseImpl.cleanupStaticData(JVMReuseImpl.java:46)
> 	at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.close(PigProcessor.java:175)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.close(LogicalIOProcessorRuntimeTask.java:338)
> 	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:181)
> 	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> 	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:171)
> 	at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:166)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message