pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mcouper99 (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3406) JsonStorage generates NPE after successfully saving records
Date Wed, 04 Dec 2013 14:28:35 GMT

    [ https://issues.apache.org/jira/browse/PIG-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838924#comment-13838924
] 

mcouper99 commented on PIG-3406:
--------------------------------

I'm not sure if this is related, but I ran into the issue this morning.  In looking at the
generated schema, there does appear to be a 'null' name in the output:

{"fields":[{"name":null,"type":15,"description":"autogenerated from Pig Field Schema","schema":null},{"name":"group::valid_events::flight_id","type":15,"description":"autogenerated
from Pig Field Schema","schema":null},{"name":"imps","type":15,"description":"autogenerated
from Pig Field Schema","schema":null},{"name":"clicks","type":15,"description":"autogenerated
from Pig Field Schema","schema":null}],"version":0,"sortKeys":[],"sortKeyOrders":[]}

In this case, I suspect it has something to do with the fact that I'm using a derived field
in a group-by that doesn't have an associated alias:

group events by ((long)epoch/3600000 * 3600, flight_id);


> JsonStorage generates NPE after successfully saving records
> -----------------------------------------------------------
>
>                 Key: PIG-3406
>                 URL: https://issues.apache.org/jira/browse/PIG-3406
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11.1
>         Environment: OSX 10.8.4, running PigServer in local mode from a Groovy script.
>            Reporter: Kirk Stork
>
> Make a new PigServer("local") instance, process some queries and save an alias using
> PigServer piggy = new PigServer("local")
> ... some queries..
> def job = piggy.store("c_observer_id", "obs.json", 'org.apache.pig.builtin.JsonStorage');
> The local directory obs.json is created and populated with correct results.
> but then an NPE is thrown, apparently during some attempt to store the schema.
> HadoopVersion	PigVersion	UserId	StartedAt	FinishedAt	Features
> 1.1.0	0.11.1	kirk	2013-08-01 14:09:41	2013-08-01 14:09:42	GROUP_BY
> Success!
> Job Stats (time in seconds):
> JobId	Alias	Feature	Outputs
> job_local_0001	c_observer_id,g_observer_id,obs	GROUP_BY,COMBINER	obs.json,
> Input(s):
> Successfully read records from: "/Volumes/Work/work/combatxxi-acceptance-testing/pig-input/Ambush_Mine_RPG-7.cxxi/Replication_1_SIMKIT_CONGRUENTIAL/ObserveLogger.log"
> Output(s):
> Successfully stored records in: "obs.json"
> Job DAG:
> job_local_0001
> org.apache.pig.PigException: ERROR 1002: Unable to store alias c_observer_id
> 	at org.apache.pig.PigServer.storeEx(PigServer.java:935)
> 	at org.apache.pig.PigServer.store(PigServer.java:898)
> 	at org.apache.pig.PigServer$store.call(Unknown Source)
> 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
> 	at edu.nps.cxxi.testbench.lff33.ObserveLoggerService.oink(ObserveLoggerService.groovy:27)
> 	at edu.nps.cxxi.testbench.lff33.ObserveLoggerService$oink.call(Unknown Source)
> 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
> 	at edu.nps.cxxi.testbench.lff33.ObserveLoggerServiceTests.testSomething(ObserveLoggerServiceTests.groovy:42)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected
error during execution.
> 	at org.apache.pig.PigServer.launchPlan(PigServer.java:1275)
> 	at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
> 	at org.apache.pig.PigServer.storeEx(PigServer.java:931)
> 	... 41 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.pig.builtin.JsonMetadata.storeSchema(JsonMetadata.java:314)
> 	at org.apache.pig.builtin.JsonStorage.storeSchema(JsonStorage.java:274)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputCommitter.storeCleanup(PigOutputCommitter.java:141)
> 	at org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.storeSchemaForLocal(HadoopShims.java:85)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:427)
> 	at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
> 	... 43 more
> 13/08/01 14:09:42 INFO spring.GrailsWebApplicationContext: Closing org.codehaus.groovy.grails.commons.spring.GrailsWebApplicationContext@784ed19b:
startup date [Thu Aug 01 14:09:39 PDT 2013]; root of context hierarchy
> 13/08/01 14:09:42 INFO support.DefaultListableBeanFactory: Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4d838b:
defining beans [grailsApplication,pluginManager,messageSource,orgGrailsBeansConstraintsEvaluator,grailsProxyHandler,observeLoggerService];
root of factory hierarchy
> If I save to PigStorage, it works fine.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message