hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-114) store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc
Date Tue, 08 Apr 2008 17:53:24 GMT

    [ https://issues.apache.org/jira/browse/PIG-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586907#action_12586907
] 

Alan Gates commented on PIG-114:
--------------------------------

Sorry for the way long delay, but I have a question on this code.  If I understand it correctly,
it forces all load and store functions to be reversible since the engine currently (wrongly)
makes that assumption.  Is that correct?  If so, that doesn't seem good.  We have loader functions
that users are using that are not store functions.  We can't break them now just because internally
we try to do the wrong thing.  We should fix the underlying problem.  I'm not saying the Reversible
interface is a bad idea, because that can allow the system to optimize better.  But we can't
force all load functions to be reversible immediately.

> store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc
> ---------------------------------------------------------------------------------
>
>                 Key: PIG-114
>                 URL: https://issues.apache.org/jira/browse/PIG-114
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.0.0
>            Reporter: Johannes Zillmann
>            Assignee: Pi Song
>             Fix For: 0.1.0
>
>         Attachments: PIG114_FixOptimize1.patch, PIG114_FixOptimize_Sample.patch, pigPatch-storeTwice-620665.patch
>
>
> Calling PigServer#store() twice for an alias results in following exception :
> {noformat}
> java.lang.RuntimeException: java.lang.ClassCastException: org.apache.pig.test.DummyStoreFunc
cannot be cast to org.apache.pig.LoadFunc
> 	at org.apache.pig.backend.local.executionengine.POLoad.<init>(POLoad.java:59)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:167)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:184)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:184)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:111)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:90)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:1)
> 	at org.apache.pig.PigServer.store(PigServer.java:330)
> 	at org.apache.pig.PigServer.store(PigServer.java:317)
> 	at org.apache.pig.test.StoreTwiceTest.testIt(StoreTwiceTest.java:31)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:589)
> 	at junit.framework.TestCase.runTest(TestCase.java:164)
> 	at junit.framework.TestCase.runBare(TestCase.java:130)
> 	at junit.framework.TestResult$1.protect(TestResult.java:110)
> 	at junit.framework.TestResult.runProtected(TestResult.java:128)
> 	at junit.framework.TestResult.run(TestResult.java:113)
> 	at junit.framework.TestCase.run(TestCase.java:120)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:228)
> 	at junit.framework.TestSuite.run(TestSuite.java:223)
> 	at org.junit.internal.runners.OldTestClassRunner.run(OldTestClassRunner.java:35)
> 	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:45)
> 	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.ClassCastException: org.apache.pig.test.DummyStoreFunc cannot be
cast to org.apache.pig.LoadFunc
> 	at org.apache.pig.backend.local.executionengine.POLoad.<init>(POLoad.java:57)
> 	... 28 more
> {noformat}
> I will attach a patch with a test scenario for this. Basically the code is as follow:
> {noformat}PigServer pig = new PigServer(ExecType.LOCAL);
>         pig
>                 .registerQuery("A = LOAD 'test/org/apache/pig/test/StoreTwiceTest.java'
USING "
>                         + DummyLoadFunc.class.getName() + "();");
>         pig.registerQuery("B = FOREACH A GENERATE * ;");
>         File outputFile = new File("/tmp/testPigOutput");
>         outputFile.delete();
>         pig.store("A", outputFile.getAbsolutePath(), DummyStoreFunc.class
>                 .getName()
>                 + "()");
>         outputFile.delete();
>         pig.store("B", outputFile.getAbsolutePath(), DummyStoreFunc.class
>                 .getName()
>                 + "()");
>         outputFile.delete();
>         assertEquals(2, _storedTuples.size());
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message