hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefan Groschupf (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-114) store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc
Date Mon, 25 Feb 2008 22:02:51 GMT

    [ https://issues.apache.org/jira/browse/PIG-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572264#action_12572264
] 

Stefan Groschupf commented on PIG-114:
--------------------------------------

Hi Pi, 
from my point of view 2. would be the cleanest and best solution. We should add a interface
named something like ReversibleStorage (please give me a better name). This interface extends
from Load and StoreFunc - java doc clearly mentioned that this interface implementation has
to guarantee  ruly reversibility. 
Then we change all internal store and load function to implement this interface. Finally we
check during materialization if we can reuse output based if this interface is implemented.

If people agree I would be happy to work on a patch asap - since this is kind of a blocker
issue for our project.

Thoughts?

> store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc
> ---------------------------------------------------------------------------------
>
>                 Key: PIG-114
>                 URL: https://issues.apache.org/jira/browse/PIG-114
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>            Reporter: Johannes Zillmann
>         Attachments: pigPatch-storeTwice-620665.patch
>
>
> Calling PigServer#store() twice for an alias results in following exception :
> {noformat}
> java.lang.RuntimeException: java.lang.ClassCastException: org.apache.pig.test.DummyStoreFunc
cannot be cast to org.apache.pig.LoadFunc
> 	at org.apache.pig.backend.local.executionengine.POLoad.<init>(POLoad.java:59)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:167)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:184)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:184)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:111)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:90)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:1)
> 	at org.apache.pig.PigServer.store(PigServer.java:330)
> 	at org.apache.pig.PigServer.store(PigServer.java:317)
> 	at org.apache.pig.test.StoreTwiceTest.testIt(StoreTwiceTest.java:31)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:589)
> 	at junit.framework.TestCase.runTest(TestCase.java:164)
> 	at junit.framework.TestCase.runBare(TestCase.java:130)
> 	at junit.framework.TestResult$1.protect(TestResult.java:110)
> 	at junit.framework.TestResult.runProtected(TestResult.java:128)
> 	at junit.framework.TestResult.run(TestResult.java:113)
> 	at junit.framework.TestCase.run(TestCase.java:120)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:228)
> 	at junit.framework.TestSuite.run(TestSuite.java:223)
> 	at org.junit.internal.runners.OldTestClassRunner.run(OldTestClassRunner.java:35)
> 	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:45)
> 	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.ClassCastException: org.apache.pig.test.DummyStoreFunc cannot be
cast to org.apache.pig.LoadFunc
> 	at org.apache.pig.backend.local.executionengine.POLoad.<init>(POLoad.java:57)
> 	... 28 more
> {noformat}
> I will attach a patch with a test scenario for this. Basically the code is as follow:
> {noformat}PigServer pig = new PigServer(ExecType.LOCAL);
>         pig
>                 .registerQuery("A = LOAD 'test/org/apache/pig/test/StoreTwiceTest.java'
USING "
>                         + DummyLoadFunc.class.getName() + "();");
>         pig.registerQuery("B = FOREACH A GENERATE * ;");
>         File outputFile = new File("/tmp/testPigOutput");
>         outputFile.delete();
>         pig.store("A", outputFile.getAbsolutePath(), DummyStoreFunc.class
>                 .getName()
>                 + "()");
>         outputFile.delete();
>         pig.store("B", outputFile.getAbsolutePath(), DummyStoreFunc.class
>                 .getName()
>                 + "()");
>         outputFile.delete();
>         assertEquals(2, _storedTuples.size());
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message