hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pi Song (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-114) store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc
Date Fri, 07 Mar 2008 22:09:46 GMT

    [ https://issues.apache.org/jira/browse/PIG-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12576399#action_12576399
] 

Pi Song commented on PIG-114:
-----------------------------

That's right. This implementation doesn't rely on marker interface concept.

However you still cannot make it 100%. There's still risk if someone implements ReversibleLoadStoreFunc
without really having "LoadFunc (StoreFunc( x )) = x". Therefore what we can do is just enforce
the contract in documentation. In order to do optimization based on UDF, I think there is
no 100% safe. (For example, in the current LoadFunc, you intention is to have the class reading
data from filename supplied in bind() right? but people can still read from somewhere else.
Someone might just open /etc/passwd and send it out )

> store one alias/logicalPlan twice leads to instantiation of StoreFunc as LoadFunc
> ---------------------------------------------------------------------------------
>
>                 Key: PIG-114
>                 URL: https://issues.apache.org/jira/browse/PIG-114
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.0.0
>            Reporter: Johannes Zillmann
>            Assignee: Pi Song
>             Fix For: 0.1.0
>
>         Attachments: PIG114_FixOptimize1.patch, PIG114_FixOptimize_Sample.patch, pigPatch-storeTwice-620665.patch
>
>
> Calling PigServer#store() twice for an alias results in following exception :
> {noformat}
> java.lang.RuntimeException: java.lang.ClassCastException: org.apache.pig.test.DummyStoreFunc
cannot be cast to org.apache.pig.LoadFunc
> 	at org.apache.pig.backend.local.executionengine.POLoad.<init>(POLoad.java:59)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:167)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:184)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.doCompile(LocalExecutionEngine.java:184)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:111)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:90)
> 	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.compile(LocalExecutionEngine.java:1)
> 	at org.apache.pig.PigServer.store(PigServer.java:330)
> 	at org.apache.pig.PigServer.store(PigServer.java:317)
> 	at org.apache.pig.test.StoreTwiceTest.testIt(StoreTwiceTest.java:31)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:589)
> 	at junit.framework.TestCase.runTest(TestCase.java:164)
> 	at junit.framework.TestCase.runBare(TestCase.java:130)
> 	at junit.framework.TestResult$1.protect(TestResult.java:110)
> 	at junit.framework.TestResult.runProtected(TestResult.java:128)
> 	at junit.framework.TestResult.run(TestResult.java:113)
> 	at junit.framework.TestCase.run(TestCase.java:120)
> 	at junit.framework.TestSuite.runTest(TestSuite.java:228)
> 	at junit.framework.TestSuite.run(TestSuite.java:223)
> 	at org.junit.internal.runners.OldTestClassRunner.run(OldTestClassRunner.java:35)
> 	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:45)
> 	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
> 	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
> Caused by: java.lang.ClassCastException: org.apache.pig.test.DummyStoreFunc cannot be
cast to org.apache.pig.LoadFunc
> 	at org.apache.pig.backend.local.executionengine.POLoad.<init>(POLoad.java:57)
> 	... 28 more
> {noformat}
> I will attach a patch with a test scenario for this. Basically the code is as follow:
> {noformat}PigServer pig = new PigServer(ExecType.LOCAL);
>         pig
>                 .registerQuery("A = LOAD 'test/org/apache/pig/test/StoreTwiceTest.java'
USING "
>                         + DummyLoadFunc.class.getName() + "();");
>         pig.registerQuery("B = FOREACH A GENERATE * ;");
>         File outputFile = new File("/tmp/testPigOutput");
>         outputFile.delete();
>         pig.store("A", outputFile.getAbsolutePath(), DummyStoreFunc.class
>                 .getName()
>                 + "()");
>         outputFile.delete();
>         pig.store("B", outputFile.getAbsolutePath(), DummyStoreFunc.class
>                 .getName()
>                 + "()");
>         outputFile.delete();
>         assertEquals(2, _storedTuples.size());
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message