asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Westmann" <ti...@apache.org>
Subject Re: External function dependency problem
Date Mon, 04 Jul 2016 06:37:37 GMT
Sounds good to me!

Cheers,
Till

On 3 Jul 2016, at 23:34, Michael Blow wrote:

> I think a better solution might be to invert the normal class loader
> resolution order with an extension of URLClassLoader that overrides
> loadClass(), which tries resolving classes internally before 
> delegating to
> the parent. Boot classpath classes can still be forced to load 
> correctly. I
> think this will solve the use case without the regression due to 
> inability
> to resolve *db classes.
>
> Thanks,
>
> -MDB
>
> On Sat, Jul 2, 2016 at 11:44 PM Xikui Wang <xikuiw@uci.edu> wrote:
>
>> The error log is as follow:
>>
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>> at
>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>> at
>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>> at
>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>> ... 8 more
>> Caused by: java.lang.ClassCastException:
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>> org.apache.asterix.external.api.IFunctionFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>> ... 12 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>> at
>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>> at
>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>> at
>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>> ... 8 more
>> Caused by: java.lang.ClassCastException:
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>> org.apache.asterix.external.api.IFunctionFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>> ... 12 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>> ... 4 more
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>> at
>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>> at
>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>> ... 5 more
>> Caused by:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>> at
>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>> ... 8 more
>> Caused by: java.lang.ClassCastException:
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>> org.apache.asterix.external.api.IFunctionFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>> ... 12 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341)
>> ... 4 more
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143)
>> at
>>
>> org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39)
>> at
>>
>> org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127)
>> ... 5 more
>> Caused by:
>> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException:
>>  Unable to load/instantiate class
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53)
>> at
>>
>> org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39)
>> at
>>
>> org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41)
>> at
>>
>> org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109)
>> ... 8 more
>> Caused by: java.lang.ClassCastException:
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to
>> org.apache.asterix.external.api.IFunctionFactory
>> at
>>
>> org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72)
>> ... 12 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>> ... 4 more
>> Caused by: java.lang.InterruptedException
>> at java.lang.Object.wait(Native Method)
>> at java.lang.Object.wait(Object.java:502)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>> ... 6 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>> ... 4 more
>> Caused by: java.lang.InterruptedException
>> at java.lang.Object.wait(Native Method)
>> at java.lang.Object.wait(Object.java:502)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>> ... 6 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>> ... 4 more
>> Caused by: java.lang.InterruptedException
>> at java.lang.Object.wait(Native Method)
>> at java.lang.Object.wait(Object.java:502)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>> ... 6 more
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
>> at org.apache.hyracks.control.nc.Task.run(Task.java:297)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException:
>> java.lang.InterruptedException
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43)
>> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344)
>> ... 4 more
>> Caused by: java.lang.InterruptedException
>> at java.lang.Object.wait(Native Method)
>> at java.lang.Object.wait(Object.java:502)
>> at
>>
>> org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124)
>> ... 6 more
>> testFile
>>
>> src/test/resources/runtimets/queries/external-library/tweetGeoTag/tweetGeoTag.3.update.aql
>> raised an exception:
>> java.lang.Exception: HTTP operation failed: 2
>> STATUS LINE: HTTP/1.1 500 Server Error
>> SUMMARY: Error: Failure in feed
>>
>> STACKTRACE: org.apache.asterix.common.exceptions.AsterixException: 
>> Failure
>> in feed
>> at
>>
>> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertNoFailure(FeedLifecycleEventSubscriber.java:62)
>> at
>>
>> org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertEvent(FeedLifecycleEventSubscriber.java:55)
>> at
>>
>> org.apache.asterix.aql.translator.QueryTranslator.handleConnectFeedStatement(QueryTranslator.java:2238)
>> at
>>
>> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:366)
>> at
>>
>> org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:253)
>> at
>>
>> org.apache.asterix.api.http.servlet.RESTAPIServlet.handleRequest(RESTAPIServlet.java:195)
>> at
>>
>> org.apache.asterix.api.http.servlet.UpdateAPIServlet.handleRequest(UpdateAPIServlet.java:30)
>> at
>>
>> org.apache.asterix.api.http.servlet.RESTAPIServlet.doPost(RESTAPIServlet.java:162)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>> at 
>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
>> at
>>
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:228)
>> at
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:956)
>> at
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
>> at
>>
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:188)
>> at
>>
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:891)
>> at
>>
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
>> at
>>
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)
>> at org.eclipse.jetty.server.Server.handle(Server.java:353)
>> at
>>
>> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:598)
>> at
>>
>> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1076)
>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:805)
>> at 
>> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218)
>> at 
>> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:427)
>> at
>>
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:510)
>> at
>>
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)
>> at
>>
>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
>> at
>>
>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> at
>>
>> org.apache.asterix.test.aql.TestExecutor.executeHttpMethod(TestExecutor.java:275)
>> at
>>
>> org.apache.asterix.test.aql.TestExecutor.executeUpdate(TestExecutor.java:331)
>> at
>> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:496)
>> at
>> org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:762)
>> at
>> org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:106)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:497)
>> at
>>
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>> at
>>
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>> at
>>
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>> at
>>
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
>> at
>>
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>> at
>>
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>> at org.junit.runners.Suite.runChild(Suite.java:127)
>> at org.junit.runners.Suite.runChild(Suite.java:26)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>> at
>>
>> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>> at
>> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
>> at
>>
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
>> at
>>
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
>> at
>>
>> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
>> at 
>> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:497)
>> at 
>> com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
>> ...Unexpected!
>>
>> On Sat, Jul 2, 2016 at 7:23 PM, Raman Grover 
>> <ramangrover29@gmail.com>
>> wrote:
>>
>>> i am missing the attachment
>>> On Jul 2, 2016 6:39 PM, "Xikui Wang" <xikuiw@uci.edu> wrote:
>>>
>>>> Hi Raman,
>>>>
>>>> Thanks for your help. I tried this quick fix on my branch, but it
>>>> introduces some new exceptions. I think this causes Asterix fails 
>>>> at
>>>> entering the external function. The error message is attached.
>>>>
>>>> Best,
>>>> Xikui
>>>>
>>>> On Fri, Jul 1, 2016 at 10:11 AM, Raman Grover 
>>>> <ramangrover29@gmail.com
>>>
>>>> wrote:
>>>>
>>>>> Operations related to setting up an external library are contained 
>>>>> in
>>>>> ExternalLibraryUtil
>>>>> <
>>>>>
>>>
>> https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java
>>>>>>
>>>>>
>>>>> At line 382, we have
>>>>> // create and return the class loader
>>>>>
>>>>> ClassLoader classLoader = new URLClassLoader(urls, 
>>>>> parentClassLoader);
>>>>> return classLoader;
>>>>>
>>>>> Above, we have the parentClassLoader set to the classloader for
>>>>> ExternalLibraryUtil which is the application class loader 
>>>>> (AsterixDB's
>>>>> classloader that loads the dependencies from pom.xml). The 
>>>>> proposed
>>>>> solution (a) in earlier thread - skipping application classloader
>> would
>>>>> translate to replacing the above code with
>>>>>
>>>>> ClassLoader classLoader = new URLClassLoader(urls, null);
>>>>>
>>>>> Regards,
>>>>> Raman
>>>>>
>>>>> On Thu, Jun 30, 2016 at 4:41 PM, Xikui Wang <xikuiw@uci.edu> 
>>>>> wrote:
>>>>>
>>>>>>>
>>>>>>> Hi Abdullah,
>>>>>>
>>>>>> I reverted my code to reproduce the problem. Noticing this 
>>>>>> external
>>>>>> function has a couple of other bugs but the dependency one is
>> blocking
>>>>>> others, so this should be enough to reproduce the problem.
>>>>>>
>>>>>>>
>>>>>> The external function package is geoTag.zip.
>>>>>>
>>>>>>> ‚Äč
>>>>>>>
>>>>>>> ‚ÄčTest scripts are in tweetGeoTag.zip
>>>>>>
>>>>>> External function is loading data from data/, i.e.: 
>>>>>> data/state.json
>> .
>>> So
>>>>>> all json files in data.zip need to be placed under
>>>>>> ../asterixdb/asterix-app/data/
>>>>>>
>>>>>> The real_tweets_adm.adm used in ddl is also attached.
>>>>>>
>>>>>> This setting will cause
>>>>>>
>>>>>> java.lang.NoSuchMethodError:
>>>>>>> com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z
>>>>>>> at
>>>>>>>
>>>>>
>>> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:541)
>>>>>>> at
>>>>>>>
>>>>>
>>> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:452)
>>>>>>> at
>>> org.wololo.geojson.GeoJSONFactory.<clinit>(GeoJSONFactory.java:17)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoJSONIndex.loadShape(IGeoIndex.scala:29)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$.loadShape(USGeoGnosis.scala:101)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:20)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:18)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
>>>>>>> at scala.collection.immutable.List.foreach(List.scala:381)
>>>>>>> at
>>>>> scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
>>>>>>> at scala.collection.immutable.List.map(List.scala:285)
>>>>>>> at
>>>>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.load(USGeoGnosis.scala:18)
>>>>>>> at
>>>>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.<init>(USGeoGnosis.scala:15)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> edu.uci.ics.cloudberry.noah.feed.GeoTagFunction.initialize(GeoTagFunction.java:34)
>>>>>>> at
>>>>>>>
>>>>>>
>>>>>
>>>
>> org.apache.asterix.external.library.ExternalFunction.initialize(ExternalFunction.java:113)
>>>>>>>
>>>>>>
>>>>>> The attachments exceed the size limits. I moved them under this
>> google
>>>>>> drive directory:
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>> https://drive.google.com/folderview?id=0B_6Dzy3OTjaNRUVJWTRvWEtKSU0&usp=sharing
>>>>>>
>>>>>> Let me know if you need anything else. Thanks.
>>>>>>
>>>>>> Best,
>>>>>> Xikui
>>>>>>
>>>>>> On Thu, Jun 30, 2016 at 12:06 PM, abdullah alamoudi <
>>> bamousaa@gmail.com
>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> I will look into that.
>>>>>>>>
>>>>>>>> @Xikui,
>>>>>>>> Do you have a reproducable use case where such a behavior
can 
>>>>>>>> be
>>>>>> observed?
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Abdullah.
>>>>>>>>
>>>>>>>> On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover <
>>>>> ramangrover29@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Each external library has an separately associated custom

>>>>>>>>> class
>>>>> loader
>>>>>>>>> (URLClassLoader) that loads the contained functions and
any
>>>>>> dependencies
>>>>>>>>> (jar files) packaged inside the library. The custom loader

>>>>>>>>> fits
>>>>> into
>>>>>> the
>>>>>>>>> natural hierarchy with system (classpath based) classloader
as
>>> the
>>>>>>>>> immediate parent and bootstrap classloader as the root.
As the
>>>>>>>> delegation
>>>>>>>>> model works, a class loader attempts to load a class
only 
>>>>>>>>> after
>>>>>>>> parent(s)
>>>>>>>>> in the lineage have failed.
>>>>>>>>>
>>>>>>>>> In your specific case, the system class loader is able
to find
>>> the
>>>>>>>> jackson
>>>>>>>>> library from the configured classpath and so the library
>>>>> classloader
>>>>>> is
>>>>>>>> not
>>>>>>>>> involved at all for this jar. The problem arising due
to
>>> different
>>>>>>>> versions
>>>>>>>>> of a jar present across the hierarchy of classloaders
is
>> similar
>>> to
>>>>>> one
>>>>>>>>> faced in web containers such as Tomcat or application
servers
>>> such
>>>>> as
>>>>>>>>> Websphere where system loaded classes can override any
>>> application
>>>>>>>> specific
>>>>>>>>> classes due to name conflict.
>>>>>>>>>
>>>>>>>>> The way you are defining a library is correct. The fix
needs 
>>>>>>>>> to
>>> be
>>>>> in
>>>>>>>> the
>>>>>>>>> way the classes are loaded by AsterixDB (here acting
as an
>>>>> application
>>>>>>>>> server).
>>>>>>>>>
>>>>>>>>> I see two ways:-
>>>>>>>>>
>>>>>>>>> a) Skipping the system class loader: AsterixDB uses a
>>>>> URLClassloader
>>>>>> to
>>>>>>>>> load libraries. Setting the  parent of the this classloader
as
>>> null
>>>>>>>> implies
>>>>>>>>> the bootrstrap classloader (root) as the immediate parent.
The
>>>>>> bootstrap
>>>>>>>>> classloader loads rt.jar (java runtime classes) and i18n.jar
>> and
>>>>> these
>>>>>>>>> definitely do not have conflicts with your library 
>>>>>>>>> dependencies
>>> (if
>>>>>> they
>>>>>>>>> do, not sure why one would override the classes in rt.jar
and
>> so
>>> we
>>>>>>>> need to
>>>>>>>>> fix the library). Note that the system classloader is
out of
>> the
>>>>>>>> hierarchy
>>>>>>>>> here and so any other jars loaded by AsterixDB from its
>> classpath
>>>>>>>> (pom.xml)
>>>>>>>>> would not be visible.
>>>>>>>>>
>>>>>>>>> b) Defining a Custom Class loader: Inside AsterixDB,
we
>> provide a
>>>>>> custom
>>>>>>>>> implementation of classloader that attempts to load a
class
>> prior
>>>>> to
>>>>>>>>> delegating to the parent. This way, the library classes
and
>>>>> packaged
>>>>>>>>> dependencies override any system level classes from the
class
>>> path
>>>>> and
>>>>>>>> even
>>>>>>>>> the classes contained in rt.jar and i18n.jar.
>>>>>>>>>
>>>>>>>>> I am opening the discussion here to suggest further
>> alternatives
>>> or
>>>>>>>> provide
>>>>>>>>> preferences.
>>>>>>>>>
>>>>>>>>> I have a preference for (a)  (skipping the system class

>>>>>>>>> loader)
>>> for
>>>>>> two
>>>>>>>>> reasons:
>>>>>>>>>
>>>>>>>>> a) it is simpler
>>>>>>>>>
>>>>>>>>> b) the other option allows a custom class loader to override
>>>>> classes
>>>>>> in
>>>>>>>>> rt.jar., which is ok but not how classloaders are supposed
to
>>> work
>>>>> in
>>>>>>>>> principle.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Raman
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jun 29, 2016 11:07 PM, "Mike Carey" <dtabass@gmail.com>
>>> wrote:
>>>>>>>>>
>>>>>>>>>> Any classloader experts have suggestions...?
>>>>>>>>>> On Jun 29, 2016 10:26 PM, "Xikui Wang" <xikuiw@uci.edu>
>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Devs,
>>>>>>>>>>>
>>>>>>>>>>> We found a problem when trying to build external
functions
>>> for
>>>>> the
>>>>>>>>>>> cloudberry demo.
>>>>>>>>>>>
>>>>>>>>>>> When the external function is depend on certain
library,
>> the
>>>>>> library
>>>>>>>>> that
>>>>>>>>>>> comes with the external function will be blocked
by same
>>>>> library
>>>>>> in
>>>>>>>>>>> AsterixDB. In our case, our external function
'geoTag' uses
>>>>>> jackson
>>>>>>>>>> v2.7.1,
>>>>>>>>>>> and we packed all dependencies into one single
jar. When
>>>>> running
>>>>>>>>> 'geoTag'
>>>>>>>>>>> on Asterix, it will call jackson v2.0.0 in AsterixDB
which
>>>>> causes
>>>>>>>>>>> NullPointerException. We have to manually change
pom.xml in
>>>>>>>> AsterixDB
>>>>>>>>> to
>>>>>>>>>>> fix that.
>>>>>>>>>>>
>>>>>>>>>>> We are wondering is that because we load the
external
>>> function
>>>>> in
>>>>>> a
>>>>>>>>> wrong
>>>>>>>>>>> way, or this could be one possible interesting
problem
>> which
>>> is
>>>>>>>> worth
>>>>>>>>>>> noticing. Thanks.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Xikui
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Raman
>>>>>
>>>>
>>>>
>>>
>>

Mime
View raw message